Computer Vision & AI
Equip robots with sight and spatial awareness. Master camera calibration, 3D point cloud processing, real-time object detection with YOLO and TensorRT, visual odometry, fiducial markers (AprilTags/ArUco), and deep reinforcement learning.
All Guides in Computer Vision & AI
Camera Calibration & Distortion Correction with OpenCV: Pinhole Model to 3D Rays
Step-by-step camera calibration using chessboard grids: compute intrinsic camera matrix K, radial/tangential distortion coefficients (k1, k2, p1, p2), and unproject 2D pixels to 3D spatial rays.
The Intrinsic Matrix K maps 3D camera coordinates to 2D image pixels via focal lengths (fx, fy) and optical center (cx, cy).
AprilTag & ArUco 6-DOF Visual Pose Estimation for Robotic Servoing
Detect fiducial markers (AprilTag 36h11 & ArUco) in real time, solve Perspective-n-Point (PnP) for precise 6-DOF translation and rotation, and execute visual servoing.
AprilTags provide millimeter-accurate 6-DOF relative poses for docking stations, charging pads, and robotic grasp targets.
Real-Time Object Detection with YOLOv8 & ROS 2 Integration
Train custom YOLOv8 models, export optimized ONNX/TensorRT engines, and publish 2D/3D bounding boxes and class probabilities over ROS 2 topics.
YOLOv8 uses an anchor-free split head with task-aligned assigner, achieving higher mean Average Precision (mAP) and faster inference.
RGB-D Depth Cameras: Intel RealSense D435 vs OAK-D & Time-of-Flight
Compare active infrared stereo (RealSense D435i/D455), onboard Myriad X/Keem Bay VPU stereo (OAK-D), and direct Time-of-Flight (ToF) cameras for robotics navigation and 3D reconstruction.
Active IR stereo projects an invisible dot pattern to provide texture for depth calculation in completely dark or featureless rooms.
3D Point Cloud Processing with PCL: Voxel Filtering & RANSAC Plane Extraction
Process raw 3D LiDAR and RGB-D depth clouds using C++ Point Cloud Library (PCL): downsample with VoxelGrid, remove outliers, segment ground/tabletop planes with RANSAC, and cluster objects.
Raw RGB-D point clouds contain 300,000+ points per frame; VoxelGrid downsampling to 1cm voxels slashes computation time by 90% while retaining geometry.
Visual Odometry (VO) with ORB Features & Epipolar Geometry
Estimate camera 6-DOF trajectory from monocular and stereo video frames using Oriented FAST and Rotated BRIEF (ORB) feature matching and Essential Matrix decomposition.
Visual Odometry estimates frame-to-frame incremental camera poses without building a global persistent map or running loop closure.
Edge AI Acceleration on NVIDIA Jetson with TensorRT & DeepStream
Deploy neural network inference on NVIDIA Jetson Orin Nano / AGX: convert PyTorch models to ONNX, build FP16/INT8 TensorRT engines, and achieve sub-10ms latency.
NVIDIA Jetson shares physical LPDDR5 RAM between CPU and GPU; using CUDA Unified Pinned Memory eliminates PCIe transfer overhead.
Deep Reinforcement Learning for Locomotion with NVIDIA Isaac Gym
Train end-to-end neural network locomotion policies for quadruped and humanoid robots in Isaac Gym: massively parallel physics simulation, PPO, domain randomization, and sim-to-real transfer.
NVIDIA Isaac Gym executes thousands of robot physics instances simultaneously on the GPU tensor cores, collecting millions of transition steps per minute.