Visual Localization

A GPS-denied localization system that estimates a drone's position, altitude, and orientation from camera imagery and onboard motion sensors. Developed as part of the Autonomous UAV Tracking thesis, it continues to provide a reliable estimate of the aircraft's pose when GPS is unavailable.

Visual Pipeline

Each camera frame is compared with geo-referenced map tiles. SuperPoint identifies distinctive visual features, while SuperGlue matches them between the live image and the map. Because the tiles have a known real-world scale, these matches can be translated into reference points for estimating the camera's location, orientation, and altitude. A Perspective-n-Point (PnP) solver, combined with RANSAC to reject incorrect matches, produces the final absolute pose estimate.

Sensor Fusion

The camera-based estimates are combined with IMU readings in an Extended Kalman Filter (EKF). Accelerometer and gyroscope data predict the aircraft's motion between frames, while confident visual matches correct that prediction. This keeps the estimate smooth and continuous even when the camera briefly loses a good match.

Integration

Built with OpenCV, the system integrates with a PX4 flight controller and can operate alongside GPS-based navigation. It automatically takes over when GPS quality falls below a configurable threshold. Dynamic resolution scaling keeps the workload manageable when localization runs alongside object detection and tracking.