Detect and Map Objects of Interest from Drone Videos

This project extracts the location of objects of interest from a drone video and plots them on a map. By combining the video with data from its flight log and a computer vision model trained on Roboflow, it demonstrates georeferencing a machine learning model's predictions to GPS coordinates and using them to visualize the location of detected solar panels on a map using Mapbox.

https://user-images.githubusercontent.com/870796/189461690-122f4e64-a66e-40f0-ac4b-68258a8abe7e.mov

Try It in Your Browser

The project is deployed to Github Pages here and you can test it out with this sample video and flight log.

If you have your own Drone video you'd like to use, follow the instructions in the blog post to pull your detailed flight log from Airdata.

Resources

• Accompanying Blog Post: Georeferencing Objects in Drone Videos
• Try the aerial solar panels pre-trained computer vision model in your browser on Roboflow Universe
• Browse other Aerial Imagery Datasets and Pre-Trained Models
• Train Your Own Computer Vision Model to use with this repo

Run It Locally

• Clone this repo
• Run npm install in the main directory
• Run npm run build:dev to start a webpack build with livereload
• Open a new terminal window and run npx serve dist
• Open http://localhost:3000 in your browser

Customize It

This repo can easily be changed to run any custom model trained with Roboflow including the thousands of pre-trained models shared on Roboflow Universe. Simply swap out your publishable_key and the model ID and version in the ROBOFLOW_SETTINGS at the top of main.js.

There are also some additional configuration options available at the top of renderMap.js.

For example, changing the model to swimming-pool-b6pz4 to use this swimming pool computer vision model from Roboflow Universe changes the functionality from plotting solar panels to plotting pools:

https://user-images.githubusercontent.com/870796/190296751-02b46989-7e18-4fcb-93c4-67e492cff401.mp4

Other ideas for how to use this repo:

• Search & Rescue
• Monitoring swimmer safety in triathlons
• Helping governments identify zoning violations
• Monitoring core infrastructure like pipelines for potential hazards like excavators and construction equipment nearby
• Finding people with bonfires in high risk fire areas
• Tracking wildlife
• Mapping oil wells
• Monitoring land use and tree cover
• Finding boats fishing in restricted areas
• Counting cars in parking lots
• Tracking human rights violations
• Other Aerial Datasets & Models to get your gears turning

Getting Your Flight Log

You can get the detailed flight log from a DJI drone using Airdata. The sample video and flight log were taken from a DJI Mavic Air 2. Full details are in the blog post.

Training a Custom Model

If you can't find a pre-trained model that accurately detects your particular object of interest on Roboflow Universe you can create a dataset and train your own custom model using Roboflow.

Roboflow is an end-to-end computer vision platform that has helped over 100,000 developers use computer vision. The easiest way to get started is to sign up for a free Roboflow account and follow our quickstart guide.

Once you've trained a custom model, update your publishable API Key, model ID, and version in the configuration at the top of main.js.

Contributing

Pull requests are welcome to improve this repo. Ideas for improvements that could be made:

• Taking into account changes in the ground elevation & their impact on the distance calculations
• Intelligently choosing the correct part of the flight log based on the duration of isVideo compared to the duration of the loaded video
• Exporting a JSON file of the detected objects
• Adding a CLI for processing outside of a web browser
• Rendering the flight video and predictions into a single image (patching video frames together)
• Video controls (play/pause, scrubbing)
• Option to show the video in a static position vs flying over the flight path
• Add smoothing to the video positioning to account for the flight-log only having a 100ms resolution
• Allowing dynamically swapping the model endpoint & version in the UI to easily try other models in the UI without having to change the code
• Improve the solar panel model or swimming pool model to make better predictions