Race Against the Machine

A Fully-annotated, Open-design Dataset of Autonomous and Piloted High-speed Flight

https://github.com/tii-racing/drone-racing-dataset/assets/14661511/ead6d6d2-0307-4e31-bcef-8b17c8cdf961

This repository contains a dataset characterized by:

fast (>21m/s) and aggressive quadrotor flight
autonomous and human-piloted flight, on multiple trajectories
high-resolution, high-frequency collection of visual, inertial, and motion capture data
it includes drone racing gates—with bounding boxes and individual corner labels
it includes control inputs and battery voltages

If you use this repo, you can cite the companion paper as:

@ARTICLE{10452776,
         author={Bosello, Michael and Aguiari, Davide and Keuter, Yvo and Pallotta, Enrico and Kiade, Sara and Caminati, Gyordan and Pinzarrone, Flavio and Halepota, Junaid and Panerati, Jacopo and Pau, Giovanni},
         journal={IEEE Robotics and Automation Letters}, 
         title={Race Against the Machine: A Fully-Annotated, Open-Design Dataset of Autonomous and Piloted High-Speed Flight}, 
         year={2024},
         volume={9},
         number={4},
         pages={3799-3806},
         doi={10.1109/LRA.2024.3371288}
}

Installation/Download

Tested on October 2023 with Ubuntu 20.04 LTS (Python 3.8), macOS 14 (Python 3.11), Windows 11 (Python 3.9).

Clone the repository and install the requirements:

git clone https://github.com/Drone-Racing/drone-racing-dataset.git
cd drone-racing-dataset
pip3 install -r requirements.txt

From folder drone-racing-dataset/, use the following scripts to download the dataset files.

On Ubuntu and macOS:

sudo apt install wget   # or, e.g, `brew install wget` on macOS
sudo chmod +x data_downloader.sh
./data_downloader.sh

On Windows, double-click on file drone-racing-dataset/data_downloader.cmd

This will create and populate 2 folders in the root of the repository:

data/piloted/
data/autonomous/

data/piloted/ and data/autonomous/ contain 12 and 18 flight-.../ folders respectively.

Data Format

For each flight, 2 CSV files are provided:

One sampled/interpolated at the timestamps of the camera frames (..._cam_ts_sync.csv)
One sampled/interpolated at 500Hz (..._500hz_freq_sync.csv)

Each CSV file contains the following columns:

Column Number and Quantity Name	Unit	Data Type
0. `elapsed_time`	$s$	float
1. `timestamp`	$\mu s$	int
2. `img_filename`	n/a	string
3. `accel_[x/y/z]`	$m/s^2$	float
6. `gyro_[x/y/z]`	$rad/s$	float
9. `thrust[0-3]`	$1$	float $\in [0,1]$
13. `channels_[roll/pitch/thrust/yaw]`	$1$	int $\in [1000,2000]$
17. `aux[1-4]`	$1$	int $\in [1000,2000]$
21. `vbat`	$V$	float
22. `drone_[x/y/z]`	$m$	float
25. `drone_[roll/pitch/yaw]`	$rad$	float
28. `drone_velocity_linear_[x/y/z]`	$m/s$	float
31. `drone_velocity_angular_[x/y/z]`	$rad/s$	float
34. `drone_residual`	$m$	float
35. `drone_rot[[0-8]]`	$1$	float
44. `gate[1-4]_int_[x/y/z]`	$m$	float
56. `gate[1-4]_int_[roll/pitch/yaw]`	$rad$	float
68. `gate[1-4]_int_residual`	$m$	float
72. `gate[1-4]_int_rot[[0-8]]`	$1$	float
108. `gate[1-4]_marker[1-4]_[x/y/z]`	$m$	float

Image Format

For each flight, 2 folders contain the Arducam video capture data:

camera_flight-.../ contains all the captured frame in JPEG format
labels_flight-.../ contains the racing gates' bounding boxes and corner labels in the TXT format described below

Labels Format

Each TXT file contains lines in the form 0 cx cy w h tlx tly tlv trx try trv brx bry brv blx bly blv where:

0 is the class label for a gate (the only class in our dataset)
cx, cy , w, h ∈ [0, 1] are a gate's bounding box center’s coordinates, width, and height, respectively
tlx, tly ∈ [0, 1], tlv ∈ [0; 2] are the coordinates and visibility (0 outside the image boundaries; 2 inside the image boundaries) of the top-left internal corner. Similarly for tr, bl, br, the top-right, bottom-left, and bottom-right corners.

All values are in pixel coordinates normalized with respect to image size. The keypoints label format follows the COCO definition. The gates/lines in each TXT file are not ordered.

Visualization Scripts

The scripts in the scripts/ folder can be used to visualize the data and to convert the data to other formats.

To plot one of the CSVs, for example, use:

cd scripts/
python3 ./data_plotting.py --csv-file ../data/autonomous/flight-01a-ellipse/flight-01a-ellipse_cam_ts_sync.csv

To visualize the Arducam frames and labels, use (press SPACE to advance, CTRL+C to exit), for example:

cd scripts/
python3 ./label_visualization.py --flight flight-01a-ellipse

FPV Racing Drone Open Design

Folder quadrotor/ contains the bill of material and STL files of the COTS/open design of the racing drone used to collect the dataset. In the same folder, you can also find the Betaflight parameters backup used to configure the drone. A tutorial to assemble the drone is on YouTube.

ROS2 Bags

Tested on October 2023 with Ubuntu 20.04 LTS

ROS2 .sqlite3 bags are stored in the ros2bag_.../ folder of each flight.

The rosbags contain topic with custom messages defined in the repository drone-racing-msgs.

To play a rosbag:

Install ROS2 Humble following the official guide
Install the custom messages defined in drone-racing-msgs

mkdir -p ~/drone_racing_ws/src
cd ~/drone_racing_ws/src
git clone https://github.com/tii-racing/drone-racing-msgs.git
cd ..
rosdep install --from-paths src --ignore-src -r -y
colcon build --symlink-install

Source the ROS2 workspace and play the rosbag

source ~/drone_racing_ws/install/setup.bash
ros2 bag play data/autonomous/flight-01a-ellipse/ros2bag_flight-01a-ellipse

Print the topic list and echo a topic

source ~/drone_racing_ws/install/setup.bash
ros2 topic list
ros2 topic echo sensors/imu

Additional Resources

drone-racing-dataset
├── camera_calibration
│   ├── calibration_results_trackRATM.json - Camera parameters of trackRATM flights in JSON format.
│   ├── calibration_results_trackRATM.npz - Camera parameters of trackRATM flights in NumPy format.
│   ├── calibration_results.json - Camera parameters of ellipse and lemniscate flights in JSON format.
│   ├── calibration_results.npz - Camera parameters of ellipse and lemniscate flights in NumPy format.
│   └── drone_to_camera.json - Drone to camera in JSON format. Translation was measured by MoCap, rotation is an estimation, and includes the FLU to RDF conversion.
├── ...
└── scripts
    ├── trajectory_generation
    │   ├── ellipse.py - Script used to generate the trajectories of autonomous ellipse flights.
    │   ├── lemniscate.py - Script used to generate the trajectories of autonomous lemniscate flights.
    │   ├── trajectory.py - Base class for trajectory generation.
    ├── camera_calibration.py - Script used to generate the files in `camera_calibration/`.
    ├── create_std_bag.py - Script used to generate standard ROS2 bags with `Image`, `Imu`, and `PoseStamped` messages. 
    │                       Standard bags are not provided in the dataset because of their size (>10GB each).
    │                       You will need the ROS2 workspace with the custom messages installed (see section "ROS2 Bags").
    ├── data_interpolation.py - Script used to generate the comprehensive CSV files interpolated at arbitrary frequencies.
    ├── ...
    ├── measure_dtw.py - Script used to measure the DTW distance between trajectories of same mode and shape.
    └── reference_controller.py - Python implementation of the PID controller used for the autonomous flights.

License

The dataset released in Race Against the Machine A Fully-annotated, Open-design Dataset of Autonomous and Piloted High-speed Flight by TII Racing Jetsetters is licensed under CC BY 4.0

tii-racing / drone-racing-dataset

readme