OmniNxt: A Fully Open-source and Compact Aerial Robot with Omnidirectional Visual Perception

IROS 2024 Oral
Peize Liu, Chen Feng^†, Yang Xu, Yan Ning, Hao Xu^†, and Shaojie Shen

HKUST Aerial Robotics Group   
^†Corresponding Authors

📢News

• [30/06/2024]: OmniNxt is accepted to IROS 2024.
• [15/08/2024] V0.1 Release

Outline

[TOC]

🤖Build your own OmniNxt

You should have basic knowledge of the standard quadrotor's electronic system.

BOM

Our latest design can be accessed by Fusion360.🔗Link Access code: hkustUAV

Multi-fisheye Camera Module

ROS wrap driver 🔗oak_ffc_4p, which contains hardware information, ROS1 driver and test tools.

Calibration tool 🔗 quater-calib

Jetson Orin

Environment Info

We suggest you follow these settings to avoid conflict (especially CUDA and TensorRT).

Nxt-FC

We open-source hardware designs at 🔗NXT-FC. Nxt-FC supports PX4 and Ardupilot firmware.

Follow the instructions for setting up the Nxt-FC part to configure your flight controller.

The PX4 parameter of OmniNxt can be downloaded from Onedrive.

💾Runtime setup

Our Omni-VINS and Omni-Depth are developed from D2SLAM. Since building the Docker images on the local machine(Jetson Orin) usually takes a long time, we suggest you pull the images from the Docker Hub. If you want to build the Docker image on your local machine, follow the instructions [Build Docker Images On Local Machine](#Build Docker images in local machines).

the docker images structure is as follows:

Step 1 Clone the repository into your local machine

git clone --branch pr_fix_main https://github.com/HKUST-Aerial-Robotics/D2SLAM.git

Step 2 Configure your quad-cam parameters and VINS parameters following our template

cd ./D2SLAM/config/quadcam_drone_nxt_tmp

If you have already calibrated your quad fisheye camera set, you can simply replace the files with the same name, which includes:

1. stereo_calib_n_m_240_320.yaml (n and m is the camera number; used in Omni-Depth).
2. fisheye_cams.yaml. (used in Omni-VINS)

Step 3 Configure ./start_docker.sh script with your local environment.

./start_docker.sh is under the ./D2SLAM directory.

Modify the following parameters with the absolute path under your local host environment.

1. DATA_SET (where the data set is) (Optional)

Then run with the following command under the D2SLAM directory (very important, this will map your D2SLAM dir into the container)：

./start_docker.sh 1

Step 4 Launch algorithm modules

run all algorithm modules together (Omni-VINS & Omni-Depth)

## you should under /root/swarm_ws/
source ./devel/setup.bash
roslaunch d2vins quadcam.laucnh

Only launch Omni-VINS. Please remove the nodes in the red box

source ./devel/setup.bash
roslaunch d2vins quadcam.laucnh

Only launch Omni-Depth

source ./devel/setup.bash
roslaunch quadcam_depth_est depth-node.launch

[Notice] If you are running Omni-VINS and Omni-Depth for the first time, the initialization process of these two modules would be a little bit long because of the building of the inference engine.

Almost done

If everything works well, you will see

which means Omni-VINS(D2VINS) initialized normally.

Omni-Depth

🖥️Build Docker images in local machines

Step 1 Build Jeston_base image

cd /D2SLA/docker
make jetson_orin_base

Step 2 Build Jeston Orin image

cd /D2SLA/docker
make jetson_orin

PX4 Controller

We highly suggest you refer to Fast-Lab's PX4 controller PX4-Control

We also provide our yaw-rotation-free version of Fast-Lab's PX4 controller PX4-Control.

Please cite ZJU-Fast-Lab if these modules are useful for your research and project.

Flight planer

We modified some of the strategies in the ego-planner. Our version is here ego-planner-omni-modify

Please cite ZJU-Fast-Lab's ego-planner if these modules are useful for your research and project.

🔧Troubleshooting

💯Acknowledgment