This is the official code repository for the paper: "Geometry-Informed Distance Candidate Selection for Adaptive Lightweight Omnidirectional Stereo Vision with Fisheye Images".

Overview

We are working on providing better details about our work, including the code, datasets, pre-trained models , and more. Please stay tuned while things are progressing.

At the current moment, we provide instructions on how to run our pre-trained models locally. Please refer to the Offline validation instructions for more details.

For the developers, here is the original README page.

Code structure

(Coming soon)

Training and validation procedures and pre-trained models

(More details coming soon)

PyTorch models

Pre-trained models (need associated configs):

Update history

• 2023-12-31: G16V ( two versions used in various places, need to add config29 )

TensorRT models

We also provide the TensorRT version of some of our models. Please note that depending on what hard ware platform the user is trying to deploy, the TensorRT model provided by us may not work. For this purpose, we also provide the ONNX models associated the the TensorRT ones such that a user can do the conversion on their own hardware platform. Pleae follow the instructions to do the conversion. All the ONNX models are the sanitized version described in the instructions.

Update history

• 2023-12-31: G16V ( was config21 in the paper, now config29, need update paper )

Datasets

We created a new synthetic dataset with 3 fisheye cameras facing to the same direction. The data are collected by using the simulation environments provided by TartanAir V2, whichi itself is under development and will be released soon. We have a training set and a validation set. The total size is about 1.3T with the training set being 1.2T. There are 50 environments for the trainig set and 21 envrionments for the validation set. We made sure that there are no overlaps between them.

The dataset is currentl hosted by our own server and we provide simple scripts for downloading.

For the training set, first use the following commands to download the environment list and the downlaoding script.

wget https://airlab-share.andrew.cmu.edu:8081/MVS_Fisheye_Dataset/tar_list_train.txt
wget https://airlab-share.andrew.cmu.edu:8081/MVS_Fisheye_Dataset/download_train.sh
chmod +x download_train.sh

tar_list_train.txt is a list of envrionment names in the training set. The data size is also listed in this file. download_train.sh is the script for downloading the data. The user can inspect the script and augment it according to the use case. E.g., the user can comment out some environment names and only download a subset of data.

# To check if the URLs are all valid.
./download_trah.sh check

# To perform the download.
./download_train.sh download

For the validation set, the procedure is the same. Use the following commands to download the environment list and the script first.

wget https://airlab-share.andrew.cmu.edu:8081/MVS_Fisheye_Dataset/tar_list_validate.txt
wget https://airlab-share.andrew.cmu.edu:8081/MVS_Fisheye_Dataset/download_validate.sh
chmod +x download_validate.sh

Then use the following to check and download the data.

# To check.
./download_validate.sh check

# To download.
./download_validate.sh download

The structure of the dataset is the same with the sample dataset used in the Offline validation instructions. A separate documentation (coming soon) gives more details about the design of the dataset.

Paper (preprint)

The preprint version of the paper is available on the AirLab's website.

Citation

(Coming soon)