CGI-Stereo: Accurate and Real-Time Stereo Matching via Context and Geometry Interaction

Gangwei Xu*, Huan Zhou*, Xin Yang

Paper

SOTA results.

The proposed CGF-ACV ranks the first on the KITTI 2012 and 2015 leaderboards among all the published methods. The proposed CGI-Stereo outperforms all other published real-time methods on KITTI benchmarks.

Our proposed CGF can be easily embedded into many existing stereo matching networks, such as PSMNet, GwcNet and ACVNet. The resulting networks are improved in accuracy by a large margin.

Method	KITTI 2012 (3-noc)	KITTI 2012 (3-all)	KITTI 2015 (D1-bg)	KITTI 2015 (D1-fg)	KITTI 2015 (D1-all)
PSMNet	1.49 %	1.89 %	1.86 %	4.62 %	2.32 %
CGF-PSM	1.21 %	1.57 %	1.46 %	3.47 %	1.80 %
GwcNet	1.32 %	1.70 %	1.74 %	3.93 %	2.11 %
CGF-Gwc	1.17 %	1.52 %	1.38 %	3.34 %	1.71 %
ACVNet	1.13 %	1.47 %	1.37 %	3.07 %	1.65 %
CGF-ACV	1.03 %	1.34 %	1.31 %	3.08 %	1.61 %

How to use

Environment

NVIDIA RTX 3090
Python 3.8
Pytorch 1.12

Install

Create a virtual environment and activate it.

conda create -n CGI python=3.8
conda activate CGI

Dependencies

conda install pytorch torchvision torchaudio cudatoolkit=11.3 -c pytorch -c nvidia
pip install opencv-python
pip install scikit-image
pip install tensorboard
pip install matplotlib 
pip install tqdm
pip install timm==0.5.4

Data Preparation

Train

Use the following command to train CGI-Stereo on Scene Flow. First training,

python train_sceneflow.py --logdir ./checkpoints/sceneflow/first/

Second training,

python train_sceneflow.py --logdir ./checkpoints/sceneflow/second/ --loadckpt ./checkpoints/sceneflow/first/checkpoint_000019.ckpt

Use the following command to train CGI-Stereo on KITTI (using pretrained model on Scene Flow),

python train_kitti.py --logdir ./checkpoints/kitti/ --loadckpt ./checkpoints/sceneflow/second/checkpoint_000019.ckpt

Evaluation on Scene Flow and KITTI

Pretrained Model

Generate disparity images of KITTI test set,

python save_disp.py

Citation

If you find this project helpful in your research, welcome to cite the paper.

@article{xu2023cgi,
  title={CGI-Stereo: Accurate and Real-Time Stereo Matching via Context and Geometry Interaction},
  author={Xu, Gangwei and Zhou, Huan and Yang, Xin},
  journal={arXiv preprint arXiv:2301.02789},
  year={2023}
}

Acknowledgements

Thanks to Antyanta Bangunharcana for opening source of his excellent work Correlate-and-Excite.

gangweiX / CGI-Stereo

readme