search

SpadeLiu / Lac-GwcNet

Local Similarity Pattern and Cost Self-Reassembling for Deep Stereo Matching Networks
MIT License
44 stars 9 forks source link

Testing/evaluation phase #9

Closed Salvatore-tech closed 2 years ago

Salvatore-tech commented 2 years ago

Good evening, I just managed to finish the training on SceneFlow and finetuning step by using the training images of KITTI_2015 and i was wondering how to evaluate the model againist the testing images. Could you give me any tips? Should i run the test_kitti.py? If so, do i need to provide the checkpoint.tar generated in the finetuning step?

SpadeLiu commented 2 years ago

Good evening, I just managed to finish the training on SceneFlow and finetuning step by using the training images of KITTI_2015 and i was wondering how to evaluate the model againist the testing images. Could you give me any tips? Should i run the test_kitti.py? If so, do i need to provide the checkpoint.tar generated in the finetuning step?

Hi,

On KITTI 2015, the training 200 image pairs are split as a training set and a validation set. Both the test phase of "KITTI_ft.py" and "test_kitti.py" are used to evaluate the model on the validation set. If you want to evaluate on the test set, you need to finetune the model with 200 image pairs and generate the disparity maps of the test images. GT of test images are not provided, you need to upload the disparity maps to the online KITTI benchmark.

Salvatore-tech commented 2 years ago

Alright it makes sense now, do you have any tips for me to get disparity maps on a custom dataset without ground truths? Do you think that training the model on SceneFlow (you already did it) plus other dataset could be a good idea?

SpadeLiu commented 2 years ago

Alright it makes sense now, do you have any tips for me to get disparity maps on a custom dataset without ground truths? Do you think that training the model on SceneFlow (you already did it) plus other dataset could be a good idea?

Hi,

The first thing is to confirm that the left and the right cameras are rectified and the maximum disparity is no more than 192. After that, we should consider the problem of domain shift. You can use SceneFlow + other dataset but this "other dataset" should be close to your test images. Another solution is to use domain generalized stereo matching models. Our CVPR 2022 work "GraftNet: Towards Domain Generalized Stereo Matching with aBroad-Spectrum and Task-Oriented Feature" belongs to that category. There are also other amazing works such as CFNet (CVPR 2021), DSMNet (ECCV 2020).

Salvatore-tech commented 2 years ago

Lovely, thanks a lot for your support, i'm going to study the work you pointed out :)