Abnormal assessment results

[MModerato]

Hi,

Thank you for your work !

I downloaded the model_kitti_stereo from your page and then evaluated the model in the Kitti 2015 dataset. Although I don't quite understand these performance indicators, comparing the data in this paper, I feel that these results are not correct. I just changed all PNG to JPG as you said, then changed the format of the picture in evaluation_utils.py, and the others remained default. Here are my commands

python monodepth_main.py --do_stereo --mode test --data_path ~/monodepth/monodepth/KITTI/ --filenames_file ~/monodepth/monodepth/utils/filenames/kitti_stereo_2015_test_files.txt --output_directory ~/monodepth/monodepth/temp/kitti_eva/ --checkpoint_path ~/monodepth/monodepth/model/model_kitti_stereo/model_kitti_stereo

python utils/evaluate_kitti.py --split kitti --predicted_disp_path ~/monodepth/monodepth/temp/kitti_eva/disparities.npy --gt_path ~/monodepth/monodepth/KITTI

[mrharicot]

Hi,

Did you have a look at the generated depthmaps?

[aliko70]

Hi 111Moderato, @mrharicot I am getting almost same result when I test by the downloaded kitti model. The generated depthmaps both using the given-model as well as my own trained models all look good while their evaluation on kitti_stereo_2015_test_files gives me the same abnormal results... Did you get a way around it?

-sample of generated depth:

• after min_depth = 1e-3 max_depth = 80 pred_depth[pred_depth < min_depth] = min_depth pred_depth[pred_depth > max_depth] = max_depth

-corresponding gt-depth:

This ends up with very large values in 'thresh' when computing errors.

Will really appreciate if you can get me some hints @dantkz @gosip @Hirico .

Thanks, Ali