Result discrepancy between provided model and the paper

xy-guo / Learning-Monocular-Depth-by-Stereo

Learning Monocular Depth by Distilling Cross-domain Stereo Networks, ECCV18

https://arxiv.org/abs/1808.06586

MIT License

94 stars 8 forks source link

Result discrepancy between provided model and the paper #7

Closed kwea123 closed 5 years ago

kwea123 commented 5 years ago

I download some of the pretrained models, and found that results are different from your paper. Here's one of the example (release-StereoNoFt.ckpt): abs_rel, sq_rel, rms, log_rms, d1_all, a1, a2, a3 0.0754, 0.6812, 4.023, 0.162, 0.000, 0.931, 0.971, 0.984

In the paper (page 14) however, you have abs_rel, sq_rel, rms, log_rms, d1_all, a1, a2, a3 0.072, 0.665, 3.836, 0.153, 0.000, 0.936, 0.973, 0.986

The results obtained from your pretained model are slightly worse than what you report in the paper, could you please clarify the reason?

kwea123 commented 5 years ago

On the other hand, the performance of monocular models (StereoUnsupFt-->Mono pt and StereoUnsupFt-->Mono pt C,K for what I tried) are the same as the results in the paper.

xy-guo commented 5 years ago

For all the stereo models, the results are tested on the validation set.

kwea123 commented 5 years ago

Hi, I noticed that your evaluation script gives different results from that given with Monodepth. Could you please explain what you changed in the evaluation script and why?

xy-guo commented 5 years ago

The unsupervised stereo algorithm should give the camera-view depth instead of LIDAR-view depth.