sshan-zhao
commented
3 years ago Hi, I download the provided pre-trained models and use test.py to evaluate on Kitti dataset. However, I get the performance that is a little different from the one in the paper.
This is the performance I get: (80m) Abs Rel, Sq Rel, RMSE, RMSE log, δ < 1.25, δ < 1.25^2, δ < 1.25^3 0.1201, 1.0223, 5.162, 0.215, 0.848, 0.944, 0.974
Abs Rel, Sq Rel, RMSE, RMSE log, δ < 1.25, δ < 1.25^2, δ < 1.25^3 This is the performance I get: (80m) 0.149, 1.003, 4.995, 0.227, 0.824, 0.941, 0.973
I'm not sure if this difference is reasonable or I make some mistake in the evaluation method. Many thanks.
Hi, You got the right results. The differences result from: 1) the provided model is re-trained; 2) there is a trivial bug in the original code. For the evaluation code, there was a bug in calculating the ground truth, which I have corrected. After correction, the Abs Rel and \delta < 1.25 can be improved, while RMSE drops slightly. In addition, I retrained the model, as stated in ReadMe, but the new results are very similar to the old results.
ccc870206
Hi, I download the provided pre-trained models and use test.py to evaluate on Kitti dataset. However, I get the performance that is a little different from the one in the paper.
This is the performance I get: (80m) Abs Rel, Sq Rel, RMSE, RMSE log, δ < 1.25, δ < 1.25^2, δ < 1.25^3 0.1201, 1.0223, 5.162, 0.215, 0.848, 0.944, 0.974
Abs Rel, Sq Rel, RMSE, RMSE log, δ < 1.25, δ < 1.25^2, δ < 1.25^3 This is the performance in paper: (80m) 0.149, 1.003, 4.995, 0.227, 0.824, 0.941, 0.973
I'm not sure if this difference is reasonable or I make some mistake in the evaluation method. Many thanks.