Problems about evaluation results on the NYU-V2 dataset

[rainfall1998]

Hi, thanks for the wonderful work! I have made a evaluation on NYU-V2 test dataset and the code is based on demo.py and the pretrain model provided . I add the h5py read function and evaluation result calculation funcion.

However, I got a result lower than the paper mentioned. So I wonder whether the evaluation code is wrong or the model is wrong? The result and evaluation code is following.

Thanks for your working and looking forward to your reply

evaluation result: val_error/abs_rel | val_error/sq_rel | val_error/rmse | val_error/rmse_log | val_error/lg10 | val_acc/a1 | val_acc/a2 | val_acc/a3 0.17270 0.14038 0.59941 0.21287 0.07320 0.73734 0.93800 0.98530

evaluation code(the py file isn't supported, turn it to txt): nyutest_demo.py.txt

[choyingw]

Thank you for the interest in the work. I just updated the README.md and add more models there. Please check.

[rainfall1998]

Thank you for the quick reply ! I still have some other questions:

1. Which type you use for nyu test ? all 1449 nyu_labeled provided by NYU or 654 nyu_test provided by [Indoor-SfMLearner] ? If possible, could I get your txt of val list?
2. I didn't find the train split and val split for SimSIN dataset, where can I get them?
3. Is there any plan to publish the whole the training code later? I find some of the training code has been published.

Thank you very much for your excellent work and looking forward to your reply

[choyingw]

1. 654 is used. Some other repo also provide the processed data, such as https://github.com/JunjH/Visualizing-CNNs-for-monocular-depth-estimation
2. Our experiments use the whole SimSIN as the training data and didn't segment into train or val split further.
3. Under discussion