search

ldkong1205 / RoboDepth

[NeurIPS 2023] RoboDepth: Robust Out-of-Distribution Depth Estimation under Corruptions
https://ldkong.com/RoboDepth
264 stars 33 forks source link

Potential bugs in Monocular-Depth-Estimation-Toolbox #10

Closed zhyever closed 1 year ago

zhyever commented 1 year ago

Hi, thanks for this meaningful challenge and for kindly referring to my depth estimation repo. I notice that there are so many submissions to the codalab server suffering from a very low performance (about 0.3). This is related to a potential bug in the toolbox and I'd like to point it out.

Please notice the dataset file in the toolbox. All of them have one line:

img_infos = sorted(img_infos, key=lambda x: x['filename'])

for example, in nyu. The sort logic of str type is definitely different from int (one can try sorted on List('sample_1', 'sample_2', 'sample_10'), and List(1, 2, 10)). To ensure the order of predictions, users should comment these lines.

ldkong1205 commented 1 year ago

Hi @zhyever, thank you so much for the comment! I will update the instruction based on your suggestion.

Thanks again for meticulously maintaining the awesome Monocular Depth Estimation Toolbox codebase. :ok_man:

zhyever commented 1 year ago

:D Thanks. Best wishes.