search

SamsungLabs / saic_depth_completion

Official implementation of "Decoder Modulation for Indoor Depth Completion" https://arxiv.org/abs/2005.08607
Mozilla Public License 2.0
106 stars 17 forks source link

problems about the evaluation #2

Open chaoer opened 4 years ago

chaoer commented 4 years ago

Hi,

Thanks for your great work.

When I tried to test on Mattport3D dataset with config "DM-LRN" and config_file "_DM-LRN_efficientnet-b4pepper.yaml", I can't get similair results as illustrated in README.

The testing environment was created using conda as you wrote. The tesing code is the third line you wrote in Evaluation part.

So I am wondering is there anything I missed. OR are there some tricks should I take when conduct testing ?

P.S. Could you please tell me what's the path of the images used for illustration? Thus, I can test on them and to figure out whether I had created the testing environment correctly.

YunfanZhang42 commented 4 years ago

Hi,

Thank you for your inspiring work! I downloaded your code and pretrained model weights and tried to evaluate the prediction results on the same Matterport test split as "Deep Depth Completion of a Single RGB-D Image" by Yinda Zhang et al, but unfortunately was unable to reproduce the results in the paper. I tried all models from DM-LRN b2 to DM-LRN b4 but none of results matched what's mentioned in the paper. I am wondering what might happened here - does the depth images need to be preprocessed in a certain way, or is there a different set of model weights available? Thanks in advance for your help!

Best, Yunfan

tmanh commented 3 years ago

Hi, me too. I also tried all the models and the results were not even close to what stated in the paper. I think they had updated something right before they uploaded the code to GitHub (like refactor). And, they accidentally screwed all the settings up. Or, might be, they did some preprocessing but forgot to mention.