search

chengzhag / Implicit3DUnderstanding

Official PyTorch code of Holistic 3D Scene Understanding from a Single Image with Implicit Representation (CVPR 2021). Also includes a PyTorch implementation of the decoder of LDIF (from 3D Shape Representation with Local Deep Implicit Functions).
https://chengzhag.github.io/publication/im3d/
MIT License
200 stars 34 forks source link

confusion about the final chamfer distance for LDIF result #12

Closed Harvey-Mei closed 2 years ago

Harvey-Mei commented 3 years ago

Hi, chengzhag. Thanks for you excellent work. I tried to reproduce the mesh generation part of your work, exactly the LDIF network, and I train this part followed the parameters mentioned in the paper, the best result I got from my reproduction was wicp_avg_chamfer=7.25, which remain a certain gap with the results shown in the paper(6.72). I am not sure if i missed some thing. Another question. I used the checkpoint file provided in the readme file to get 6.72. However, the final statistical results of various categories are not quite consistent with those shown in the paper, so I would like to ask if there is any difference between the statistical results in the paper and the statistical methods in the code?
微信截图_20210830142401 微信截图_20210830142438

chengzhag commented 3 years ago

Hi, @Harvey-Mei,

Thanks for your interest in our work!

I think that the training process might introduce some randomness to the results. Maybe try again to see if there is any change to the results? For your reference, here is the wandb report of experiment 20101613380518, including the LR/training/testing curves. The report also includes another experiment 20101510203369 (green), which is almost identical with 20101613380518 (orange) in terms of settings. However, the results differ a lot since the learning rate in green drops early than orange. image The report also includes an experiment 20103116530721 which tests the weight trained on watertight meshes in 20101613380518 on non-watertight meshes. It is where we got the results in the paper.

The reason for your second question might be that the ICP algorithm provided by gaps can introduce some randomness in the process of optimizing to align the meshes. For your reference, chamfer_woICP of each category is: {'bed': 4.3483425925046015, 'wardrobe': 5.368200843907593, 'bookcase': 4.65738395402351, 'chair': 6.334140533872199, 'sofa': 5.99414974168976, 'table': 15.723936902234081, 'desk': 10.808811716871475, 'misc': 29.429790655298163, 'tool': 4.530272881545347}.

Hope this can help solve your problem.

Harvey-Mei commented 3 years ago

Hi, @chengzhag , Very grateful for your quick reply! I'll study the information you gave me. Good luck with your research.

chengzhag commented 3 years ago

PS: The watertight process we use introduces thicker object meshes thus can lead to drop on chamfer distance when evaluating on non-watertight meshes.Although this is what we report in the paper, I personally think it would be fair to use our test results on watertight meshes for comparison.