search

GAP-LAB-CUHK-SZ / Total3DUnderstanding

Implementation of CVPR'20 Oral: Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Image
MIT License
413 stars 50 forks source link

Evaluation code roomlayout.m of cooperative_scene_parsing don't work on the output #21

Open chengzhag opened 3 years ago

chengzhag commented 3 years ago

Hi Yinyu:

I followed your guide to evaluate the results using code from cooperative_scene_parsing. With GT downloaded and some essential changes to path settings, the roomlayout.m runs successfully on the results predicted by your work.

However, the roomlayout.m outputs the same IoU of 0 on every sample. Here is sample 1 as an example: image

I tried to turn on the visualization option looking to find the reason: image

The predicted layout bounding box showed in the visualization above is nothing like the visualization with your code utils/visualize.py: image image

Is there anything I missed? If some more changes need to be made on the evaluation code from cooperative_scene_parsing, Is it possible to open-source your version of the final evaluation code?

chengzhag commented 3 years ago

Sorry to bother you again at this inappropriate time. In case you missed my last email, I'm trying to do some work based on your paper. Eagerly to compare with your work on the same matrics. May I ask to get the evaluation code you implemented based on cooperative_scene_parsing? If the code is not ready to publish on github, may I get the code before it's published? Hope to not take up you much time.

YanjunLIU-ac commented 3 years ago

@pidan1231239 Hello there, I am also doing some work based on this project, and I also ran into the problem similar to you, where in my case the IoU of 3d bounding box is always 0 using the MATLAB evaluation code from "Cooperative scene parsing". Have you solved this problem? Any ideas on how to make it work properly and get the final evaluation results?

chengzhag commented 3 years ago

Hi, I've contacted Yinyu via email and got the following reply:

Sorry for the late reply as we are currently pushing for a deadline. I think we would update the evaluation code after these busy days. But I think the reason you get the IoUs with 0s is that you used the GT data from their work. Actually, we used a different world coordinate system to build our system. You can visualize (or save by yourself) our gt data from the visualization code by changing 'prediction' to 'gt' or ‘both’ in the arguments below: image

Hope this can help you.