3D detection is not working well with custom input

GAP-LAB-CUHK-SZ / Total3DUnderstanding

Implementation of CVPR'20 Oral: Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Image

MIT License

415 stars 50 forks source link

3D detection is not working well with custom input #24

Closed WenM1222 closed 3 years ago

WenM1222 commented 3 years ago

Hi, I tested with a image from 3D-Future dataset, and the 3D bounding box looks not well predicted. Is this normal? This is the test data. custom.zip

And test result.

custom_image_detect_scene_res custom_image_mesh_scene_res

Washeded commented 3 years ago

Hi, I have met the same problem. And I calibrated my camera intrinsic parameters. Do you have any solution for this?

WenM1222 commented 3 years ago

Sorry, I haven't found solution yet.

yinyunie commented 3 years ago

It is perhaps for the reason that our model is only trained on SUN RGB-D. The input distribution (intrinsics, images) is quite different from yours. I also received some suggestions about using our model on in-the-wild images. You can also try it by using the same intrinsics as ours and preprocess the images that appear similar to SUN RGB-D (scale or center crop). But it is only for visual appearance, no guarantees on accuracy.