search

aluo-x / 3D_SLN

Official code for "End-to-End Optimization of Scene Layout" -- including VAE, Diff Render, SPADE for colorization (CVPR 2020 Oral)
Apache License 2.0
52 stars 7 forks source link

Sun RGB-D data generation for indoor 3D layouts #3

Open jinzita142857 opened 11 months ago

jinzita142857 commented 11 months ago

Hello, your article is really wonderful, I am also very interested in your work, I have tried to generate a reasonable indoor 3D layout using Sun RGB-D data. But I don't know the specific implementation project, can you give some guidance?

aluo-x commented 11 months ago

Hi @jinzita142857

Could you clarify what you are trying to achieve? Are you getting errors when trying to run my code?

I would be glad to help you out with your problems.

jinzita142857 commented 11 months ago

Hello, thank you very much for your reply. (Beginner) question: I plan to implement your "Example-based scene layout synthesis" section, but I don't know the specific steps. Can you give me some specific steps to do this? Thank you again and look forward to hearing from you.

aluo-x commented 11 months ago

Our example-based scene layout synthesis was implemented approximately as follows:

  1. Given an image, we use "Cooperative Scene Parsing" to parse out 3D bounding boxes and object identities
  2. These bounding boxes are in world space coordinates, we can then take the centroid of each bounding box, and provide them to a rule based parser, which generates pair-wise object relations from the bounding box centroids. We do this using a modified "compute_rel" function, adapted to the coordinate space of "Cooperative Scene Parsing".
  3. Since we now have a scene graph, we can then generate new scenes based on 3D-SLN.
jinzita142857 commented 11 months ago

Thanks for the reply, I'm still trying.