bertjiazheng / Structured3D

[ECCV'20] Structured3D: A Large Photo-realistic Dataset for Structured 3D Modeling
https://structured3d-dataset.org
MIT License
534 stars 62 forks source link

Semantic annotation for bounding boxes? #13

Closed neyrinck closed 4 years ago

neyrinck commented 4 years ago

Hi, great dataset! I was wondering if the object semantic annotations, currently only in semantic.png files it seems, could somehow be inferred from the 3D bounding box id's in bbox_3d.json, or in any other 3D data? If not, could this easily be added? Thank you!!!

bertjiazheng commented 4 years ago

Hi, @neyrinck

Currently, the semantic annotations only saved in the semantic.png. You can get infer the semantic label for the 3D bounding box from semantic.png and instance.png in the perspective images.

Best, Jia

neyrinck commented 4 years ago

Thanks! And (it seems) from the panoramic images, too, which is easier!

bertjiazheng commented 4 years ago

The current version of the dataset does not include instance.png for the panoramic images, but we are currently working on it. Hope this helps.

neyrinck commented 4 years ago

Ah right, thank you ... but sorry if I am missing something: is instance.png necessary for inferring the semantic labels? I imagined I could just ignore wall/floor/ceiling etc. categories from semantic.png.

On Fri, Apr 24, 2020 at 10:22 AM Jia Zheng notifications@github.com wrote:

The current version of the dataset does not include instance.png for the panoramic images, but we are currently working on it. Hope this helps.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bertjiazheng/Structured3D/issues/13#issuecomment-619111475, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABXQKHLEUKM4QHFDXLUE3TLROG4EHANCNFSM4MPZMJXA .

bertjiazheng commented 4 years ago

You could get an accurate label for each 3D bounding box with the instance label. Without instance label, the semantic label can be determined by the majority voting within each bounding box.

Xia0ben commented 4 years ago

Hi @bertjiazheng !

Great dataset indeed ! I ended up finding the same issue when exploring the data : having semantic labels directly associated with the 3D bounding boxes would be so much more helpful ! Especially since it would likely be more accurate to do the association from the source data you used for producing the dataset than inferring it from panoramic pictures (that may hide much of the information because of object occlusion).

As far as you know, has any user tried to do this completely ? @neyrinck, @micaeltchapmi, did any of you succeed in computing this inference ?

Could you please consider providing these annotations ? This would really help in working with the dataset, and would not require any new information in itself.

Thank you very much for your time and consideration.