search

scene-verse / SceneVerse

MIT License
127 stars 1 forks source link

Missing scene caption files for ProcTHOR and ScanNet #3

Closed chrockey closed 3 months ago

chrockey commented 3 months ago

Hi there,

Congratulations on the remarkable project! I appreciate that you released the SceneVerse dataset.

Upon downloading the dataset, I noticed that the scene caption JSON file (scene_cap.json) is missing for the ProcTHOR and ScanNet datasets. Could you provide clarification on why these datasets do not include scene caption files?

I have two more questions:

Thanks!

Buzz-Beater commented 3 months ago

Hi,

  1. For ScanNet, we have just uploaded the scene_cap.json into the G-drive corresponding folder, thanks for the notice! For ProcTHOR, we are still refining the scene caption currently and will release it afterward.

  2. We will add the descriptions into the README, but basically, you should be expecting several ssg_ref_xxx_{gpt/template}.json files, the template file contains language data generated from template, and gpt is the gpt-refined version of its corresponding template-based generations.

As for the specific type, please refer to the supplementary of our ArXiv version, we use rel2 for pair-wise, relm for multi-object, and star for star reference. We also provide an additional version of chain that contains referrals using object A to find B and then B to find C but found it not natural enough and therefore left them as optional data. Feel free to use them based on your tasks.

chrockey commented 3 months ago

Thanks for the detailed reply! It really helps me to understand the dataset.

However, the below question regarding the 3D scene graph data is still remaining. It seems that the 3D scene graph data is not released although both scene-level and object-level captions/referrals are released. Am I understanding this correctly?

Is scene graph data used for caption generation also released? If not, could you please release the scene graph data?

Buzz-Beater commented 3 months ago

Yes, the scene graph and generation pipeline are currently not within the data release plan. We intend to iterate on the data generation process based on feedback from the current data release. We will release the generation pipeline and scene graphs in the future after refinement.

I have also added a specification on the provided data types on the README.md, feel free to let me know if it could clear your confusion!

chrockey commented 3 months ago

Thanks for the clarification! All of my confusions are resolved :)