search

aistairc / virtualhome_aist

Other
2 stars 2 forks source link

About supporting Procedural Generation #22

Closed albertcity closed 1 week ago

albertcity commented 2 weeks ago

Hello! I am finding some envrionment to generate synthetic data for training an VLM. I have read your paper and find virtualhome_aist really helpful for me, since it supports lots of animated actions. However, virtualhome_aist only support 7 scenes from virtualhome v2.2, which is not suitable to large-scale generation. I have noted that virtualhome v.2.3 has implemented procedural generation of new scenes, therefore I am wondering if it possible to combine the advantages of both？(i.e. procedural generation from virtualhome v2.2 and actions&KGs from virtualhome_aist)? Best regards.

Ease112 commented 2 weeks ago

Hi @albertcity Thank you for your interest in our project. In conclusion, it is not easy. We started developing VirtualHome-AIST more than three years ago, but VirtualHome 2.3 was not released at that time. So, VirtualHome-AIST is based on 2.2. Migrating to 2.2 and 2.3 is not easy because they have significantly different directory structures. In addition, most of the new features of 2.3 could not be reproduced because it contains many bugs. [my report] Therefore, we have not migrated to 2.3.

Instead of migration from 2.2 to 2.3, we proposed different approaches: (1) automatically generating action scripts from natural language and (2) combining generated KGs to create long scenarios.

Furthermore, we have published datasets generated by VirtualHome-AIST+VirtualHome2KG.

I hope this helps you.

albertcity commented 1 week ago

Thanks a lot for your detailed reply! I will checkout these listed paper:)