allenai / Holodeck

CVPR 2024: Language Guided Generation of 3D Embodied AI Environments.
https://yueyang1996.github.io/holodeck
Apache License 2.0
304 stars 25 forks source link

Can you give an example of complex query? #43

Closed TREEANA closed 4 weeks ago

TREEANA commented 1 month ago

I successfully ran the olodeck/main.py --query "a living room" command and loaded the scene in Unity.

Now I want to command a complex query, but I have no idea what I should do.

Can you give me an example query?

For example, if I want to create an office with a round table in the center, several computer monitors and computer towers placed under and beside the desks, and desks and chairs arranged in a row along the windows, how should I command it?

TREEANA commented 1 month ago

By the way, it would be amazing if the code could convert multiple photos of a room directly into Unity scenes.

YueYANG1996 commented 1 month ago

I successfully ran the olodeck/main.py --query "a living room" command and loaded the scene in Unity.

Now I want to command a complex query, but I have no idea what I should do.

Can you give me an example query?

For example, if I want to create an office with a round table in the center, several computer monitors and computer towers placed under and beside the desks, and desks and chairs arranged in a row along the windows, how should I command it?

You can just try python holodeck/main.py --query "an office with a round table in the center, several computer monitors and computer towers placed under and beside the desks, and desks and chairs arranged in a row along the windows" --single_room True --openai_api_key <OPENAI_API_KEY>

Since the query is long and complex, some of the requirements may not be satisfied in the generated scene.

YueYANG1996 commented 1 month ago

By the way, it would be amazing if the code could convert multiple photos of a room directly into Unity scenes.

That could be a cool feature, but currently, Holodeck only supports text inputs.

TREEANA commented 1 month ago

I successfully ran the olodeck/main.py --query "a living room" command and loaded the scene in Unity. Now I want to command a complex query, but I have no idea what I should do. Can you give me an example query? For example, if I want to create an office with a round table in the center, several computer monitors and computer towers placed under and beside the desks, and desks and chairs arranged in a row along the windows, how should I command it?

You can just try python holodeck/main.py --query "an office with a round table in the center, several computer monitors and computer towers placed under and beside the desks, and desks and chairs arranged in a row along the windows" --single_room True --openai_api_key <OPENAI_API_KEY>

Since the query is long and complex, some of the requirements may not be satisfied in the generated scene.

I used the above query, but the code got stuck when it came to selecting appropriate objects for the office.

Screenshot from 2024-06-27 11-03-29

I waited for 3 hours but there is no response.

Could you help me?

YueYANG1996 commented 4 weeks ago

Could you try "an office"? If it still stuck, it may be because of the multiprocessing issue, and you can refer to this solution.