allenai / Holodeck

CVPR 2024: Language Guided Generation of 3D Embodied AI Environments.
https://yueyang1996.github.io/holodeck
Apache License 2.0
324 stars 30 forks source link

ValueError: Invalid commit_id: 6f165fdaf3cf2d03728f931f39261d14a67414d0 #5

Closed Nziner closed 9 months ago

Nziner commented 9 months ago

Hi. I'm trying to run a test prompt, I've fixed a few errors, but I can't fix this one anymore. Can you tell me what the problem is?

Logs:

(holodeck) PS D:\Work\AI\Holodeck> python main.py --query "a living room" --openai_api_key <key>
Generating from an empty scene.
User: You are an experienced room designer. Please assist me in crafting a floor plan. Each room is a rectangle. You need to define the four coordinates and specify an appropriate design scheme, including each room's color, material, and texture.
Assume the wall thickness is zero. Please ensure that all rooms are connected, not overlapped, and do not contain each other.
Note: the units for the coordinates are meters.
For example:
living room | maple hardwood, matte | light grey drywall, smooth | [(0, 0), (0, 8), (5, 8), (5, 0)]
kitchen | white hex tile, glossy | light grey drywall, smooth | [(5, 0), (5, 5), (8, 5), (8, 0)]

Here are some guidelines for you:
1. A room's size range (length or width) is 3m to 8m. The maximum area of a room is 48 m$^2$. Please provide a floor plan within this range and ensure the room is not too small or too large.
2. It is okay to have one room in the floor plan if you think it is reasonable.
3. The room name should be unique.

Now, I need a design for a living room.
Additional requirements: N/A.
Your response should be direct and without additional text at the beginning or end.

AI: Here is the floor plan:
living room | oak parquet, satin finish | warm beige textured wallpaper | [(0, 0), (0, 6), (8, 6), (8, 0)]
AI: The floor plan is valid. (Only one room)

User: I am now designing a living room. Please help me decide the wall height in meters.
Answer with a number, for example, 3.0. Do not add additional text at the beginning or in the end.

AI: The wall height is 2.7

User: I need assistance in designing the connections between rooms. The connections could be of three types: doorframe (no door installed), doorway (with a door), or open (no wall separating rooms). The sizes available for doorframes and doorways are single (1m wide) and double (2m wide).

Ensure that the door style complements the design of the room. The output format should be: room 1 | room 2 | connection type | size | door style. For example:
exterior | living room | doorway | double | dark brown metal door
living room | kitchen | open | N/A | N/A
living room | bedroom | doorway | single | wooden door with white frames

The design under consideration is a living room, which includes these rooms: living room. The length, width and height of each room in meters are:
living room: 8.0 m x 6.0 m x 2.7 m

Certain pairs of rooms share a wall: (exterior, living room). There must be a door to the exterior.
Adhere to these additional requirements: N/A.
Provide your response succinctly, without additional text at the beginning or end.

AI: Here is the doorway plan:
exterior | living room | doorway | single | wooden door with glass panels
There are multiple ways between exterior and living room

User: Guide me in designing the windows for each room. The window types are: fixed, hung, and slider.
The available sizes (width x height in cm) are:
fixed: (92, 120), (150, 92), (150, 120), (150, 180), (240, 120), (240, 180)
hung: (87, 160), (96, 91), (120, 160), (130, 67), (130, 87), (130, 130)
slider: (91, 92), (120, 61), (120, 91), (120, 120), (150, 92), (150, 120)

Your task is to determine the appropriate type, size, and quantity of windows for each room, bearing in mind the room's design, dimensions, and function.

Please format your suggestions as follows: room | wall direction | window type | size | quantity | window base height (cm from floor). For example:
living room | west | fixed | (130, 130) | 1 | 50

I am now designing a living room. The wall height is 270 cm. The walls available for window installation (direction, width in cm) in each room are:
living room: west, 600 cm; east, 600 cm; south, 800 cm;

Please note: It is not mandatory to install windows on every available wall. Within the same room, all windows must be the same type and size.
Also, adhere to these additional requirements: Only one wall of each room should have windows.

Provide a concise response, omitting any additional text at the beginning or end.

AI: Here is the window plan:
living room | south | fixed | (240, 180) | 3 | 90

AI: Selecting objects for living room...

AI: The floor capacity of living room is 9.606128045951507m^2, which is less than 70% of the total floor capacity 19.200000000000003m^2.

AI: Here is the object selection plan:
{'living room': {'sectional_sofa': {'description': 'cozy corner sectional sofa in a neutral tone', 'location': 'floor', 'size': [350, 250, 85], 'quantity': 1, 'variance_type': 'same', 'objects_on_top': [{'object_name': 'decorative pillow', 'quantity': 6, 'variance_type': 'varied'}, {'object_name': 'throw blanket', 'quantity': 2, 'variance_type': 'varied'}, {'object_name': 'sofa arm tray', 'quantity': 1, 'variance_type': 'same'}], 'object_name': 'sectional_sofa'}, 'coffee_table': {'description': 'elegant wooden coffee table with a glass top', 'location': 'floor', 'size': [140, 90, 45], 'quantity': 1, 'variance_type': 'same', 'objects_on_top': [{'object_name': 'coffee table book', 'quantity': 5, 'variance_type': 'varied'}, {'object_name': 'vase', 'quantity': 2, 'variance_type': 'varied'}, {'object_name': 'decorative tray', 'quantity': 1, 'variance_type': 'same'}, {'object_name': 'candle holder', 'quantity': 2, 'variance_type': 'varied'}, {'object_name': 'table clock', 'quantity': 1, 'variance_type': 'same'}], 'object_name': 'coffee_table'}, 'entertainment_unit': {'description': 'sleek, low-profile media console in dark wood finish', 'location': 'floor', 'size': [200, 50, 50], 'quantity': 1, 'variance_type': 'same', 'objects_on_top': [{'object_name': 'flat screen tv', 'quantity': 1, 'variance_type': 'same'}, {'object_name': 'speaker', 'quantity': 2, 'variance_type': 'same'}, {'object_name': 'gaming console', 'quantity': 1, 'variance_type': 'same'}, {'object_name': 'decorative figurine', 'quantity': 3, 'variance_type': 'varied'}], 'object_name': 'entertainment_unit'}, 'bookshelf': {'description': 'tall, modern bookshelf for storage and display', 'location': 'floor', 'size': [100, 35, 220], 'quantity': 2, 'variance_type': 'same', 'objects_on_top': [{'object_name': 'books', 'quantity': 30, 'variance_type': 'varied'}, {'object_name': 'decorative statue', 'quantity': 4, 'variance_type': 'varied'}, {'object_name': 'potted plant', 'quantity': 4, 'variance_type': 'varied'}, {'object_name': 'photo frame', 'quantity': 4, 'variance_type': 'varied'}], 'object_name': 'bookshelf'}, 'floor_lamp': {'description': 'modern arc floor lamp for ambient lighting', 'location': 'floor', 'size': [60, 160, 210], 'quantity': 2, 'variance_type': 'same', 'objects_on_top': [], 'object_name': 'floor_lamp'}, 'wall_art': {'description': 'a set of framed contemporary art pieces', 'location': 'wall', 'size': [100, 5, 80], 'quantity': 4, 'variance_type': 'varied', 'objects_on_top': [], 'object_name': 'wall_art'}, 'accent_chair': {'description': 'plush accent chair with ottoman', 'location': 'floor', 'size': [90, 85, 100], 'quantity': 2, 'variance_type': 'same', 'objects_on_top': [{'object_name': 'small cushion', 'quantity': 2, 'variance_type': 'varied'}], 'object_name': 'accent_chair'}, 'side_table': {'description': 'round side table with wooden finish', 'location': 'floor', 'size': [50, 50, 60], 'quantity': 2, 'variance_type': 'same', 'objects_on_top': [{'object_name': 'table lamp', 'quantity': 2, 'variance_type': 'same'}, {'object_name': 'coaster set', 'quantity': 2, 'variance_type': 'same'}, {'object_name': 'succulent plant', 'quantity': 2, 'variance_type': 'varied'}], 'object_name': 'side_table'}, 'console_table': {'description': 'stylish console table for the entryway area', 'location': 'floor', 'size': [120, 40, 85], 'quantity': 1, 'variance_type': 'same', 'objects_on_top': [{'object_name': 'decorative bowl', 'quantity': 1, 'variance_type': 'same'}, {'object_name': 'key holder', 'quantity': 1, 'variance_type': 'same'}, {'object_name': 'small sculpture', 'quantity': 1, 'variance_type': 'same'}], 'object_name': 'console_table'}}}
plan for living room: High-Level Design Strategy:

For this living room, the anchor object will be the sectional_sofa-0 as it is likely the largest and serves as a primary seating arrangement. It will be placed against an edge of the room to maintain spaciousness. The entertainment_unit-0 will be placed on the opposite edge, facing the sectional sofa, creating a focal point for a TV watching area.

Bookshelves will be next, placed on the edges of the room, and potentially aligned with each other if they are on the same wall or opposite walls. Floor lamps will be placed near the seating arrangement but not necessarily at the edge, to provide adequate lighting.

Accent chairs will be positioned to complement the sectional sofa and to help form a cozy conversation area. Side tables will be placed conveniently next to seats for practicality. The coffee table will be centered in front of the sectional sofa.

The console table will likely be placed at the edge, perhaps along the entryway or against a free wall, to be used for decorative purposes or to hold keys and mail.

Constraints for each object:

sectional_sofa-0 | edge

entertainment_unit-0 | edge | opposite side of, sectional_sofa-0

bookshelf-0 | edge | far, entertainment_unit-0

bookshelf-1 | edge | far, entertainment_unit-0 | aligned with, bookshelf-0

floor_lamp-0 | middle | near, sectional_sofa-0 | side of, sectional_sofa-0

floor_lamp-1 | middle | near, accent_chair-0 | side of, accent_chair-0

accent_chair-0 | middle | near, sectional_sofa-0 | side of, sectional_sofa-0 | face to, coffee_table-0

accent_chair-1 | middle | near, sectional_sofa-0 | opposite side of, accent_chair-0 | face to, coffee_table-0

side_table-0 | edge | near, sectional_sofa-0 | side of, sectional_sofa-0

side_table-1 | edge | near, accent_chair-0 | side of, accent_chair-0

coffee_table-0 | middle | near, sectional_sofa-0 | in front of, sectional_sofa-0 | center aligned with, entertainment_unit-0

console_table-0 | edge | far, bookshelf-0 | far, bookshelf-1
constraint type opposite side of not found
constraint type aligned with not found
target object accent_chair-0 not found in the existing constraint plan
target object accent_chair-0 not found in the existing constraint plan
target object coffee_table-0 not found in the existing constraint plan
constraint type opposite side of not found
target object coffee_table-0 not found in the existing constraint plan
constraint type center aligned with not found
Restricted license - for non-production use only - expires 2024-10-28
23115.605644969895 ['sectional_sofa-0', 'coffee_table-0', 'floor_lamp-1', 'bookshelf-1', 'bookshelf-0', 'accent_chair-0']
18132.002243386094 ['entertainment_unit-0', 'floor_lamp-0', 'accent_chair-1']
5820.6050074257455 ['side_table-0']
16432.162516276214 ['side_table-1', 'console_table-0']
744.4540891790868 ['sectional_sofa-0']
8400.881576689313 ['entertainment_unit-0', 'accent_chair-1', 'floor_lamp-1']
21252.50755235555 ['bookshelf-0', 'bookshelf-1', 'side_table-0', 'coffee_table-0']
27473.96301155191 ['floor_lamp-0', 'accent_chair-0', 'console_table-0', 'side_table-1']
21827.113736528365 ['sectional_sofa-0', 'accent_chair-1', 'bookshelf-0', 'console_table-0', 'bookshelf-1']
5512.623255668515 ['entertainment_unit-0']
11129.66245957536 ['sectional_sofa-0', 'console_table-0', 'side_table-0', 'entertainment_unit-0']
14200.367297194518 ['bookshelf-0', 'accent_chair-0', 'side_table-1']
1301.2994662075798 ['sectional_sofa-0', 'coffee_table-0']
6155.398221139094 ['entertainment_unit-0', 'floor_lamp-1']
22686.883763367954 ['bookshelf-0', 'bookshelf-1', 'floor_lamp-0', 'side_table-1']
28081.505856971613 ['accent_chair-0', 'console_table-0', 'accent_chair-1', 'side_table-0']
16516.25882716701 ['sectional_sofa-0', 'entertainment_unit-0', 'coffee_table-0', 'floor_lamp-1', 'console_table-0']
41927.035121512046 ['bookshelf-0', 'accent_chair-0', 'side_table-0', 'bookshelf-1', 'side_table-1', 'floor_lamp-0']
6297.339032398482 ['accent_chair-1']
1602.5572143065024 ['sectional_sofa-0', 'accent_chair-0']
16580.22562881109 ['entertainment_unit-0', 'side_table-1', 'floor_lamp-1', 'accent_chair-1']
16816.027275145367 ['bookshelf-0', 'floor_lamp-0', 'side_table-0']
6887.649013375247 ['bookshelf-1']
3255.831180858589 ['sectional_sofa-0', 'console_table-0']
9422.541787638722 ['sectional_sofa-0', 'console_table-0', 'bookshelf-0']
22060.3261003041 ['entertainment_unit-0', 'bookshelf-1', 'side_table-0', 'accent_chair-1']
3324.10569421375 ['sectional_sofa-0', 'side_table-1']
Number of solutions found: 10

wall object constraint plan for living room:
wall_art-0 | above, sectional_sofa-0 | 160
wall_art-1 | above, bookshelf-1 | 160
wall_art-2 | above, entertainment_unit-0 | 160
wall_art-3 | above, accent_chair-0 | 160
switch-0 | N/A | 120
Time limit reached.
Time taken: 5.012984991073608
Traceback (most recent call last):
  File "D:\Work\AI\Holodeck\main.py", line 95, in <module>
    generate_single_scene(args)
  File "D:\Work\AI\Holodeck\main.py", line 21, in generate_single_scene
    args.model.generate_scene(
  File "D:\Work\AI\Holodeck\modules\holodeck.py", line 199, in generate_scene
    scene = self.generate_small_objects(scene, used_assets=used_assets)
  File "D:\Work\AI\Holodeck\modules\holodeck.py", line 141, in generate_small_objects
    controller = self.small_object_generator.start_controller(scene, self.objaverse_asset_dir)
  File "D:\Work\AI\Holodeck\modules\small_objects.py", line 220, in start_controller
    controller = Controller(
  File "C:\Users\Subd1v\miniconda3\envs\holodeck\lib\site-packages\ai2thor\controller.py", line 540, in __init__
    self._build = self.find_build(local_build, commit_id, branch, platform)
  File "C:\Users\Subd1v\miniconda3\envs\holodeck\lib\site-packages\ai2thor\controller.py", line 1418, in find_build
    raise ValueError(
ValueError: Invalid commit_id: 6f165fdaf3cf2d03728f931f39261d14a67414d0 - no build exists for arch=Windows platforms=StandaloneWindows64
(holodeck) PS D:\Work\AI\Holodeck>
sunfanyunn commented 9 months ago

Have you run pip install --extra-index-url https://ai2thor-pypi.allenai.org ai2thor==0+6f165fdaf3cf2d03728f931f39261d14a67414d0

Nziner commented 9 months ago

Yes, installed

Nziner commented 9 months ago

I reinstalled everything. Same error. gurobipy==10.0.3. On version 11 there was a bug from a neighboring ticket.

YueYANG1996 commented 9 months ago

I haven't tested the code on Windows, I will let you know once I figure out how to fix it.

YueYANG1996 commented 9 months ago

Also, please pull the latest commit, which will use the DFS as the default layout solver.

YueYANG1996 commented 9 months ago

I just confirmed with the AI2-Thor team that Windows is not currently supported. Here are the requirements. Consider installing Ubuntu on Windows as an alternative.

TheWiselyBearded commented 9 months ago

@YueYANG1996, is it possible to run the script without visualizing it, strictly to obtain the json file of the scene configuration on windows alone?

If not out of the box, would you mind pointing me to where I must edit the code to achieve this desired outcome?

Also can run on Mac, but still looking to achieve same effect: output json, disregard scene view window generated.

YueYANG1996 commented 9 months ago

I don't think that's possible in Windows because the entire codebase is highly associated with AI2-THOR, which doesn't support Windows. The scene view window generated is for placing small objects on top of larger objects using the random spawn function. You could skip this step, so it won't show the window, but there will be no small surface object.

You could also take a look at the AI2-THOR controller function, maybe it is possible to turn off the display window.