WARNING:py.warnings:/opt/conda/envs/holodeck/lib/python3.10/site-packages/transformers/tokenization_utils_base.py:1601: FutureWarning: `clean_up_tokenization_spaces` was not set. It will be set to `True` by default. This behavior will be depracted in transformers v4.45, and will be then set to `False` by default. For more details check this issue: https://github.com/huggingface/transformers/issues/31884
warnings.warn(
Generating from an empty scene.
User: You are an experienced room designer. Please assist me in crafting a floor plan. Each room is a rectangle. You need to define the four coordinates and specify an appropriate design scheme, including each room's color, material, and texture.
Assume the wall thickness is zero. Please ensure that all rooms are connected, not overlapped, and do not contain each other.
Note: the units for the coordinates are meters.
For example:
living room | maple hardwood, matte | light grey drywall, smooth | [(0, 0), (0, 8), (5, 8), (5, 0)]
kitchen | white hex tile, glossy | light grey drywall, smooth | [(5, 0), (5, 5), (8, 5), (8, 0)]
Here are some guidelines for you:
1. A room's size range (length or width) is 3m to 8m. The maximum area of a room is 48 m$^2$. Please provide a floor plan within this range and ensure the room is not too small or too large.
2. It is okay to have one room in the floor plan if you think it is reasonable.
3. The room name should be unique.
Now, I need a design for a living room.
Additional requirements: I only need one room.
Your response should be direct and without additional text at the beginning or end.
AI: Here is the floor plan:
living room | walnut hardwood, satin | light beige drywall, smooth | [(0, 0), (0, 6), (6, 6), (6, 0)]
AI: The floor plan is valid. (Only one room)
User: I am now designing a living room. Please help me decide the wall height in meters.
Answer with a number, for example, 3.0. Do not add additional text at the beginning or in the end.
AI: The wall height is 2.5
User: I need assistance in designing the connections between rooms. The connections could be of three types: doorframe (no door installed), doorway (with a door), or open (no wall separating rooms). The sizes available for doorframes and doorways are single (1m wide) and double (2m wide).
Ensure that the door style complements the design of the room. The output format should be: room 1 | room 2 | connection type | size | door style. For example:
exterior | living room | doorway | double | dark brown metal door
living room | kitchen | open | N/A | N/A
living room | bedroom | doorway | single | wooden door with white frames
The design under consideration is a living room, which includes these rooms: living room. The length, width and height of each room in meters are:
living room: 6.0 m x 6.0 m x 2.5 m
Certain pairs of rooms share a wall: (exterior, living room). There must be a door to the exterior.
Adhere to these additional requirements: N/A.
Provide your response succinctly, without additional text at the beginning or end.
AI: Here is the doorway plan:
exterior | living room | doorway | double | dark brown metal door
living room | kitchen | open | N/A | N/A
living room | bedroom | doorway | single | wooden door with white frames
There are multiple ways between exterior and living room
living room or kitchen not exist
living room or bedroom not exist
User: Guide me in designing the windows for each room. The window types are: fixed, hung, and slider.
The available sizes (width x height in cm) are:
fixed: (92, 120), (150, 92), (150, 120), (150, 180), (240, 120), (240, 180)
hung: (87, 160), (96, 91), (120, 160), (130, 67), (130, 87), (130, 130)
slider: (91, 92), (120, 61), (120, 91), (120, 120), (150, 92), (150, 120)
Your task is to determine the appropriate type, size, and quantity of windows for each room, bearing in mind the room's design, dimensions, and function.
Please format your suggestions as follows: room | wall direction | window type | size | quantity | window base height (cm from floor). For example:
living room | west | fixed | (130, 130) | 1 | 50
I am now designing a living room. The wall height is 250 cm. The walls available for window installation (direction, width in cm) in each room are:
living room: north, 600 cm; east, 600 cm; south, 600 cm;
Please note: It is not mandatory to install windows on every available wall. Within the same room, all windows must be the same type and size.
Also, adhere to these additional requirements: Only one wall of each room should have windows.
Provide a concise response, omitting any additional text at the beginning or end.
AI: Here is the window plan:
living room | south | fixed | (240, 120) | 1 | 50
AI: Selecting objects for living room...
AI: The used floor capacity of living room is 6.4m^2, which is less than 80% of the total floor capacity 14m^2.
AI: Here is the object selection plan:
{'living room': {'sectional_sofa': {'description': 'large l-shaped sectional sofa in dark blue fabric', 'location': 'floor', 'size': [350, 250, 90], 'quantity': 1, 'variance_type': 'same', 'objects_on_top': [{'object_name': 'throw blanket', 'quantity': 1, 'variance_type': 'same'}, {'object_name': 'decorative pillow', 'quantity': 7, 'variance_type': 'varied'}, {'object_name': 'coffee table book', 'quantity': 3, 'variance_type': 'varied'}, {'object_name': 'small decorative tray', 'quantity': 1, 'variance_type': 'same'}], 'object_name': 'sectional_sofa'}, 'coffee_table': {'description': 'rectangular glass coffee table with a metal frame', 'location': 'floor', 'size': [150, 70, 40], 'quantity': 1, 'variance_type': 'same', 'objects_on_top': [{'object_name': 'coaster set', 'quantity': 1, 'variance_type': 'same'}, {'object_name': 'small plant', 'quantity': 2, 'variance_type': 'varied'}, {'object_name': 'magazine', 'quantity': 5, 'variance_type': 'varied'}, {'object_name': 'candles', 'quantity': 3, 'variance_type': 'varied'}], 'object_name': 'coffee_table'}, 'tv_stand': {'description': 'sleek, modern tv stand with storage compartments', 'location': 'floor', 'size': [250, 50, 70], 'quantity': 1, 'variance_type': 'same', 'objects_on_top': [{'object_name': '65 inch tv', 'quantity': 1, 'variance_type': 'same'}, {'object_name': 'decorative box', 'quantity': 3, 'variance_type': 'varied'}, {'object_name': 'gaming console', 'quantity': 1, 'variance_type': 'same'}, {'object_name': 'picture frames', 'quantity': 2, 'variance_type': 'varied'}], 'object_name': 'tv_stand'}, 'wall_shelf': {'description': 'minimalistic wooden wall shelf in a light finish', 'location': 'wall', 'size': [150, 20, 30], 'quantity': 4, 'variance_type': 'varied', 'objects_on_top': [{'object_name': 'framed photo', 'quantity': 5, 'variance_type': 'varied'}, {'object_name': 'small succulent', 'quantity': 3, 'variance_type': 'varied'}, {'object_name': 'novel', 'quantity': 5, 'variance_type': 'varied'}, {'object_name': 'decorative vase', 'quantity': 2, 'variance_type': 'varied'}], 'object_name': 'wall_shelf'}, 'painting': {'description': 'colorful abstract painting to add character', 'location': 'wall', 'size': [120, 180, 5], 'quantity': 3, 'variance_type': 'varied', 'objects_on_top': [], 'object_name': 'painting'}, 'armchair': {'description': 'comfortable armchair upholstered in neutral fabric', 'location': 'floor', 'size': [90, 90, 100], 'quantity': 2, 'variance_type': 'varied', 'objects_on_top': [{'object_name': 'side table', 'quantity': 1, 'variance_type': 'same'}, {'object_name': 'decorative pillow', 'quantity': 2, 'variance_type': 'varied'}], 'object_name': 'armchair'}, 'floor_lamp': {'description': 'elegant floor lamp with dimmable light', 'location': 'floor', 'size': [40, 40, 160], 'quantity': 1, 'variance_type': 'same', 'objects_on_top': [], 'object_name': 'floor_lamp'}, 'bookshelf': {'description': 'tall wooden bookshelf with open shelving', 'location': 'floor', 'size': [120, 30, 200], 'quantity': 1, 'variance_type': 'same', 'objects_on_top': [{'object_name': 'assorted books', 'quantity': 10, 'variance_type': 'varied'}, {'object_name': 'small decorative sculpture', 'quantity': 3, 'variance_type': 'varied'}, {'object_name': 'photo albums', 'quantity': 2, 'variance_type': 'varied'}], 'object_name': 'bookshelf'}}}
plan for living room: In designing the living room, I will start by placing the largest items, ensuring they are positioned at the edges of the room to maximize space and maintain an open feel. The sectional sofa will be a primary focal point, aligned against one wall.
Following that, the TV stand will be placed directly across from the sofa, also at the edge of the room. The coffee table will be centrally placed in front of the sofa, allowing easy access.
The armchairs will then be positioned around the coffee table, facing towards it to create a cozy seating arrangement. They will be placed at the edge but close enough to the coffee table for practical use.
The bookshelf will be positioned at a side wall of the room, and the floor lamp will be placed in a position to provide light to the seating area while remaining unobtrusive.
Here is the arrangement with the assigned constraints:
sectional_sofa-0 | edge
tv_stand-0 | edge | far, sectional_sofa-0 | in front of, sectional_sofa-0 | center aligned, sectional_sofa-0 | face to, sectional_sofa-0
coffee_table-0 | middle | near, sectional_sofa-0 | in front of, sectional_sofa-0 | center aligned, sectional_sofa-0 | face to, sectional_sofa-0
armchair-0 | edge | around, coffee_table-0 | near, coffee_table-0 | face to, coffee_table-0
armchair-1 | edge | around, coffee_table-0 | near, coffee_table-0 | face to, coffee_table-0 | right of, armchair-0
bookshelf-0 | edge | far, tv_stand-0
floor_lamp-0 | middle | near, armchair-0 | side of, armchair-0 | face to, armchair-0
Time limit reached.
Time taken: 30.187254667282104
Number of solutions found: 27
wall object constraint plan for living room:
wall_shelf-0 | above, sectional_sofa-0 | 160
wall_shelf-1 | above, sectional_sofa-0 | 160
painting-0 | above, wall_shelf-0 | 20
painting-1 | above, wall_shelf-1 | 20
painting-2 | above, wall_shelf-1 | 20
wall_shelf-2 | above, coffee_table-0 | 45
wall_shelf-3 | above, coffee_table-0 | 45
switch-0 | N/A | 120
Time limit reached.
Time taken: 5.017381429672241
[ERROR] Could not generate scene from a living room. Traceback:
Traceback (most recent call last):
File "/home/cvsp/jhkim/Holodeck/main.py", line 56, in generate_single_scene
_, save_dir = args.model.generate_scene(
File "/home/cvsp/jhkim/Holodeck/ai2holodeck/generation/holodeck.py", line 327, in generate_scene
scene = self.generate_small_objects(scene, used_assets=used_assets)
File "/home/cvsp/jhkim/Holodeck/ai2holodeck/generation/holodeck.py", line 226, in generate_small_objects
controller = self.small_object_generator.start_controller(
File "/home/cvsp/jhkim/Holodeck/ai2holodeck/generation/small_objects.py", line 288, in start_controller
controller = Controller(
File "/opt/conda/envs/holodeck/lib/python3.10/site-packages/ai2thor/controller.py", line 548, in __init__
self.start(
File "/opt/conda/envs/holodeck/lib/python3.10/site-packages/ai2thor/controller.py", line 1546, in start
self._start_unity_thread(env, width, height, unity_params, image_name)
File "/opt/conda/envs/holodeck/lib/python3.10/site-packages/ai2thor/controller.py", line 1269, in _start_unity_thread
raise Exception(message)
Exception: Unity process has exited - check ~/.config/unity3d/Allen\ Institute\ for\ Artificial\ Intelligence/AI2-THOR/Player.log for errors. Confirm that Vulkan is properly configured on this system using vulkaninfo from the vulkan-utils package. returncode=-11
To my understanding, json file is first generated then is used in unity to visualize the scene.
Then why does this error occur while generating the json file?
I am running the codes on a headless server.
Yunduoii
commented
1 month ago
Thanks for your great work!
Now I am encountering a following error:
To my understanding, json file is first generated then is used in unity to visualize the scene. Then why does this error occur while generating the json file? I am running the codes on a headless server.
Thank you