get_top_down_frame error

[ZheningHuang]

I have encountered the following issue, could you please provide some hints for debugging please.

Selecting 2 small knit blanket for armchair-0 (living room)
Selecting 2 small knit blanket for armchair-1 (living room)
Selecting 1 table clock for console table-0 (living room)
Selecting 3 framed picture for wall-mounted floating shelves-1 (living room)
Selecting 1 catch-all tray for console table-0 (living room)
Selecting 2 hourglass for wall-mounted floating shelves-1 (living room)
Only one room in the scene. Assigning the room to Procedural0.
Traceback (most recent call last):
  File "/scratch/WIP/Holodeck/main.py", line 95, in <module>
    generate_single_scene(args)
  File "/scratch/WIP/Holodeck/main.py", line 21, in generate_single_scene
    args.model.generate_scene(
  File "/scratch/WIP/Holodeck/modules/holodeck.py", line 233, in generate_scene
    top_image = get_top_down_frame(scene, self.objaverse_asset_dir, 1024, 1024)
  File "/scratch/WIP/Holodeck/modules/utils.py", line 80, in get_top_down_frame
    if all_edges_white(top_down_frame): break
  File "/scratch/WIP/Holodeck/modules/utils.py", line 18, in all_edges_white
    if not np.all(np.all(img[0, :] == white, axis=-1)):
ValueError: operands could not be broadcast together with shapes (1024,4) (3,)

Best, Zhening

[YueYANG1996]

Hi,

Could you add an interrupt here to check the dimension of the image? It seems the one you pass into the function has 4 channels, not sure why this happened, but you could add one line of code to convert it to RGB.

[ZheningHuang]

Thanks for the quick reply. I found out that the fourth channel is 255 (not sure why it occurred), but I dropped it for generating the image. However, I have the following issue which indicates that it did not pass the test:

Only one room in the scene. Assigning the room to Procedural0.
Error: no "view" rule for type "image/png" passed its test case
       (for more information, add "--debug=1" on the command line)

Any idea what caused this problem?

Also, is there any tip for regenerating the same scene with less consumption of OpenAI tokens (to save budget)?

Best, Zhening

[YueYANG1996]

Could you first try to comment this line to see if you still have the error? https://github.com/allenai/Holodeck/blob/db15e52238135b9595bbd344eadde81b8a1b25ff/modules/utils.py#L78

Then, you could add a line of code (e.g., matplotlib) to visualize the img here: https://github.com/allenai/Holodeck/blob/db15e52238135b9595bbd344eadde81b8a1b25ff/modules/utils.py#L13

If you can see the image, then use PIL to convert it to RGB.

[ZheningHuang]

I managed to generate the image now. Thanks.

BTW, this is really great work. Congrats!

[arjuntheprogrammer]

Hi, I also got the same error. I wanted to know why are we closing the issues before merging the fixes to the main branch?