xavierpuigf / virtualhome

API to run VirtualHome, a Multi-Agent Household Simulator
http://virtual-home.org
MIT License
454 stars 65 forks source link

Large number of unplaceable objects in scene #77

Open edessa opened 2 years ago

edessa commented 2 years ago

Hello,

I am attempting to load 15 random objects in the scene through the comm.expand_scene(graph, randomize=True, random_seed=seed, ignore_placing_obstacles=True) command, run for each object. I find that of the 15 objects, on average only 8 to 10 are successfully loaded, where the unsuccessfully loaded objects show up as unplaced in the message returned from the expand_scene command. I am making sure the loaded objects adhere to the arrangements specified in object_script_placing.json. Are there any more constraints when it comes to loading objects? A log of unsuccessfully loaded objects is shown below, shown as: success obj obj_parent relation message:

False FryingPan sink Inside {'unplaced': ['fryingpan.1000']} False sponge sink INSIDE {'unplaced': ['sponge.1001']} False sponge kitchencounter ON {'unplaced': ['sponge.1001']} False sponge sink INSIDE {'unplaced': ['sponge.1001']} False sponge bathroomcounter ON {'unplaced': ['sponge.1001']} False sponge dishwasher INSIDE {'unplaced': ['sponge.1001']} False sponge bathroomcabinet INSIDE {'unplaced': ['sponge.1001']} False plate kitchentable On {'unplaced': ['plate.1002']} False plate kitchencounter ON {'unplaced': ['plate.1002']} False plate microwave INSIDE {'unplaced': ['plate.1002']} False plate kitchencabinet INSIDE {'unplaced': ['plate.1002']} False plate cabinet INSIDE {'unplaced': ['plate.1002']} False plate sink INSIDE {'unplaced': ['plate.1002']} False plate coffeetable ON {'unplaced': ['plate.1002']} False plate dishwasher INSIDE {'unplaced': ['plate.1002']} False plate stove INSIDE {'unplaced': ['plate.1002']} False plate stove ON {'unplaced': ['plate.1002']} False FryingPan sink Inside {'unplaced': ['fryingpan.1003']} False FryingPan kitchencounter ON {'unplaced': ['fryingpan.1003']} False Cutting_board sink On {'unplaced': ['cuttingboard.1005']} False Cutting_board kitchencounter ON {'unplaced': ['cuttingboard.1005']} False food_onion kitchentable On {'missing_prefabs': ['foodonion'], 'unplaced': ['foodonion.1006']} False food_onion kitchencounter ON {'missing_prefabs': ['foodonion'], 'unplaced': ['foodonion.1006']} False food_onion microwave INSIDE {'missing_prefabs': ['foodonion'], 'unplaced': ['foodonion.1006']} False food_onion plate ON {'missing_prefabs': ['foodonion'], 'unplaced': ['foodonion.1006']} False food_onion stove INSIDE {'missing_prefabs': ['foodonion'], 'unplaced': ['foodonion.1006']} False food_onion stove ON {'missing_prefabs': ['foodonion'], 'unplaced': ['foodonion.1006']} False FryingPan floor Inside {'unplaced': ['fryingpan.1009']} False FryingPan kitchencounter ON {'unplaced': ['fryingpan.1009']} False cloth_napkin washingmachine INSIDE {'unplaced': ['clothnapkin.1011']} False plate chair On {'unplaced': ['plate.1012']} False plate kitchencounter ON {'unplaced': ['plate.1012']} False plate microwave INSIDE {'unplaced': ['plate.1012']} False plate kitchencabinet INSIDE {'unplaced': ['plate.1012']} False plate cabinet INSIDE {'unplaced': ['plate.1012']} False plate sink INSIDE {'unplaced': ['plate.1012']} False plate coffeetable ON {'unplaced': ['plate.1012']} False plate dishwasher INSIDE {'unplaced': ['plate.1012']} False plate stove INSIDE {'unplaced': ['plate.1012']} False plate stove ON {'unplaced': ['plate.1012']}

kswain55 commented 2 years ago

Hi! Seems to be an issue with the coordinates or overcrowding. If you give the coordinates and rotation a lower precision number (preferably ending with a 0), the issue could be solved since every reset causes the coordinates to changes slightly due to Unity Engine related factors. This in turn causes the relations to sometimes change or in other cases prevent objects from being placed in the environment. @xavierpuigf @MicroSTM please feel free to join in if you think it might be some other issue.

mullenj commented 1 year ago

@edessa Did you ever resolve this issue? I am having a similar one at the moment with the same unhelpful failure message when I try to place objects on a kitchen table.

kswain55 commented 1 year ago

@mullenj Hmmm, would it be possible to show some screenshots of the kitchen table, so I can replicate the issue. Thanks!!!

mullenj commented 1 year ago
image

@kswain55 Thanks for the help! It's the kitchen table from room 0. I've cleared off all the objects so collisions wouldn't be the issue. I've also tried a number of random seeds and removing the random arguments entirely. This sample is with 'salmon' and a set pre-fab but I've had the same issue with a number of objects including the candle, chef knife, dishwashingliquid, and painkillers. The only objects I've successfully been able to spawn on the table are milk and beer.

mullenj commented 1 year ago

Things seem to work fine with v2.2.4 but in version 2.3 I encounter this issue with almost every object.

MicroSTM commented 1 year ago

If you are using Linux, could you try this v2.3.0 executable: https://www.dropbox.com/scl/fi/jcaq5zu9yudrf4pdbifaq/linux_exec.v2.3.0.x86_64?rlkey=z0pbmm0droahes55a09o6j9hm&dl=0 I have been using this without problems

mullenj commented 1 year ago

Thanks @MicroSTM I've been running it in windows as I eventually will be using unity to add some custom objects but I'll spin things up on my ubuntu machine and try that executable.

mullenj commented 1 year ago

@MicroSTM Also, which version of ubuntu are you using? My machine is currently running 22 but I've seen reports that 22 isn't fully supported and I have been having some issues. Trying to figure out if the issues are on my end or not.

MicroSTM commented 1 year ago

I use 22.04

mullenj commented 1 year ago

@MicroSTM Sorry for the delay, I was away. I'm still running into the same issues with this executable on Ubuntu. However, I have no issues when using the v2.2.4 executable. I was hoping to use the procedural generation ability of v2.3.0 as some point so if you have any ideas on how to solve this or have some code that allows you to place salmon or painkillers on the table in scene 0, let me know so I can try it. Thanks for your help!