allenai / ai2thor

An open-source platform for Visual AI.
http://ai2thor.allenai.org
Apache License 2.0
1.16k stars 215 forks source link

Cannot place EggCracked on a receptacles #302

Open jaejaywoo opened 5 years ago

jaejaywoo commented 5 years ago

Hi,

Is there some way that I can force some objects to be placed on a receptacle (i.e. EggCracked on Plate or EggCracked on Pan)? Some floor plans like FloorPlan28 works but for some other floor plans like FloorPlan1, I cannot make that happen.

Thank you.

winthos commented 5 years ago

This might be due to different receptacles having slightly smaller "receptacle zones" than others. If the plate or pan you are trying to put the cracked egg into is one of the smaller variants, there is a chance it will fail to be placed when using the PutObject action.

Could you provide more details like object coordinates, and the specific floorplan and UniqueID of objects that are failing so I can try and track down the cause of this?

jaejaywoo commented 5 years ago

Hi @winthos . Thanks for the response. Here are some details of the objects and the floor plan:

When I try to put the cracked egg on these two receptacles, it does not work in this floor plan (although some other floor plans do). Also besides this, there seems to be some issues with Bowl receptacle as well. For example, in the Floorplan1, I cannot place Apple and Tomato in the bowl unless they are sliced. I will also put the details of these objects as well just in case:

winthos commented 5 years ago

So looking at those examples, this seems to be working as intended. So all receptacle objects have a "receptacle box" zone that can allow objects to be placed in. Bowls and Plates, due to being circular, have boxes that don't encompass the entirety of their usable surface area. Because of this, using the PutObject action, which teleports the object to the target receptacle and places it perfectly flush with the surface, can often fail due to the unique shapes of Bowls and Plates.

Bowls additionally have collision around the curve of the bowl, which makes it difficult to place larger objects. This is why sliced fruit, which are very small, can fit more easily into bowls.

While the automatic PutObject action is limited by these colliders and bounds, you can attempt to drop a held item by using a combination of the MoveHand and DropObject actions to let an egg or apple simply fall into the bowl/plate using physics to resolve where it ends up.

jaejaywoo commented 5 years ago

Thank you for the response @winthos . So whether the certain object is Putdown-able on some receptacle using PutObject action depends on the physical shape of the receptacle. Is there a way to know which objects are Putdown-able to which receptacle without manually trying them all?