allenai / ai2thor

An open-source platform for Visual AI.
http://ai2thor.allenai.org
Apache License 2.0
1.14k stars 215 forks source link

[Feature Request] Stacking of objects when using PutObject with object_id #632

Open aishwaryap opened 3 years ago

aishwaryap commented 3 years ago

Currently, PutObject called using object_id is very restrictive in terms of how many things can fit into a receptacle. For example, while trying to place plates in cabinets, in most scenes, only one plate can be placed in one cabinet. Would it be possible to enable some amount of stacking objects on top of each other (eg: allow multiple plates to be stacked in a cabinet or a plate to be placed over some forks in a sink)?

winthos commented 3 years ago

This issue has a similar explanation to #631, please refer to my response there for more details.

Basically, the PutObject action's logic is abstracted in such a way that stacking things like this is not possible via just the PutObject action. Stacking objects has a lot of physics resolution in order to determine where the final positions and orientations of the stacked objects would be.

aishwaryap commented 3 years ago

Does GetSpawnCoordinatesAboveReceptacle account for what the receptacle currently contains? For example, if there is a plate in the cabinet, will the z-coordinates returned be "above" the existing plate or will they be the positions available when the cabinet is empty?

winthos commented 3 years ago

GetSpawnCoordinatesAboveReceptacle' generates a set of points above the "receptacle zone" of a receptacle sim object. This makes them good initial points to try withPlaceObjectAtPoint` since placing an object at that point requires being able to pass in a point of space that is above some surface. It doesn't alter the height (y value) of these potential points based on what may already be inside of the receptacle, so if a tall enough object was already placed inside this receptacle and it happened to intersect with some of those generated points, the action would likely fail.