search

allenai / ai2thor

An open-source platform for Visual AI.
http://ai2thor.allenai.org
Apache License 2.0
1.09k stars 210 forks source link

PutObject vs DropObject in WebGL template #1124

Open sabraaap opened 1 year ago

sabraaap commented 1 year ago

Hi,

We're keen to reuse the WebGL template for a project but noticed that the PutObject action is not implemented, but only the DropObject action. This is problematic when trying to place objects in receptacles (e.g. an apple in a bowl). Is there a specific reason for this behaviour, or would it be possible to enable the PutObject action?

Thanks!

winthos commented 1 year ago

The backend of the WebGL template uses some slightly different logic compared to the python controller. PutObject requires the input of a target sim object, either by objectId or by some sort of screen based raycast to get the coordinates of the surface of a valid receptacle object.

Because of this, passing in a target receptacle via objectId is not really an option without a major rework to the web template, and because the web client uses WASD for movement and mouselook for aiming, it is also difficult to guarantee a clear raycast while holding different sized objects, as it was never really designed to be used in that way.

I believe it would be possible to alter the WebGL logic to allow PutObject, but it would be a fairly major rework as the WebGL interface was designed to be just a quick way to navigate and get a feel for the scenes, rather than extensive data collection or experiments.