search

askforalfred / alfred

ALFRED - A Benchmark for Interpreting Grounded Instructions for Everyday Tasks
MIT License
360 stars 77 forks source link

Is there a way to know if "PutObject" will work before trying the action? #78

Closed soyeonm closed 3 years ago

soyeonm commented 3 years ago

Hello,

Thank you all the time for your kind responses. For other interactions (e.g. "PickupObject", "ToggleOn"), the objects that are 'visible' inside last_event.metadata["objects"] indicate whether it is interactable (e.g. Pickupable). In other words, when the agent tries to pick up an object that is visible, it does not fail.

However, for "PutObject" actions, even if the receptacle (e.g. "Table") is visible, the actions fails frequently. I realized that at most times, the agent is required to move closer to the receptacle beyond jus the 'visible' range.

Is there a way to know in advance whether the receptacle is within the interactable range of "PutObject" before actually trying "PutObject"?

Thanks.

soyeonm commented 3 years ago

So I am asking whether there is something that corresponds to "VisibilityDistance" for "PutObject" actions. (Since "PutObject" does not always seem to work even when the receptacle is visible.)

MohitShridhar commented 3 years ago

@soyeonm, I believe THOR has a "minimum interaction distance" for picking up and placing objects. I am not sure what exactly the threshold for this is. Maybe you can try asking here.

Given this threshold, you could potentially determine if the agent is close enough by using the 6-DOF poses in the metadata. But note that you can only do this during training time. During test time you need a leant object-pose and agent localization module to give you 6-DOF or 3-DOF estimates.

MohitShridhar commented 3 years ago

Closing due to inactivity.