search

askforalfred / alfred

ALFRED - A Benchmark for Interpreting Grounded Instructions for Everyday Tasks
MIT License
360 stars 77 forks source link

Question on using returned values from "va_interact" for leaderboard submissions #88

Closed soyeonm closed 3 years ago

soyeonm commented 3 years ago

Hello,

In line 583 of "thor_env.py", va_interact returns "target_instance_id". Can one use this "target_instance_id" (from the previous step or the current step) as part of input to the model that decides which action to take? I am asking whether this is allowed or not allowed in the submissions to the leaderboard.

Thank you all the time for your replies.

soyeonm commented 3 years ago

Also, "va_interact" uses event.metadata['lastActionSuccess'] inside itself, to decide what to return for "success". Can I use event.metadata['lastActionSuccess'] inside "step", which is a function called inside "va_interact"? I just want to use it to keep an approximate pose of an agent (not getting the pose from the metadata, but updating an approximate one by myself).

MohitShridhar commented 3 years ago

  1. No sorry. va_interact is only for executing your output. Nothing from that function should be used as input to your model. As stated in the rules “You are only allowed to use RGB and language instructions (goal & step-by-step) as input for your agents”.

  2. No sorry. As stated in 1. you can only use RGB and language instructions.