Closed wcarvalho closed 3 years ago
Even if we don't use it for training, this information seems useful for analysis
@wcarvalho we don't provide metadata for test scenes on purpose. This is a standard practice is most datasets/benchmarks to prevent users from exploiting the metadata instead of actually solving the challenge. ALFRED is a vision-language challenge where we ultimately want to train agents that purely operate with ego-centric vision and language instructions as input. So the test scenes are specifically reserved for assessing this.
However, for internal analysis, you can use the validation sets (val_seen an val_unseen), which should contain pddl_params etc.
thanks - that makes sense!
How come pddl_params are only available for the train data? Is there a way to know the task "item" + "receptacle" are for validation/test?