search

askforalfred / alfred

ALFRED - A Benchmark for Interpreting Grounded Instructions for Everyday Tasks
MIT License
352 stars 77 forks source link

Video annotation for generate trajectories #122

Closed RishiHazra closed 1 year ago

RishiHazra commented 1 year ago

Hi, quick question with the video saver in generate_trajectories.py. I wish to extend the script to record exact time-steps for the beginning and end of both low-level and high-level actions as performed in the video. I noticed that you are saving images after every action, making it easier to denote which image corresponds to which action. Can the same be done on the video?

Also, some extended trajectories require some wait-time for plan generation which gets added to the generated video and is undesirable. I was wondering if I can start the video saver after the plan generation is complete.

Thanks for your help!

MohitShridhar commented 1 year ago

@RishiHazra, if I remember correctly, the video saver in generate_trajectories.py saves a frame after executing every low-level action. If smoothed navigation commands like MoveAhead take tiny steps to move forward, every tiny step should be recorded as an image frame. Further, each frame will have a corresponding low-level and high-level action in the JSON file. So the video frames can't go any slower than this action-execution speed. Although, each subgoal includes some filler frames to help with generating smooth videos.

The planner is used in an open-loop manner, i.e. plan once at the start, and then execute blindly. So you shouldn't be seeing any extra video frames because of planning, because the video recording only starts with the execution phase. You might have come across some "filler frames" at the start and end, and before/after each subgoal; see the FAQ regarding this.

RishiHazra commented 1 year ago

Thanks for the clarification. Will go ahead and close the issue!