rlworkgroup / garage

A toolkit for reproducible reinforcement learning research.
MIT License
1.89k stars 311 forks source link

Add option to Evaluator for recording videos #1862

Open avnishn opened 4 years ago

avnishn commented 4 years ago

This is definitely a nice to have -- It'd be nice if as a part of snapshots, we could have the option to record the policy being rolled out deterministically for a few rollouts, so that we can watch its behavior in our various environments. I haven't had much success using the out of the box tool that open ai gym provides for this, called the gym Monitor wrapper. It requires that all environments internally return a Done signal after max_path_length steps. Perhaps we can get around this by wrapping environments with the open ai gymtimelimit wrapper.

Alternatively someone could look at the stable baselines vec video recorder wrapper. It seems that they have some type of video recorder, however it doesn't work out of the box with garage since stable baselines has pinned versions of TF and other libraries necessary to garage.

ryanjulian commented 4 years ago

This is more relevantly a feature of evaluation, not the plotter. The current plotter will soon be moved to something like garage.envs.GymVisualizer anyway.