search

rlworkgroup / garage

A toolkit for reproducible reinforcement learning research.
MIT License
1.86k stars 310 forks source link

Add "must have" features to make non-RL applications practical to write #2120

Open krzentner opened 3 years ago

krzentner commented 3 years ago

Hey, so people (including myself) are starting to use garage more for IL (and related topics). I have a short list of "must have" features which I think would make that much better. When I proposed them seperately in the past people have seemed a little hesitant on supporting them, so I figured putting them here in one place with a unified goal might make them more understandable.

  1. Add a Trainable class, which RLAlgorithm inherits from, and make Trainer use that interface (this is already 90% done, the Trainer already only requires that interface, but it's confusing and un-obvious).
  2. Allow setting the default X-axis with wrap_experiment, instead of locking it to TotalEnvSteps (pretty necessary for BC, I have a commit for it).
  3. Allow setting total_env_steps on Trainer, and make garage.Sampler have a total_env_steps property, which the Sampler is responsible for counting (I already have a commit for this). This allows much more flexibility to the algorithm about what sampling is for training, and what is for evaluation, etc.
  4. Add a Sampler class for sampling from a PathBuffer. This removes the complicated "source" concept that BC currently has (I also already have a commit for this).
ryanjulian commented 3 years ago

These all look great to me and I look forward to reviewing the PRs!

If you don't think they will all land around the same time, feel free to split this into 4 or more separate issues for tracking purposes.