offline training?

[Robokan]

Thanks for getting this running on pytorch! That was my only stumbling block to using it.

I want to re-implement DayDreamer which used Dreamerv2 to train an offline agent on the Go1 robot. I see references to "offline" and "parallel" in the codebase. How do I use this to create 2 processes one that collects data from a single environment and one that takes that data and trains the agent on it?

[NM512]

Thanks for reaching out.

To satisfy that requirement, you'll need to implement multiprocessing between the training and rollouts. This differs from the existing parallel setup found in parallel.py, which focuses on parallelizing rollouts. I'm confident that Ray's Actors can satisfy that requirement by having one process collect data from the parallelized environments while another process trains the policy on the collected data.