Implement different latent dynamics models

maltemosbach commented 3 years ago

Different architectures can be used to implement the latent dynamics model for a model-based agent. Implement the following models in the subdirectories of models/dynamics_models. All dynamics models should inherit from the base class in latent_dynamics_model.py and implement the abstract methods. Implement the dynamics model:

As a fully deterministic recurrent neural network (e.g. RNN, LSTM, GRU)
As a fully stochastic state space models
As a recurrent state space model (RSSM) [Hafner et al. 2019]
Implement latent overshooting for the RSSM and SSM and evaluate its effect
Propose your own architecture to be used as a dynamics model and implement it

For all models you should implement an encoder e_t = enc(o_t), a decoder p(o_t | s_t), prior and posterior transition dynamics p(s_t | s_t-1, a_t-1) and q(s_t | s_t-1, a_t-1, e_t). Ideally, encoder and decoder are shared between all architectures and only the transition dynamics are implemented separately. You must also implement a reward model that maps form state to reward as it is needed by the planner. Further, in the PlaNet agent, you should create the appropriate functions to calculate your model loss, such that the parameters can be optimized. The only files that should really be relevant for you are algos/planet.py and the dynamics models you should implement in subdirectories of models/dynamics_models. Helpful repositories to get started are:

References

Hafner, D., Lillicrap, T., Fischer, I., Villegas, R., Ha, D., Lee, H. Davidson, J.. (2019). Learning Latent Dynamics for Planning from Pixels. ICML