search

hill-a / stable-baselines

A fork of OpenAI Baselines, implementations of reinforcement learning algorithms
http://stable-baselines.readthedocs.io/
MIT License
4.16k stars 725 forks source link

Is it possible to add an auxiliary output to an algorithm? #1078

Closed mitchellostrow closed 3 years ago

mitchellostrow commented 3 years ago

Hi! I am trying to add an auxiliary output predicting a feature of the state vector onto an A2C agent with an LSTM actor critic policy. However, my efforts so far have come up short. How can I do this? Thanks in advance.

Miffyli commented 3 years ago

Hey. These issues on Github are mainly for bugs and enhancement purposes, not for technical support. The only way to go about it is to see where loss is created and trying to fit in your modifications there.

Also, if LSTM is not necessary, I suggest taking a look at stable-baselines3 which is done in PyTorch. It should be much easier to modify. You have better access to all elements involved in computing loss there, so you could fit in the modifications in that very same loop.