search

farizrahman4u / recurrentshop

Framework for building complex recurrent neural networks with Keras
MIT License
765 stars 218 forks source link

Meta-RL or feeding output *and* loss back into the input #27

Open jpeg729 opened 7 years ago

jpeg729 commented 7 years ago

Do you intend to work on meta-RL any time soon?

In order to better train an agent that interacts with a world, the basic idea involves building an RNN capable of keeping track of the particularities of its environment from one time step to the next. https://arxiv.org/pdf/1611.05763v3.pdf

This involves concatenating the last action and its reward to the inputs for the next time step.

I am interested in a simpler case which would only need me to add the last output and its associated loss to the input at each time step. And I can see that being particularly inefficient in Keras, not to mention that with timesteps set to 1 there can be no significant BPTT.

crosleythomas commented 7 years ago

I think this would be done by setting readout='pack'

see https://github.com/datalogai/recurrentshop/blob/master/recurrentshop/engine.py#L272

but there seems to be a bug in RecurrentShop right now because Keras backend does not have a pack function. @farizrahman4u was that meant to be a call to the Tensorflow backend?