search

Denys88 / rl_games

RL implementations
MIT License
819 stars 138 forks source link

what is self.mb_rnn_states #207

Closed 1tac11 closed 1 year ago

1tac11 commented 1 year ago

hi there, what does the value self.mb_rnn_states stands for? self.rnn_states is already set, but this value is updated when horizon episode is finished.

kind regards

Denys88 commented 1 year ago

self.rnn_states is the current rnn state before next step. mb_rnn_states are saved for training rnn states. For example we have 1 env with horizon_length 64 and seq_length 16. It means we will split 64 into 4 sequences with length 16 and we need to have some start rnn state - mb_rnn_states.