search

facebookresearch / torchbeast

A PyTorch Platform for Distributed RL
Apache License 2.0
738 stars 114 forks source link

Why doesn't the test code use LSTM? #35

Open MXD6 opened 2 years ago

MXD6 commented 2 years ago

Hello author! Why doesn't the test code use LSTM? model forward propagation code: if self.use_lstm and len(core_state) is not 0: The lstm state is not initialized in the test code: agent_state = model.initial_state(batch_size=1) lstm state is not passed in during inference: agent_outputs = model(observation) This will result in: Even if the parameter --use_lstm is used when training the model, LSTM is still not used in the test.

heiner commented 2 years ago

Hey MaXiaodong,

Thanks for your interest in TorchBeast!

You are right that the test code of monobeast doesn't support using LSTMs currently.

We are happy to accept a pull request that fixes that. Truth be told, we stopped using the test mode recently and instead observe the returns during training.

MXD6 commented 2 years ago

Thank you for your reply.