What paper or reference is the RNN implementation trying to replicate?

[BFAnas]

• [x] I have marked all applicable categories:
  • [ ] exception-raising bug
  • [x] RL algorithm bug
  • [ ] documentation request (i.e. "X is missing from the documentation.")
  • [ ] new feature request
• [x] I have visited the source website
• [x] I have searched through the issue tracker for duplicates
• [x] I have mentioned version numbers, operating system and environment, where applicable:

  import tianshou, torch, numpy, sys
  print(tianshou.__version__, torch.__version__, numpy.__version__, sys.version, sys.platform)

  0.4.6.post1 1.10.0 1.21.2 3.9.10 | packaged by conda-forge | (main, Feb 1 2022, 21:24:11) [GCC 9.4.0] linux

Hi,

I think RNN as it is implemented right now is not working. I've been testing it with some algorithms, and these are the results I got:

• DQN on the environments:
  1. Cartpole it learns in less than one epoch, but if it is run for more time it "forgets"
  2. Lunarlander doesn't learn
  3. MountainCar also doesn't learn
• SAC on the environments:
  1. Ant-v3 doesn't learn
  2. HalfCheetah-v3 doesn't learn

Could you please refer me to the implementation you are trying to replicate?

[Trinkle23897]

There're known issues pointing out at #486. Unfortunately, I have no time to fix it until two months later (after graduation)...Feel free to submit PRs and I'm really sorry about that.