ram usage increase overtime

[eric000888]

• [x] I have marked all applicable categories:
  • [1 ] exception-raising bug
  • [ ] RL algorithm bug
  • [1 ] system worker bug
  • [1n ] system utils bug
  • [ ] code design/refactor
  • [ ] documentation request
  • [ ] new feature request
• [1 ] I have visited the readme and doc
• [1 ] I have searched through the issue tracker and pr tracker
• [ ] I have mentioned version numbers, operating system and environment, where applicable: I modified R2D2_GtrXL config file for cartpole and apply to gym anytrading and found the RAM usage continue increase and after a couple of hours the system run out of memory(128G RAM total). Also i got a warning shows the input shape can not match the memory shape. The original cartpole run also show increased memory usage but the training finished before the system stall. I changed the replay buffer but that didn't work. ############################################################################################ xxx.pdf

Following is the config file i used.

[zjowowen]

Hi Eric, I think this may due to the misleading concept of "replay_buffer_size" in R2D2_GtrXL, which is actually the size of sequences rather than env steps. If use replay buffer size with 10^6, it contains 10^6 * sequence length env steps. For cartpole, the sequence length is 200. So it is a huge number. I test R2D2_GtrXL with your config on my PC. It seems that if use sequence length 25, and collect 10^6 sequences, the python process will take more than 100Gi memory for the data in the replaybuffer. You can use a small replaybuffer, I suggest.