search

jurgisp / memory-maze

Evaluating long-term memory of reinforcement learning algorithms
MIT License
129 stars 13 forks source link

Reducing FPS and Memory Usage #24

Open subho406 opened 1 year ago

subho406 commented 1 year ago

Hey, I am trying to use this environment with Sample Factory. My current setup is 12 CPUs, 80GB Memory, 1XA100 GPU. I'm training using Async PPO with a total of 24 parallel environment instances. However, I'm noticing extremely high environment step times and RAM usage, which is making it difficult for me to increase the number of environment instances beyond 24. Is there a way reduce the memory usage and the environment step times? Maybe the tricks used in the Sample Factory DMLab experiments to cache the level data might help? Thanks

jurgisp commented 1 year ago

That's strange, I have never noticed high RAM usage per environment. Are you sure it is the environment that is using up RAM, and not something else? The slowness of environments could be a side effect if something else is gobbling up the resources.

I suggest you try to benchmark the environment individually to confirm it is indeed the source of the problem.

If it were, it would be interesting to know, but in that case the issue would have to be raised with MuJoCo, since there is nothing resource-intensive that is happening in the memory-maze itself.