Ray Out Of Memory Error

[kan-s0]

Describe the bug A clear and concise description of what the bug is.

To Reproduce python main.py --async --config config.r2d2.atari --env.name breakout python main.py --async --config config.muzero.atari --env.name qbert

Expected behavior RayOutOfMemoryError

Screenshots

Development Env. (OS, version, libraries): Linux python 3.7.11 jorldy:0.3.0

Additional context Add any other context about the problem here. https://stackoverflow.com/questions/60175137/out-of-memory-with-ray-python-framework https://github.com/ray-project/ray/issues/5572

It seems that GC for ray shared memory doesn't work properly.

[kan-s0]

It seems that the cause of this issue is not "RayOutOfMemoryError", but the size of the replay buffer is too large.

RayOutOfMemoryError occurs when learning with seq=20, n_step=3, burn_in=10 as in the config above.

However, it was confirmed that if only the buffer size was reduced from 2M -> 0.5M in the same config, it was learned without any problem.

When I measured the size of one transition stored in the buffer, it is approximately 5MB in size.

In the end, this issue seems to be caused by the size of the transition and buffer, so close it.