Open smart-fr opened 1 year ago
I could resolve via reducing the memory buffer size to 40000 samples as suggested by @EngrStudent in https://github.com/jonathan-laurent/AlphaZero.jl/issues/116#issuecomment-1140285523.
This raises the question: do you still have plans to add an option to store memory buffer samples on disk?
A large memory buffer should normally not cause GPU out-of-memory errors. Indeed, the memory buffer is stored on CPU RAM. What could cause a GPU OOM is using too large a batch-size but it is unlikely to be happening here if the problem only arises after ten iterations.
Therefore, I see two possibilities:
A good compromise for my game looks like batch_size=16
(yes, only 16) and mem_buffer_size=PLSchedule([0], [80_000])
. Any higher value for either variable (as well as both variables, obviously) leads to a GPU OOM, as curious as it seems.
Re: it is worth having a look (whether AlphaZero.jl is doing something suboptimal when batching training samples, making memory consumption depend on the total number of samples): how can I achieve this?
Unfortunately my agent seemed to stop improving despite training over more than 50 iterations, and I suspect it is because of a limited memory buffer size (80000) which prevents the MCTS from retaining enough experience, while the NN doesn't exactly compensate through "intuition".
If there is a chance that storing the samples on disk would allow a larger mem_buffer_size
, I am definitely interested!
Once again, I do not think storing samples on disk is going to solve any problem here since you are encountering GPU OOMs. Implementing this is also not a priority right now. If you really want this feature though, it should not be too hard to implement and this may even be a nice opportunity for a contribution. :-)
Thank you, I got that this shouldn't solve my problem.
Maybe I would rather have a look whether AlphaZero.jl is doing something suboptimal when batching training samples, making memory consumption depend on the total number of samples -do you have an intuition which would help me start?
Can you try and run some training where you restart Julia after each iteration? If the problem is related to memory leaks, then it should disappear. Otherwise, it will be pointing to the second category of problems I mentioned.
Will do and report here.
When the problem arises, it appears even after a fresh restart of Julia.
The 10th training iteration for my game crashes with the following error. This happens not only on my PC with a 16GB RTX3080 Laptop, but also on a cloud VM with a 40GB A100 GPU. Could be related to https://github.com/jonathan-laurent/AlphaZero.jl/issues/1#issue-585853693? I am not quite sure how to work around this one.