Open Vintarel opened 1 month ago
Without profiling, I suspect this is because the VRAM is not freed in time (because GC does not know about GPU memory space).
This creates memory pressure, and when it is too high, we manually trigger GC. But that is slow. And I think it just continues to build up.
This is a known problem and sadly there are no bulletproof solution for it at the moment.
One possible thing to try is to create in your project directory a LocalPreferences.toml
file with following:
[AMDGPU]
soft_memory_limit = "80 %"
hard_memory_limit = "80 %"
Thanks for the answer !
I've added this, changed nothing ; I then went down to "1%"
and it changed nothing, then I lowered again to "3 MiB"
where it started going even slower, because apparently GC is triggered everytime.
Now I see that the problem has been discussed many times... e.g. https://github.com/JuliaGPU/CUDA.jl/issues/137 So i guess I have no choice at the moment but to find a way to use large batches !
These memory limit parameters only control how soon the GC is triggered manually under-the-hood, so it won't help you avoiding GC calls. And here you can see that those GC calls are very wasteful.
Probably the best solution for this is to introduce Reference-Counting as a garbage collection mechanism to Julia (along with current GC mechanism), but that is not trivial to do (although some work has been done in that direction).
This small code:
is slower and slower to compute on my computer. Namely, the full code:
prints the following computation times:
so it goes from 3sec to more than 20sec, for the same part of code ! I checked the VRAM of my GPU card (with
cat /sys/class/drm/card1/device/mem_info_vram_used
) and is strictly increasing during the computation of the code above. Maybe this is the source of the problem ? But I'm unable to empty it.I tried many trivial things such as
finalize(input)
, but I was not able to solve the problem. The cpu version of it works well. Please help !The GPU card is
AMD Radeon RX 6700 XT
, I am on Manjaro, kernel 6.9.9-1.