Open RomeoV opened 2 months ago
Sorry for the slow reply - this is probably a known memory leak, also reported offline by @mofeing in a similar case. I'll investigate and see if I can resolve it.
Using the example above, I've found the initial source of retained memory, and am fixing it in https://github.com/JuliaParallel/Dagger.jl/pull/558 (branch is very WIP, expect it to not work right now). I'll close this issue once that PR is merged, which should fully address this.
Allocating matrices on one worker and copying them to another worker repeatedly leads to a memory leak on my computer, and the Julia session being killed.
The
foo
matrix and its copy should be garbage collected, which I don't think they are. But even then, each matrix is 0.8GB, so if they exist 5 times on both workers we have 5 2 0.8GB=8GB of memory, which should not overflow my RAM. (I have at least 16GB free).Session is a clean
temp
project withDagger v0.18.12
, Julia 1.10.4,