search

meggart / DiskArrayEngine.jl

MIT License
10 stars 1 forks source link

DiskArrayEngine crashes Julia with too much memory consumption #9

Open felixcremer opened 1 year ago

felixcremer commented 1 year ago

When I run the PyramidScheme fill_pyramids function with a 45000 x 45000 matrix the memory consumption of the julia process keeps slowely increasing until there is no RAM left on the computer and julia crashes. This happens with threaded=true. With threaded=false it seems as if the memory consumption stays roughly constant. This is currently running and I will post an update as soon as this run through or crashed as well.

meggart commented 1 year ago

Can you quickly dump the chunking structure of your 45000 x 45000 matrix, i.e. the result of eachchunk(matrix).chunks?