Open cedricr opened 4 months ago
I'm having the same issue (using Linux Mint): terra
is doing processing from disk, when it there is easily enough available RAM to do load all the raster into memory. Thanks for showing how to do the cache flush @cedricr
Would be great to have a fix for this.
I’ve noticed that
mem_info(rast())
was reporting unexpectedly low numbers on my system, even with only the R session open. These figures are consistent with thefree
column of runningfree -g
in the shell, whereas I would expect to see the number in theavailable
column instead.As the free memory of a Linux system tends to go to 0 in normal use, it looks like a rather serious problem because terra will probably never process stuff in memory after the system has been in use for a while, as the memory caches are filled.
Ex:
and at the same time in the shell:
So in that case, I have 45 GB available, but terra only sees 33.
If I then flush the cache manually (as documented at https://linux-mm.org/Drop_Caches) with
free
now returns the same value for free and availableand so does mem_info:
I initially thought that setting
ram = memInfo.freeram + memInfo.bufferram
here: https://github.com/rspatial/terra/blob/e27f4e53e94fc829bbbf46472abe52d2b4151cf3/src/ram.cpp#L44-L45 could do the trick, but apparently it’s more complex than that, and according to this commit to the linux kernel, the best way would be to extract theMemAvailable
field from /proc/meminfo.