Closed ARLundborg closed 2 years ago
Most likely fixed in progressr 0.9.0, which is on CRAN since end of September, cf. https://progressr.futureverse.org/news/index.html
You're also quite behind for the other "future" packages too, so I recommend updating them all.
Hi Henrik, thanks for the quick response, you're right, I'll be sure to update my packages a bit more often to avoid these blunders!
Consider running the following dummy code:
When I do this on my Macbook (with sessionInfo as seen at the bottom of this post) I get insane levels of RAM usage (upwards of 5 Gb used towards the end of the apply-function, scaling with the amount of calls to the progress bar). This memory will not release unless I explicitly
gc
(sometimes several times) and if I cancel the running of the apply function halfway through, the memory cannot be retrieved unless I restart R. The problem only occurs when running the progress bar with workers >1. If workers = 1 then no RAM issue occurs.Contrast this with the same operation without a progress bar:
which runs without any RAM issues whatsoever. I've created this example based on an identical issue that I was having with a
future_apply
run on a remote cluster.Am I abusing the
progressr
-package somehow or is there a memory problem somewhere?Here is my sessionInfo: