search

mllg / batchtools

Tools for computation on batch systems
https://mllg.github.io/batchtools/
GNU Lesser General Public License v3.0
172 stars 51 forks source link

how to get memory usage reported? #117

Closed vjcitn closed 7 years ago

vjcitn commented 7 years ago

I am just transitioning to batchtools ... the getJobTable for my initial forays reports NA under memory. Do I need to set a parameter to get the memory consumption estimates recorded?

mllg commented 7 years ago

Sorry for the late response.

Indeed, you need to set the resource measure.memory to enable a heuristic based on gc() (https://mllg.github.io/batchtools/reference/submitJobs.html#note). It turned out that repeatedly calling gc() is quite expensive if you have many thousands of chunked jobs. Therefore this is disabled per default. You can enable it for all future registries by setting 

default.resources = list(measure.memory = TRUE)

in your config. I will try to support https://github.com/sosy-lab/benchexec in the future which gives more reliable measurements (i.e. allocations in compiled or threaded code).

vjcitn commented 4 years ago

Just wondering whether the comment on benchexec connection for batchtools has been followed up. Thanks.

mllg commented 4 years ago

Unfortunately, no, not yet.