However, detectCores() returns all available cores on the machine, even if the process is not allowed to use them as is the case for, for example, cluster jobs that do per-core allocations (e.g. in our case a slurm cluster that uses cgroups for resource limitations).
the availableCores() function from the future package however takes resource allocations into consideration (see https://cran.r-project.org/web/packages/future/vignettes/future-1-overview.html). Would you consider switching to availableCores()? Or, alternatively, allow the size of the parallel cluster to be determined.
Hi,
in testing.r, you use
detectCores()to decide how many worker processes to use for parallel execution.https://github.com/humanlongevity/HLA/blob/34221ea2bb3e09177e237b33ba946404214cc270/bin/typing.r#L15
However,
detectCores()returns all available cores on the machine, even if the process is not allowed to use them as is the case for, for example, cluster jobs that do per-core allocations (e.g. in our case a slurm cluster that uses cgroups for resource limitations).the
availableCores()function from thefuturepackage however takes resource allocations into consideration (see https://cran.r-project.org/web/packages/future/vignettes/future-1-overview.html). Would you consider switching to availableCores()? Or, alternatively, allow the size of the parallel cluster to be determined.Wolfgang