BLAS libraries use maximum number of available cores.

[sritchie73]

This has come up recently when diagnosing problems with one of the nodes on our cluster. Another user was using this MOFA R package, and that R process had spawned 64 threads even though the user had requested only 1 cpu on the cluster. We were able to solve this problem by restricting the number of threads available to BLAS/OMP.

This issue commonly arises when using matrix algebra code (e.g. SVD), which themselves call lower level matrix algebra libraries (e.g. Open BLAS) - as is the case in R.

I'm not familiar with this R package, but you can solve this problem by adding the following to the relevant R function that uses these libraries (e.g. any calls to svd(), or prcomp()):

library(RhpcBLASctl)  # needs to be added as an Imports: in the DESCRIPTION and NAMESPACE files

mofa <- function(..., nThreads) {
  # This code explicitly allows the user to control how many threads the matrix algebra calls can use,
  # then restores the previous state:
  oldOMPThreads <- omp_get_max_threads()
  oldBLASThreads <- blas_get_num_procs()

  omp_set_num_threads(nThreads)
  blas_set_num_threads(nThreads)

  # Restore to previous state when the function exits (either successfully or on error)
  on.exit({
    omp_set_num_threads(oldOMPThreads)
    blas_set_num_threads(oldBLASThreads)
  }, add=TRUE)
}

[QinqinHuang]

Hi Scott, FYI it’s the runMOFA() function.

[rargelaguet]

Hi Scott, thanks for posting the function, I will include it in the next release.