search

broadinstitute / ssGSEA2.0

Single sample Gene Set Enrichment analysis (ssGSEA) and PTM Enrichment Analysis (PTM-SEA)
Other
230 stars 79 forks source link

ssGSEA2 error in HPC environments when available cores is not equal to total number of cores. #32

Open cwlkr opened 2 months ago

cwlkr commented 2 months ago

Hi,

I am using ssGSEA2 on a larger data set and therefore I am running it on a HPC-cluster with job scheduler (Slurm).

Issue

ssGSEA2 will crash with Error in socketAccept(socket = socket, blocking = TRUE, open = "a+b", : all connections are in use when the allocated number of cores with Slurm is smaller than the total available cores of the hardware.

This is because parallel::detectCores() does not respect the allocation of Slurm and returns the total number of cores available on the CPU.

In my opinion the expected functionality of using par=TRUE in ssGSEA2 would imply detecting the available number cores, as is the behavior of parallelly::availableCores()as opposed to parallel::detectCores().

Workaround

set spares to:

n_cores = 64  # number of cores, same as in  --cpus-per-task
spares = 2 # number of spares 
spares = (parallel::detectCores()-n_cores)+spares