search

GabrielHoffman / variancePartition

Quantify and interpret divers of variation in multilevel gene expression experiments
http://gabrielhoffman.github.io/variancePartition/
60 stars 14 forks source link

BiocParallel #73

Closed nickhsmith closed 1 year ago

nickhsmith commented 1 year ago

Hi there, I don't have an issue per-say, but I saw your FAQ about BiocParallel giving you trouble, and it's the same error I am getting in my package. Error in serialize(data, node$con, xdr = FALSE) : ignoring SIGPIPE signal

I was wondering if you had solved it, or had found a good workaround besides just making it not parallelized.

Thanks in advance

GabrielHoffman commented 1 year ago

Hi Nick, Thanks for reaching out. BiocParallel uses forks to launch parallel processes. When the parent processes loses contact with the child process it forked, it can give that error. It is more likely as you increase the number of threads. It also happens if the children ask for too much memory, and the machine kills on of the children. For this reason, I always set the default BPPRAM=SerialParam() so the user can specify a number of threads with SnowParam() themselves. Once I see this error, I need to restart R before launching another parallel process.

Gabriel