Open lgatto opened 7 years ago
I've been seeing weird issues with MulticoreParam hanging or throwing errors recently (on my laptop, not on Travis). I didn't have time to debug them, so I just switch to DoparParam (the one based on foreach backends) and that worked fine.
The problems are likely either a port not being available (use options(ports=...)
to specify an available port) or attempting to serialize very large objects.
Thanks @mtmorgan. I am looking into the ports suggestion, as there's no (large) serialisation. What ports, if any, are used in MulticoreParams
?
It chooses a random port in 11000-12000. Also, options(bphost="localhost")
overrides the default host choice Sys.info()[["nodename"]]
I have had a similar problem: my unit tests suddenly fail when I run them on Travis:
test_distance.R:19: error: whole genome is read correctly
'bplapply' receive data failed:
error reading from connection
1: calculateDistance(mmapprData) at /home/travis/build/kjohnsen/MMAPPR2/tests/testthat/test_distance.R:19
2: .runFunctionInParallel(chrList, .calcDistForChr, param = mmapprData@param) at /home/travis/build/kjohnsen/MMAPPR2/R/distance.R:32
3: BiocParallel::bplapply(inputList, functionToRun, ..., BPPARAM = bpParam) at /home/travis/build/kjohnsen/MMAPPR2/R/main.R:4
4: BiocParallel::bplapply(inputList, functionToRun, ..., BPPARAM = bpParam)
5: bploop(structure(list(), class = "lapply"), X, lapply, ARGFUN, BPPARAM)
6: bploop.lapply(structure(list(), class = "lapply"), X, lapply, ARGFUN, BPPARAM)
7: .recv1(cl, "bplapply")
8: tryCatch({
parallel:::recvOneData(cluster)
}, error = function(e) {
stop(.error_worker_comm(e, sprintf("'%s' receive data failed", id)))
})
9: tryCatchList(expr, classes, parentenv, handlers)
10: tryCatchOne(expr, names, parentenv, handlers[[1L]])
11: value[[3L]](cond)
And my examples then fail as well:
Error: 'bplapply' receive data failed:
error reading from connection
Strangely enough, it seems this problem went away when I set sudo: required
in my .travis.yml
file.
I recently experienced the following error running parallel code with
BiocParallel
on Travis:The error was reported here happened when running
MSnbase::quantify
that usesBiocParallel
. Registering aSerialParam
instead of the defaultMulticoreParam
fixes the issue and now builds fine on Travis. I raised the issue on Travis directly, and it was suggested that there might be something wrong inBiocParallel
.Any idea?