Bioconductor / BiocParallel

Bioconductor facilities for parallel evaluation
https://bioconductor.org/packages/BiocParallel
67 stars 29 forks source link

Use of DoparParam #97

Open mtmorgan opened 5 years ago

mtmorgan commented 5 years ago

(from @vjcitn) The BiocParallel vignette mentions foreach support and DoParParam ... but for package authors the steps of makeCluster, registerDo and stopCluster the vignette should indicate what a package author* foreach devotee would do to take advantage of BiocParallel

1) leave registration and any cluster construction to the user outside of package code 2) whatever cluster configuration is done is done using BiocParallel facilities, culminating in a DoParParam that the user manages and passes in via BPPARAM 3) ensure that default behavior is to run in serial mode

Ultimately it seems that the effects of switching to BiocParallel are only to remove code and ensure that BPPARAM is present and used properly. So it is a gain for simplicity and maintainability, but old habits die hard

LTLA commented 5 years ago

At least one current problem is that I always initialize the backend with bpstart at the top of functions that involve several parallelized steps, and the following fails:

library(BiocParallel)
BPPARAM <- DoparParam()
bpstart(BPPARAM)
## Error in (function (classes, fdef, mtable)  :
##   unable to find an inherited method for function ‘bpbackend’ for signature ‘"DoparParam"’

This should probably just be a no-op.

Session info ``` R version 3.6.0 Patched (2019-05-10 r76483) Platform: x86_64-apple-darwin17.7.0 (64-bit) Running under: macOS High Sierra 10.13.6 Matrix products: default BLAS: /Users/luna/Software/R/R-3-6-branch-dev/lib/libRblas.dylib LAPACK: /Users/luna/Software/R/R-3-6-branch-dev/lib/libRlapack.dylib locale: [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8 attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] BiocParallel_1.19.0.1 loaded via a namespace (and not attached): [1] compiler_3.6.0 parallel_3.6.0 codetools_0.2-16 iterators_1.0.12 [5] foreach_1.4.7 ```