Closed Ax3man closed 6 years ago
Hi Wouter,
I think this is very useful. I have merged your commit and added you as a contributor. It would be great if you could do the same thing for phyloglm.
Best, Lam
May I tease you with the following alternative (disclaimer: I'm the author of future, future.apply, ...):
bootmatrix <- future.apply::future_lapply(as.data.frame(booty), boot_model)
bootmatrix <- do.call(rbind, bootmatrix)
i.e. replace lapply()
with future_lapply()
. Global variables should be taken of automatically.
Note also that user don't have to pass an argument parallel
; instead user choose what kind of backend they wish to use - the default is sequential. For instance,
library(future)
plan(multiprocess)
will use PSOCK cluster on Windows and forked process on Linux/macOS (utilizing the parallel package). If the user got access to more machines, they can use:
plan(clusters, workers = c("n1", "n2", "n2", "n2"))
or an HPC scheduler;
plan(future.batchtools::batchtools_sge)
Sure, it's easy to do it that way instead. Also, if external dependencies are an issue, one can put future.apply
in Suggests
instead.
I can change the implementation to future
, depending on what you prefer, @lamho86 & @cecileane.
@Ax3man I don't have a strong preference regarding this. It's up to you.
Following issue 10, I now finally had some time to implement this.
Can you please have a look whether you think this is worthwhile, and if you agree with the implementation? Specifically:
parallel
as anImport
.parallel
parameter.If you think this is useful, I can do the same for
phyloglm
.It was mostly easy, using a function with
lapply
rather than afor
loop. Additionally,minus2llh
now takes ay
values, since otherwise scoping rules dictate the use of the value ofy
at the point of creation of that function (during the normal model fitting).Results between this and the original function remain the same:
For larger bootstrap samples, and for larger trees in general, parallelization creates a significant performance benefit (as expected):
par
is the parallel runs (7 cores) andnew
is the non-parallel runs. Right panel is a small tree (50 species), left panel is a larger tree (500 species).(I included a run of the old code, with the
for
loop to demonstrate that performance is the same as the no-parallel default of the new function.)