HighlanderLab / SIMplyBee

SIMplyBee R package extends AlphaSimR for simulating honeybee populations and breeding programmes
http://www.simplybee.info/
Other
0 stars 5 forks source link

Parallelise MultiColony functions #513

Open gregorgorjanc opened 1 year ago

gregorgorjanc commented 1 year ago

Most (all?) MultiColony issues are looping over colonies and applying a function to each colony, which can be easily parallelised since each colony is considered as independent.

Which R approach should we use for this?

janaobsteter commented 1 year ago

@gregorgorjanc , are you familiar with some of these approaches? If yes, could you just list them and then I can dig around a bit more? Thanks!

gregorgorjanc commented 1 year ago

There is a number of approaches - quick Google throws this out - I don't have much experience with this high-level parallelisation - I have been doing it via OpenMP in Fortran and I know that Chris is also using OpenMP at C++ (via Rcpp) level in AlphaSimR - say for meiosis and other parts.

https://nceas.github.io/oss-lessons/parallel-computing-in-r/parallel-computing-in-r.html

https://www.tutorialspoint.com/parallel-programming-in-r

https://www.r-bloggers.com/2017/08/implementing-parallel-processing-in-r/

gregorgorjanc commented 8 months ago

Here is one more nice tutorial https://www.jottr.org/2019/01/11/parallelize-a-for-loop-by-rewriting-it-as-an-lapply-call

gregorgorjanc commented 8 months ago

HOWEVER, discusing with @janaobsteter revealed that we can not use these approaches because our code often touches and modifies the global SP object, so we can not use foreach() approach at all (that one runs jobs in separate threads that don't support working with the global objects):((((