florianhartig / BayesianTools

General-Purpose MCMC and SMC Samplers and Tools for Bayesian Statistics
https://cran.r-project.org/web/packages/BayesianTools/index.html
115 stars 29 forks source link

Improve parallelisation documentation and handling #181

Open florianhartig opened 5 years ago

florianhartig commented 5 years ago

From the feedback that I'm getting, it seems that many people run into problems when going from serial to parallel settings. The problems vary, and are often generic R parallelisation problems that would have also occurred when trying a parApply, but still, I think it would be useful to go another time through our parallelisation handling and documentation, and see if things can be made easier for the user.

Subpoints of that (please add points):

evodevosys commented 5 years ago

It would be helpful to have an example of coverting a non-parallel-compatible likelihood function to a parallel-compatible one. For example, the documentation just says it needs to be able to operate on a matrix of proposals but it isn't clear (to a novice) what this matrix looks like - what does entry A[i,j] represent vs. A[j,i]?

florianhartig commented 5 years ago

Thanks for the feedback. I just added a few remarks in the help of createBayesianSetup. You will have to install the development version to see this, but it's nothing major.

The matrix is explained in the help - a matrix with parameters as columns (and the different model runs as rows)

There is an example for an external parallelisation in the vignette.

That being said, I would only recommend this if parallel = T does not work for you!

ashiklom commented 4 years ago

How would you feel about using future for parallelization? That will allow the user to control exactly which parallelization backend they would like to use (via environment variables or the plan function).

florianhartig commented 12 months ago

So, parallelisation is still confusing for users, see #249 - should improve documentation and UX. Will also consider future.