Parallelize SMC routines

ArtPoon commented 7 years ago

Using function mclapply from R package parallel. Good place to start may be simulation of trees.

ArtPoon commented 7 years ago

It looks like it will be too difficult to parallelize at the level of simulating trees per particle, because we are calling a generic function that holds a reference to some other simulation routine - we can't modify all the different simulation routines that we will want to use. Instead, we should parallelize at the level of particles:

  # loop over particles
  # TODO: multi-threaded implementation
  for (i in 1:nparticle) {
    if (ws$weights[i] == 0) {

ArtPoon commented 7 years ago

We have to be careful about parts of the code where the workspace ws is being updated - if we call this within a paralellized function then we have to make sure the results are communicated to some main thread...

ArtPoon commented 7 years ago

Starting on new branch parallel

gtng92 commented 7 years ago

Yule Model

100 particles

nthreads = 1 Step 86 epsilon: 0.05 ESS: 90.34295 accept: 0.1010101 elapsed: 432.3 s nthreads = 10 Step 85 epsilon: 0.05 ESS: 97.50259 accept: 0.11 elapsed: 131.4 s

1000 particles

nthreads = 1 Step 98 epsilon: 0.05 ESS: 143.2098 accept: 0.09539474 elapsed: 2927.5 s nthreads = 10 Step 53 epsilon: 0.2143712 ESS: 66.28805 accept: 0.09473684 elapsed: 317.9 s I think the above was a fluke though, so I ran it again: Step 97 epsilon: 0.05 ESS: 145.3974 accept: 0.1277372 elapsed: 874.8 s and this time it makes more sense.

SIRD Model

100 particles

nthreads = 1 Step 96 epsilon: 0.1166726 ESS: 52.70801 accept: 0.1639344 elapsed: 10125.8 s nthreads = 10 Step 92 epsilon: 0.103304 ESS: 67.39651 accept: 0.1558442 elapsed: 1932.8 s

1000 particles

nthreads = 10 Step 92 epsilon: 0.1 ESS: 187.8923 accept: 0.3122924 elapsed: 11417 s

gtng92 commented 7 years ago

Merging branch parallel into master.

PoonLab / Kaphi