search

kosukeimai / fastLink

R package fastLink: Fast Probabilistic Record Linkage
272 stars 48 forks source link

%do% versus %dopar% #31

Closed Weekend-Warrior closed 6 years ago

Weekend-Warrior commented 6 years ago

Does it make sense to use %do% for single core tasks and skip the creation of a single core cluster? This is a big speed up on your basic example. Also, I think it makes sense to use on.exit to close the cluster in case the intermediate operation fails to execute.

bfifield commented 6 years ago

Thanks, @Weekend-Warrior - this is a good catch. I think there are a few other places in the package where we should be doing this, which I'll add to this PR before merging it into the main branch.

Weekend-Warrior commented 6 years ago

Great, glad it was helpful!