search

tmastny / leadr

https://tmastny.github.io/leadr/
MIT License
26 stars 2 forks source link

Thread safety #29

Open alexpghayes opened 6 years ago

alexpghayes commented 6 years ago

This is awesome! It would be cool if there were some thread-safe way to update the leaderboard to facilitate training multiple models in parallel. Not sure what the best way to go about that would be.

tmastny commented 6 years ago

This and other ideas prompted this very long and fruitful discussion in https://github.com/ropensci/drake/issues/376.

Basically I would wait until all the models are fitted before running board. Using this example:

model_plan <- drake_plan(
  data = get_data(),
  informal_look = inspect_data(data, mu = mu__, rep = rep__),
  bayes_model = bayesian_model_fit(data, prior_mu = mu__, rep = rep__)
) %>%
  evaluate_plan(
    rules = list(
      mu__ = c(3, 9),
      rep__ = 1:2
    ),
    trace = TRUE
  ) %>%
  gather_by(mu__, mu___from)

results_plan <- drake_plan(
  walk(target_3_bayes_model, board); file_out("leadrboard.RDS")
)

plan <- bind_plans(model_plan, results_plan)