EmuKit / emukit

A Python-based toolbox of various methods in decision making, uncertainty quantification and statistical emulation: multi-fidelity, experimental design, Bayesian optimisation, Bayesian quadrature, etc.

https://emukit.github.io/emukit/

Apache License 2.0

605 stars 128 forks source link

Parallel objective evaluation tutorial #338

Closed ekalosak closed 3 years ago

ekalosak commented 3 years ago

Issue:

255

Description of changes: I've written a tutorial on how to run Bayesian optimization with a parallelized external evaluation of batched suggestions using Emukit & asyncio.

sample_plot_output

This PR is rebased on top of PR #337 to build on the updated notebooks/index.ipynb format introduced there.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

apaleyes commented 3 years ago

That's amazing, thanks for doing this! You had a very productive break @ekalosak

It seems that we have some issues with Travis at the moment. I think it's past time we moved to Github Actions anyway, so please hang in there while I am working on the transition. I promise to have a closer look at all the PRs once this is done, shouldn't take more than a few days.

codecov-io commented 3 years ago

Codecov Report

Merging #338 (7790915) into master (184da9f) will not change coverage. The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #338   +/-   ##
=======================================
  Coverage   89.40%   89.40%           
=======================================
  Files         123      123           
  Lines        4022     4022           
  Branches      459      459           
=======================================
  Hits         3596     3596           
  Misses        330      330           
  Partials       96       96

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 184da9f...7790915. Read the comment docs.

ekalosak commented 3 years ago

@apaleyes no rush, please let me know if there's anything to fix here. Thanks for your efforts maintaining emukit 💪

ekalosak commented 3 years ago

Looks like the issue in the tests is

FAILED integration_tests/emukit/notebooks/test_notebooks.py::test_notebook_runs_without_errors[Emukit-tutorial-sensitivity-montecarlo.ipynb]

which has nothing to do with the present PR. Is this a known flaky test? Or a known testing issue?

apaleyes commented 3 years ago

@ekalosak i think this PR conflicts with your other PR #337 , which is now merged. Can you rebase and bring this one up to date please?

ekalosak commented 3 years ago

@apaleyes rebased 👍 ,