search

aspuru-guzik-group / olympus

Olympus: a benchmarking framework for noisy optimization and experiment planning
https://aspuru-guzik-group.github.io/olympus/
MIT License
82 stars 22 forks source link

Using Olympus `Planner`-s to optimize SDL-Demo #17

Open sgbaird opened 2 years ago

sgbaird commented 2 years ago

I plan to adapt this notebook to one using the supported Planner-s in Olympus. Any interest in making a Jupyter notebook implementation PR in the notebooks directory using the simulator described in 2.2-sensor-simulator.ipynb? I'd then re-run this using the experimental demo and add co-authors where appropriate and if of interest. If that's asking a bit much, no worries, but figured I'd float the idea👍Planning to implement it one way or another, though it seems like it would be a lot more efficient for someone already familiar with the internals to work on it https://github.com/aspuru-guzik-group/olympus/issues/16

It would be interesting to make an interactive Plotly figure with legend groups or similar defined according to the categories in Planners. See https://github.com/sparks-baird/self-driving-lab-demo/issues/23 for additional context.

rileyhickman commented 1 year ago

Hello,

Would you be willing to include the SDL demo (at least the simulated one) as an Olympus dataset? This might make the implementation of this example even easier.

sgbaird commented 1 year ago

@rileyhickman I'd be happy to do that. Any idea how many pre-computed points you'd want?

rileyhickman commented 1 year ago

Well the more the merrier, but I imagine anything over 100 points would suffice. You could even just use the experimental points that you have already collected with your previous optimization experiments.

sgbaird commented 1 year ago

Here's a basic implementation of optimizing SDL-Demo using GpyOpt via Olympus https://colab.research.google.com/github/sparks-baird/self-driving-lab-demo/blob/main/notebooks/6.0-olympus-benchmarking-basic.ipynb based on the example you shared in #16. Still need to get the custom dataset ready.

sgbaird commented 1 year ago

Here's the error vs. iteration number: image

sgbaird commented 1 year ago

Any suggestions on an upper limit for the number of datapoints? Is 10k too much? (there are six adjustable parameters - three relate to the sensor settings)

EDIT: I have a file with ~40k datapoints across 7 parameters (~5 repeats per configuration due to noise) and 8 wavelength outputs