search

force-h2020 / force-bdss

Business Decision System general interface
BSD 2-Clause "Simplified" License
2 stars 2 forks source link

Explore a ScikitOptimizerEngine class to enhance optimization library #331

Open flongford opened 4 years ago

flongford commented 4 years ago

The scikit-optimize library provides a wide range of optimization algorithms, with an "ask and tell" interface, similar to Nevergrad.

Creating a BaseOptimizerEngine subclass that acts as an entry point to this library may therefore be beneficial. Whether this should be included as a core feature in the BDSS, or implemented as a plugin (like Nevergrad) should also be discussed.

sparsonslab commented 4 years ago

Probably should be core: is it likely you will need to extend beyond 2 or 3 optimizers?

It still might be an idea to separate out the multi-objective method (https://en.wikipedia.org/wiki/Multi-objective_optimization) from the underlying optimizer, where possible. One may want to "mix-and-match" the two. e.g. weighting ("scalarisation"), can be with any optimizer.

flongford commented 4 years ago

Probably should be core: is it likely you will need to extend beyond 2 or 3 optimizers?

The dependencies of scikit-optimize include scikit-learn, which is quite heavy. This feels like something we could introduce in a plugin (maybe convert the Nevergrad plugin to a general 'optimisers' plugin), and keep the BDSS as light weight as possible.

It still might be an idea to separate out the multi-objective method (https://en.wikipedia.org/wiki/Multi-objective_optimization) from the underlying optimizer, where possible. One may want to "mix-and-match" the two. e.g. weighting ("scalarisation"), can be with any optimizer.

Yes, the first step towards this would be #321. The weighted components could then even be turned into a mixin class that could be used with any BaseOptimizerEngine

sparsonslab commented 4 years ago

Would the user want to be able to use ask/tell AND weight-sampling, or just one or the other?

flongford commented 4 years ago

Would the user want to be able to use ask/tell AND weight-sampling, or just one or the other?

Let's not worry too much about the "ask and tell" for the time being - it's more of a means to an end. The feature that we would use it for would be to prime the optimizer with existing data sets. There may be a different way to do that, depending on the optimizer's API.

flongford commented 4 years ago

Let's keep this issue open for now, we may wish to make an IOptimizer interface for scikit-optimize in future (though I expect this would be implemented in a separate repo).