search

EmuKit / emukit

A Python-based toolbox of various methods in decision making, uncertainty quantification and statistical emulation: multi-fidelity, experimental design, Bayesian optimisation, Bayesian quadrature, etc.
https://emukit.github.io/emukit/
Apache License 2.0
591 stars 128 forks source link

implementing random search baseline #206

Closed aaronkl closed 5 years ago

aaronkl commented 5 years ago

It would be very convenient for developing and benchmarking to have a random search baseline that implements the OuterLoop interface such that, for example, it can easily be plugged into the benchmarking module A trivial way to do that would be to implement a dummy ModelUpdate and a new CandidatePointSelector which simply returns a random data point. What do you think?

marpulli commented 5 years ago

I agree this would be very useful and the way you suggest is sensible.

It might also be interesting to include some other, potentially stronger, baselines such as Sobol sequences or Latin Hypercubes. The implementation would be a bit more complex however.