Should we consider adding bayesmark to the benchmarking suite

MLBazaar / BTB

A simple, extensible library for developing AutoML systems

https://mlbazaar.github.io/BTB/

MIT License

173 stars 41 forks source link

Should we consider adding bayesmark to the benchmarking suite #203

Open kveerama opened 4 years ago

kveerama commented 4 years ago

The bayesmark package is another wrapper hyper parameter tuning library. We can add this to our benchmarking suite. Per their documentation, they wrap around:

The builtin optimizers are wrappers on the following projects:

    HyperOpt
    Nevergrad
    OpenTuner
    PySOT
    Scikit-optimize

https://github.com/uber/bayesmark/

And we already benchmark against HyperOpt. Note that OpenTuner is a previous package developed at MIT in 2014.

kveerama commented 4 years ago

We have in the past tried Nevergrad. Alternatively, we can just add Nevergrad, Scitkit-optimize and PySOT individually.