Support more automl tuners

[kmax12]

Right now we only support searching for parameters using Skopt. We should support other methods like randomized search and other libraries

[jeremyliweishih]

https://github.com/hyperopt/hyperopt

Currently three algorithms are implemented in hyperopt:

• Random Search
• Tree of Parzen Estimators (TPE)
• Adaptive TPE

Hyperopt has been designed to accommodate Bayesian optimization algorithms based on Gaussian processes and regression trees, but these are not currently implemented.

All algorithms can be parallelized in two ways, using:

• Apache Spark
• MongoDB

[kmax12]

Here is a promising tuner library by Facebook: https://ax.dev

Has both bandit optimization for picking pipeline we should try, as well as bayesian optimization for the parameters.

We'd want to use either the service or developer API probably

[jeff-hernandez]

Recently, I came across this open source AutoML toolkit by Microsoft [1]. It can be used for hyper-parameter tuning as well. Here's a list of the supported tuners [2]. Just thought it looked interesting.

[1] https://github.com/microsoft/nni [2] https://github.com/microsoft/nni/blob/master/docs/en_US/Tuner/BuiltinTuner.md#built-in-tuners

[exalate-issue-sync[bot]]

Issue has been cancelled.

[exalate-issue-sync[bot]]

Issue has been cancelled.