Support for new algorithms

sherpa-ai / sherpa

Hyperparameter optimization that enables researchers to experiment, visualize, and scale quickly.

http://parameter-sherpa.readthedocs.io/

GNU General Public License v3.0

333 stars 54 forks source link

Closed anirudhacharya closed 5 years ago

anirudhacharya commented 5 years ago

[x] Hyperband - https://www.sysml.cc/doc/100.pdf
[ ] Parzen-window estimator - as per https://github.com/sherpa-ai/sherpa/pull/27#issuecomment-473526343 hyperopt can be an option.
[ ] Particle Swarm Optimization

LarsHH commented 5 years ago

Asynchronous Successive Halving is implemented ( https://github.com/sherpa-ai/sherpa/blob/8d6029307d27a2cef953c0ca36f197f14d321815/sherpa/algorithms/successive_halving.py#L27 ) which according to https://arxiv.org/pdf/1810.05934.pdf outperforms Hyperband. ASHA can actually be used to do Hyperband by iterating over different settings, but I'll leave that to future work for now.

LarsHH commented 5 years ago

Looks like https://github.com/DEAP could be a good starting point for the particle swarm optimization

LarsHH commented 5 years ago