Open matbun opened 2 months ago
Add a documentation page on how to optimize a model with hyper-parameter optimization.
Some references from Eric to be added at the bottom of the page:
Some of these videos are short, others longer. I found them all helpful at some point in the past. • Andrew Ng: https://www.youtube.com/watch?v=wKkcBPp3F1Y&t=4s • Geoff Hinton: https://www.youtube.com/watch?v=i0cKa0di_lo&t=683s • Dataroots: https://www.youtube.com/watch?v=hboCNMhUb4g • Paretos: https://www.youtube.com/watch?v=M-NTkxfd7-8
Papers covering some of the most popular HPO algorithms. You really don’t need to study them all closely to start with, but it’s probably good to read them at some point. • Hyperband: https://arxiv.org/abs/1603.06560 • ASHA: https://arxiv.org/abs/1810.05934 • BOHB: https://arxiv.org/abs/1807.01774 • GP and TPE in Bayesian opt: https://proceedings.neurips.cc/paper_files/paper/2011/file/86e8f7ab32cfd12577bc2619bc635690-Paper.pdf
Finally, I think this page in the Ray Tune docs is quite nice to get an overview of how HPO works in Ray Tune: https://docs.ray.io/en/latest/tune/key-concepts.html?_gl=1*1i6k6z8*_ga*ODM5ODM1MjI4LjE3MjY0Nzk2ODQ.*_up*MQ..*_ga_0LCWHW1N3S*MTcyNjQ3OTY4Ny4xLjEuMTcyNjQ3OTY4OS4wLjAuMA
Add a documentation page on how to optimize a model with hyper-parameter optimization.
Some references from Eric to be added at the bottom of the page:
Some of these videos are short, others longer. I found them all helpful at some point in the past. • Andrew Ng: https://www.youtube.com/watch?v=wKkcBPp3F1Y&t=4s • Geoff Hinton: https://www.youtube.com/watch?v=i0cKa0di_lo&t=683s • Dataroots: https://www.youtube.com/watch?v=hboCNMhUb4g • Paretos: https://www.youtube.com/watch?v=M-NTkxfd7-8
Papers covering some of the most popular HPO algorithms. You really don’t need to study them all closely to start with, but it’s probably good to read them at some point. • Hyperband: https://arxiv.org/abs/1603.06560 • ASHA: https://arxiv.org/abs/1810.05934 • BOHB: https://arxiv.org/abs/1807.01774 • GP and TPE in Bayesian opt: https://proceedings.neurips.cc/paper_files/paper/2011/file/86e8f7ab32cfd12577bc2619bc635690-Paper.pdf
Finally, I think this page in the Ray Tune docs is quite nice to get an overview of how HPO works in Ray Tune: https://docs.ray.io/en/latest/tune/key-concepts.html?_gl=1*1i6k6z8*_ga*ODM5ODM1MjI4LjE3MjY0Nzk2ODQ.*_up*MQ..*_ga_0LCWHW1N3S*MTcyNjQ3OTY4Ny4xLjEuMTcyNjQ3OTY4OS4wLjAuMA