Automate hyperparameter optimization - Githubissues

wri-dssg-omdena / policy-data-analyzer

Building a model to recognize incentives for landscape restoration in environmental policies from Latin America, the US and India. Bringing NLP to the world of policy analysis through an extensible framework that includes scraping, preprocessing, active learning and text analysis pipelines.

Other

34 stars 9 forks source link

Automate hyperparameter optimization #73

Open thefirebanks opened 3 years ago

thefirebanks commented 3 years ago

[ ] Look for information on automatic hyperparameter tuning optimization and its viability for our project
[ ] Define hyperparameters to be optimized for
[ ] Test new methods like population-based optimization or bayesian optimization

Bayesian Optimization explained

Conceptually: https://towardsdatascience.com/a-conceptual-explanation-of-bayesian-model-based-hyperparameter-optimization-for-machine-learning-b8172278050f

Population Based Optimization

Concept + comparison with Bayesian: https://medium.com/distributed-computing-with-ray/hyperparameter-optimization-for-transformers-a-guide-c4e32c6c989b

Huggingface + Ray + W&B implementation of hyperparameter tuning

Blog detailing actual process: https://huggingface.co/blog/ray-tune

jordiplanescutxi commented 3 years ago

Fix training variability issue

jordiplanescutxi commented 3 years ago

Fix training variability issue