Open olympiquemarcel opened 5 months ago
SusanneWenzel
commented
1 month ago @olympiquemarcel do you have a schedule or any other information I should add to the website?
Regarding your requirements above, please note that I cannot care about compute. Please consider applying for a HAICORE budget.
olympiquemarcel
commented
4 weeks ago Hi @SusanneWenzel
our updated schedule and requirements for running the code, would be nice if you could put this on the respective website.
Agenda: 13:30 - 14:15 General Introduction to Hyperparameter Optimization and Algorithms 14:15 - 15:45 Hands-On with Ray Tune Library 15:45-16:00 Break 16:00 - 16:30 Hands-On with Weights and Biases Library for Logging and Visualization of HPO 16:30-16:45 Discussion of Results
Participants should have access to Google Colab (https://colab.research.google.com/), as well as an account with Weights and Biases (https://wandb.ai/) for the hands-on part.
SusanneWenzel
commented
4 weeks ago olympiquemarcel
commented
3 weeks ago Link to jupyter notebook: https://colab.research.google.com/drive/1Tpse37fOTDKNUAny9coUiNdnQGBKttrY?usp=sharingL
Title
Tutorial on Efficient Hyperparameter Optimization for Machine Learning
Responsible person(s)
Marcel Aach (m.aach@fz-juelich.de), JSC and University of Iceland Xin Liu (xi.liu@fz-juelich.de), JSC
Format
Tutorial with presentations and hands-on sessions
Timeframe
Half day (3-4 hours)
Description
The performance of machine learning models is highly dependent on their hyperparameters that are set by the user before the training. The hyperparameters define the general architecture of the model (e.g., via the number of layers or the neurons per layer in a neural network) and control the learning process (e.g., via the learning rate of the optimizer or the weight decay). However, searching for optimal hyperparameter values is a long and resource-intensive process, as many different combinations need to be evaluated and the final performance of a combination can usually only be measured after a machine learning model is fully trained.
This tutorial presents a systematic introduction to the field of Hyperparameter Optimization (HPO) and demonstrates how to make use of resource-efficient methods to reduce the runtime of HPO in small and large settings on High-Performance Computing systems. Two HPO optimization libraries (Ray Tune and DeepHyper) are introduced, making use of evolutionary, Bayesian, and early stopping-based algorithms. As HPO is a general method and can be adapted to any machine learning model, it is useful for scientists from many different domains.
Requirements
Room with presentation equipment and enough power outlets for Laptops Participants should bring their own Laptops and be familiar with running code on a cluster Compute time reservation and accounts on a GPU-based system at JSC for the participants to run code during the hands-on sessions 5-20 participants
Agenda: 13:30 - 14:15 General Introduction to Hyperparameter Optimization and Algorithms 14:15 - 15:45 Hands-On with Ray Tune Library 15:45-16:00 Break 16:00 - 16:30 Hands-On with Weights and Biases Library for Logging and Visualization of HPO 16:30-16:45 Discussion of Results
Participants should have access to Google Colab (https://colab.research.google.com/), as well as an account with Weights and Biases (https://wandb.ai/) for the hands-on part.