Selecting the best machine learning pipeline

[RLeenings]

Title

PHOTONAI

Short description and the goals for the OHBM BrainHack

INTRODUCTION

With photonai, we have designed a machine learning software that abstracts and condenses machine learning analyses to the most important design decisions, so that it can be applied from practitioners in medicine and the Life Sciences. As a part of that, we have automated the nested cross-validation and hyperparameter optimization loop. However, the optimal approach for choosing the most suitable hyperparameter configuration continues to be an open problem. For example, the hyperparameter configuration that performs best on the validation set might be overly specific to the particularities of the validation set and thus underperform in new applications.

GOALS FOR THE BRAIN HACK

• brainstorm strategies on how to select the best hyperparameter configuration to maximize generalization performance
• implement ensembling techniques, i.e. select several hyperparameter configurations

Link to the Project

github.com/wwu-mmll/photonai

Image for the OHBM brainhack website

No response

Project lead

Jan Ernsting, Github: jernsting, Discord: Jan#5471 Ramona Leenings, Github: rleenings, Discord: ramona_photonai

Main Hub

Montreal

Other Hub covered by the leaders

• [ ] Montreal
• [ ] Asia / Pacific
• [ ] Europe / Middle East / Africa
• [ ] Americas

Skills

Basic Python Basic Machine Learning or Statistic Knowledge

Recommended tutorials for new contributors

• [X] Set up a brainhack friendly computing environment
• [ ] A introduction to Bash
• [X] Python: Writing a script
• [X] Python: Data analysis with Python
• [ ] Python: Visualisation
• [X] Machine learning basics
• [ ] BIDS
• [ ] Machine learning for neuroimaging
• [X] VCS: Using Git and Github
• [ ] VCS: Data management with Datalad
• [ ] Virtualization
• [ ] High Performance Computing: Oracle cloud resource

Good first issues

No response

Twitter summary

No response

Short name for the Discord chat channel (~15 chars)

photonai

Please read and follow the OHBM Code of Conduct

• [X] I agree to follow the OHBM Code of Conduct during the hackathon

[ufangYang]

hi @bhvieira can you provide the content and/or figure for this project to be added to Brainhack Proceeding 2023 by 15th May? Also, please don't forget to fill out your team member's details using this form: https://forms.gle/Ku3AVHcXvpUqVVkV7 . Cheers