berndbischl
commented
4 years ago UMSTRUKTURIERUNG
Train Error und Test Error eigener Chunk LDA / QDA eigener chunk Naive Bayes eiegener Chunk NACH LDA / QDA
ML Basics und Terminologie (anhand von Regression) Intro to ML (recht allgemein, 5 min, “Black box modelling” raus, nach 2 cultures verschieben) Einschränken Subset von ML: Supervised Learning (Begriffe: Daten, Features, Label/target), KEIN Modell Ein paar nette Regr-Task-Beispiele zur Illustrierung (war vorher d1slot2) Was ist ein Learner und ein Modell? + H-R-O Prinzip Risk Minimization Losses for Regression (L2, L1) LM (als erster volles Bsp aller dieser Konzepte) Polynomial Regression KNN Regression & Distance Measures
ML Basics Klassifikation Der Klassifiation Task, und Classif-Task-Beispiele (war vorher d1slot2) und Classif-Terninologoie: DiscrFunctions, Class boundaries Linear classifiers Discriminative Approach and Losses for Classification (01, Brier, LogLoss) LogisticFunction und LogReg SoftmaxFunction und SM-Regression KNN for classification Generative Approach LDA / QDA Naive Bayes (ausbauen?)
Eine "Meta" / Philosophie Bereich / Vogelperspektive
Überblick über ML-Ontologie (Unsupervised, Supervised, semisupervised) ML as BlackBox Stats vs ML
Performance Estimation
Perf Estimation Intro, Konzept von Metriken und Inner vs Outer Sagen dass alle bisherigen Losses auch gehen (ohne alles nochmal zu machen) R2 Confusion MatrixConcepts of Misclassif-Costs Train Error Test Error Bias and Variance of Train and Test Error Estimator ROC Motivation, binaryclassif, imbalancy and costs ROC Metriken FPs, FNs, abgeletete ROC maße, F1 ROC Space (for Labeling Classfiers) Scoring classifiers und ROC curve, AUC und pAUC Resampling and Subsampling Bias Variance of Subsampling Crossvalidation Practical Hints for Resampling (incl Stratification)
Model Complexity, Curse of Dim and Overfitting Curse of Dim and Highdim Data Overtitting Regularisierung Bias Variance
Day4:
TREE Hypo Raum (4 slides) mit dem Regr Bild Tree Split Criteria (Regression, Classification), ca 6 slides TREE Growing - ca 7 slides Exhaustive search over features, splits Stop Criteria Pruning Cat Features advantages/disadvantages of trees - 5 slides Invariance to monotone trans Missings
BAGGING (d4-slot3, 3slides + ) Analyse Bagging Varaince RF DEFINITION (d4-slot3, 8 slides): Definition RF Random feature sampling OOB errors RF VarImp RF Discussion advantages/disadvantages of RF Benchmark RF vs CART vs KNN vs bagged CART vs bagged KNN
Tuning Intro und Problem - 6 slides Tuning Algos - 5 slides Grid Search Random Search Name-Dropping of Advanced Techniques
Pipeline Evaluation and Nested Resampling Why does "optimizing the test error" fail in terms of unbiased evaluation? 5 slides untouched test set principle + train validation test - 3 slides tuning as part of model building + nested resampling
HeidiSeibold
BITTE DIE BEIDEN TOP POSTS EDITIEREN WENN MAN WAS ÄNDERT !!!
IST ZUSTAND
Day1:
TERMINOLOGY: (14 slides, d1slot1 teil 1) Definition ML Data: Features, Targets Tasks Supervised Learning
MODELS & LEARNERS: (4 slides, d1slot1 teil 2) Model = Learner + data Learner = Hypothesenraum + Risikofunktion + Optimization (H-R-0)
TASKS: (d1slot2, 15 slides) Supervised Regression&Classification Unsupervised, SemiSupervised
KNN: (d1slot3, 16 slides) Distance measures H-R-O of KNN
Risk Minimization and Losses for Regression (d1slot4, 8 slides) Loss Funktion Risk-Funktion Minimieren des Risikos um Parameter zu schätzen
Linear Models (d1slot5-teil1, 16 slides): H-R-O
Polynomial Features and -Regression (d1slot5-teil2, 7 slides)
Day2:
CLASSIFICATION BASICS: (d2slot1, 9 slides) Terminology Class boundaries Linear classifiers Generative (H-R-O) / Discriminative (H-R-O)
GENERATIVE CLASSIFICATION (d2slot2, 14 slides) LDA QDA NaiveBayes
LOGISTIC REGRESSION (d2slot2-teil1, 11 slides) Logistic / bernoulli loss Class prediction H-R-O
MULTICLASS/SOFTMAX (d2slot2-teil2, 4 slides) Softmax Redundant params H-R-O
2 CULTURES (d2-slot4, 15 slides) Definition: Data-Modeling Definition: Algorithmic-Modeling Prediction & Interpretability Comparison Terminology
Day3:
Perf Estimation Intro (d3slot1, 7 slides) types of perf estimation, overview concept of generalization error estimation inner vs outer loss
Simple Performance Metric (d3slot2, 10) MSE, MAE, R^2 MCE, ACC Confusion Matrix Concepts of Misclassif-Costs Brier, LogLoss
Train and Test Error (d3slot3a, 19 slides) Train Error Test Error Bias and Variance of both estimator
Overfitting (d3slot3b, 5 slides) Overfitting Tradeoff zwischen model complexity und overfitting
ROC1 (d3slot4a, 10 slides) binaryclassif, imbalancy and costs conf matrix, FPs, FNs abgeleitete ROC maße F1
ROC2 (d3slot4b, 18 slides) ROC Space Scoring classifiers und ROC curve AUC und pAUC
Resampling (d4slot5, 12 slides) Crossvalidation Stratification Bootstrap und Subsampling Bias Variance of Subsampling
Day4:
TREE LEARNER (d4-slot1, 3 slides): Binary trees Splitting rules Trees are products of step functions
TREE OPTIMIZATION (d4-slot1, 14 slides): Exhaustive search over features, splits Stop Criteria Split Criteria (Regression, Classification)
TREE DETAILS (d4-slot2, 16 slides): Invariance to monotone trans Efficient split search over nominal features Pruning Missing values advantages/disadvantages of trees
BAGGING (d4-slot3, 3slides)
RF DEFINITION (d4-slot3, 8 slides): Definition RF Bagging variance Random feature sampling OOB errors
RF DETAILS (d4-slot4, 9 slides) Variable importance: permutation-based, improvement-based advantages/disadvantages of RF Benchmark RF vs CART vs KNN vs bagged CART vs bagged KNN
Day5:
Tuning (d5-slot1, 11 slides) definition of tuning types of hyperpars Grid Search Random Search Name-Dropping of Advanced Techniques
Nested Resampling (d5-slot2, 14 slides) Why does "optimizing the test error" fail in terms of unbiased evaluation? untouched test set principle train validation test tuning as part of model building nested resampling