03 performance evaluation / outro

[berndbischl]

it would be nice to have a small wrap up lecture, thats summarizes a bit and also gives more hints for practice and lists common mistakes.

[berndbischl]

1) always compare against baseline models, and move from very simply to slightly more complex model

e.g.

• constant
• linear
• tree
• random forest

simple tuning, simple feature selection

[berndbischl]

2) visual analysis is always good. e.g ROC but also simple residual and error plots the more you can see the better numbers are mainly good if MANY models are involed

[berndbischl]

3. Often it is hard to pin down a SINGLE metric for evaluation. then look at multiple metrics

[berndbischl]

4. really summarize common mistakes

[ludwigbothmann]

still valid