Compute metrics during evaluation, not during training - Githubissues

BlueBrain / morphoclass

Neuronal morphology preparation and classification using Machine Learning.

https://morphoclass.readthedocs.io

Apache License 2.0

8 stars 4 forks source link

Compute metrics during evaluation, not during training #48

Open FrancescoCasalegno opened 2 years ago

FrancescoCasalegno commented 2 years ago

Context

Currently we are computing all metrics during the model training. This is true both for:

Training of TMD and DL classifiers: https://github.com/BlueBrain/morphoclass/blob/35bbe615bbf3a0f085e2d2ca1ebcf67c4d95a868/src/morphoclass/console/cmd_train.py#L139-L141
Training of Morphometrics classifiers: https://github.com/BlueBrain/morphoclass/blob/35bbe615bbf3a0f085e2d2ca1ebcf67c4d95a868/src/morphoclass/console/cmd_morphometrics.py#L198-L200

Actions

[ ] Make sure that nowhere else in the code (e.g. during evaluation) we are assuming that the checkpoint.chk produced during training contains metrics scores.
[ ] Move the computation of metrics scores from training stage to evaluation stages.

See also

https://github.com/BlueBrain/morphoclass/pull/45#issuecomment-1115874293