BlueBrain / morphoclass

Neuronal morphology preparation and classification using Machine Learning.
https://morphoclass.readthedocs.io
Apache License 2.0
8 stars 4 forks source link

Compute chance agreement on 'splits' instead of 'all' #65

Closed FrancescoCasalegno closed 2 years ago

FrancescoCasalegno commented 2 years ago

Fixes #64

Description

With this PR, we compute chance_agreement using the ground_truths of the "splits" instead the ground_truth of "all", which may be missing, i.e. the solution proposed in https://github.com/BlueBrain/morphoclass/issues/64#issuecomment-1130009203.

Notice that there could be two ways to compute the chance_agreement based on the splits.

  1. Compute chance_agreement on each split, then compute mean of the results to get an "average chance agreement"
  2. Concatenate all splits and then compute chance_agreement to get a "global chance agreement"

For the computation of the various metrics (e.g. accuracy, f1-score, ...) in the performance-table CLI, we are following approach 1. However, for chance_agreement in this PR we follow approach 2.

Checklist