Notice that there could be two ways to compute the chance_agreement based on the splits.
Compute chance_agreement on each split, then compute mean of the results to get an "average chance agreement"
Concatenate all splits and then compute chance_agreement to get a "global chance agreement"
For the computation of the various metrics (e.g. accuracy, f1-score, ...) in the performance-table CLI, we are following approach 1.
However, for chance_agreement in this PR we follow approach 2.
Checklist
[x] This PR refers to an issue from the issue tracker.
(if it is not the case, please create an issue first).
Fixes #64
Description
With this PR, we compute
chance_agreementusing theground_truthsof the"splits"instead theground_truthof"all", which may be missing, i.e. the solution proposed in https://github.com/BlueBrain/morphoclass/issues/64#issuecomment-1130009203.Notice that there could be two ways to compute the
chance_agreementbased on the splits.chance_agreementon eachsplit, then computemeanof the results to get an "average chance agreement"splitsand then computechance_agreementto get a "global chance agreement"For the computation of the various metrics (e.g.
accuracy,f1-score, ...) in theperformance-tableCLI, we are following approach 1. However, forchance_agreementin this PR we follow approach 2.Checklist