Centre-IRM-INT / GT-MVPA-nilearn

GT MVPA nilearn from Marseille
3 stars 3 forks source link

How to compare the performance of different decodings ? What is AUC vs accuracy ? #17

Open JeanLucAnton opened 3 years ago

JeanLucAnton commented 3 years ago

In the article The Journal of Neuroscience, March 14, 2018 • 38(11):2755–2765 • 2755, the authors compare the performance of different search-light decodings in each voxel : "Contrasting classifier performance. To compare performance across the different classifiers, subtraction analyses were performed contrasting classification by each cue type. For each contrast, whole-brain AUC maps were subtracted at the subject level. The resulting maps were tested against zero at the group level, using FSL Randomize to perform nonparametric one-sample t tests."

Two questions:

SylvainTakerkart commented 3 years ago

Two questions:

* Is it relevant to compare the value of two accuracies with a simple subtraction ? In particular, what about accuracies below the chance level ?

* Is what they call "Area Under the Curve" the same value as what we call accuracy ?