search

davidbau / dissect

Code for the Proceedings of the National Academy of Sciences 2020 article, "Understanding the Role of Individual Units in a Deep Neural Network"
Other
301 stars 50 forks source link

Is it actually reasonable to use single-class accuracy? #4

Open AwesomeLemon opened 3 years ago

AwesomeLemon commented 3 years ago

Hi, I'd like to ask a question regarding your paper. I'm interested in Fig 2.C and Fig S4.a,b. If I understand correctly, there you remove 256 neurons least relevant to a specific class, and measure change in balanced single-class accuracy. However, whether balanced or unbalanced, that accuracy includes true negative rate which is dependant on performance on other classes, which would also be influenced by the removal of the neuron. Therefore this measure does not depend on a single-class performance, but rather on all the classes.

Consider this: since the neurons being removed are very likely relevant to those other classes, it is no wonder that their true negative rate suffers, which influences the single-class accuracy for the currently targeted class.

My question then is: am I missing something in my thought process? Maybe you have considered this and decided it's irrelevant?

Thanks in advance.

AwesomeLemon commented 3 years ago

To explain my concern more clearly in an example: Accuracy for class 1 includes as a term True negative rate for classess 2..365.

Suppose we remove a neuron that doesn't influence class 1 at all (completely irrelevant), but that is relevant for class 2.

Then true negative rate for class 2 changes, which influences the single-class accuracy for class 1.

But this influence is solely due to class 2, not class 1