marcotcr / lime

Lime: Explaining the predictions of any machine learning classifier
BSD 2-Clause "Simplified" License
11.64k stars 1.81k forks source link

Binary Indicator vs Output probabilities #615

Closed anisham197 closed 3 years ago

anisham197 commented 3 years ago

I see that the paper mentions that we can use a binary indicator for f(x) instead of a probability.

f(x) is the probability (or a binary indicator) that x belongs to a certain class

I do not see code examples where a binary indicator is used instead of a probability i.e. a model outputs only 0 and 1 labels instead of probabilities of the 2 classes

I am trying to understand how using a binary indicator vs probabilities would affect LIME's explanations? Is there a reason probabilities are preferred over binary indicators?

marcotcr commented 3 years ago

The way to do it would be to just reshape predictions into probabilities that are always 0 or 1. We prefer probabilities (when available) because they allow us to use the signal from changes in prediction probability that do not necessarily change the prediction (e.g. if you change P(1) from 1 to 0.6, that has signal that we can use for explanation, perhaps more than if you change from 0.51 to 0.49)