search

marcotcr / lime

Lime: Explaining the predictions of any machine learning classifier
BSD 2-Clause "Simplified" License
11.55k stars 1.8k forks source link

A question on how to measure the faithfulness of the explainer #626

Closed lk1983823 closed 3 years ago

lk1983823 commented 3 years ago

First, in the paper, it is mentioned: pp4. our choice of G (sparse linear models) means that if the underlying model is highly non-linear even in the locality of the prediction, there may not be a faithful explanation. However, we can estimate the faithfulness of the explanation on Z, and present this information to the user. Where can I find the faithfulness score in your code? Specifically, in your "Using lime for regression" tutorial, does the result of 

"Intercept 20.59284896606426
Prediction_local [36.39704149]
Right: 33.159500000000236"

shows the faithfulness of the explainer? Or the exp.score which is the R^2 value of the returned explanation, is the so-called faithfulness in your paper?

Second, from lime_base.py, I learn the " Right: 33.159500000000236" above, is from neighborhood_labels[0, label]. So I wonder why you use 0 instead of others or the average of more neighbors? Does this mean you only use the cloest neighbors?

Thanks for your answers.

marcotcr commented 3 years ago

shows the faithfulness of the explainer? Or the exp.score which is the R^2 value of the returned explanation, is the so-called faithfulness in your paper?

exp.score

Second, from lime_base.py, I learn the " Right: 33.159500000000236" above, is from neighborhood_labels[0, label]. So I wonder why you use 0 instead of others or the average of more neighbors? Does this mean you only use the cloest neighbors?

The first row is always the original example. Prediction local is the explanation's prediction on the original example, and 'right' is the model's prediction.