amindadgar
commented
1 year ago Hello Sai, I've been investigating the LIME method for a while and I can answer some of your questions. I can give an answer to your second question as below
In terms of trustworthiness I haven't seen a metric that quantitatively measures it, but having other metrics such as fidelity can help us. Actually, the r2 score of the surrogate model can give us an idea of how fidel the surrogate model is to the black-box model, and having a high r2 score can represent that the surrogate model is very fidel to the black-box model.
In general, I cannot assume LIME method can generate trustworthy explanations because of two things
- First, it generates different explanations for a sample by running the method multiple times
- Second, It uses perturbed unmeaningful and out-of-distribution samples to generate its explanations
So I guess LIME method is not a highly trustworthy method to generate explanations.
For your third question, my answer is given below
Finding the best parameters in order to produce explanations with higher r2 score and more efficient, can't convince me of the reason why we need that (since LIME method is not a great fit for all works). And I've recently seen an article that better investigates it to find more stable results. The article is S-LIME: Stabilized-LIME for Model Explanation, and it investigates the best kernel_width and number of samples to produce in order to find more stable explanations (in terms of robustness it increased method's robustness and by that, we can somehow conclude that it can increase trustworthiness).
I highly encourage you to read the article mentioned above and also the article Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods since they are a great fit to your questions.
If you had any more questions, please don't hesitate to ask in this feed or contact me individually.
xyFreddie
Hello, Thank you very much for the package!
I'm attempting to find the best set of parameters for the local explanations, such as kernel_width and num_samples, in order to produce the local explanations in less time and with a good r2 score. Based on my observations, I have some questions:
Can we use the explanations.score to determine whether the explanations are meaningful or not? Should we consider the explanations untrustworthy if their R-squared score is low, as shown in the above figure, because the local surrogate model did not accurately approximate the black-box model?
If I understand correctly, I have to determine the best set of kernel_width and num_samples to get local explanations for a test instance with a good R-squared score and less runtime. Because each instance may have its own set of parameters that produces the best results, I may have to tune the parameters for each and every test instance. How can we find the parameter values that work for all test instances? For example, I used LIME to build a multi-class classification model on tabular data with 77 variables. It took 1.14 seconds to get local explanations for a single instance with num_features=77 and top_labels=1 and all other parameters set to default. I used Bayesian optimization to find the kernel_width and num_samples that maximize the R-squared score. Kernel width ranges from sqrt(77)0.1 to sqrt(77)1.0, and num samples range from 1000 to 5000 in the search space. It took 31 seconds to tune the parameters. When you add the time it takes to tune parameters and generate local explanations, each instance takes about 32 seconds. Is it possible to find a single set of parameters that produces explanations with a high score in a short amount of time for all test instances?