cloudgravity
commented
2 years ago The below article has an interesting explanation evaluation section (3.2) Antwarg, L., Miller, R. M., Shapira, B., & Rokach, L. (2021). Explaining anomalies detected by autoencoders using Shapley Additive Explanations. Expert Systems with Applications, 186, 115736
Notable methods/metrics include:
- [1] propose three types of evaluations: (1) Application-grounded — real humans, real tasks, (2) Human-grounded — real humans, simplified tasks, and (3) Functionally-grounded — no humans, proxy task.
- [2] divided explanation effectiveness into five categories: mental model, task performance, trust assessment, correctability, and user satisfaction.
- [3] defined key concepts for measuring explanation of an AI system: the goodness of explanations, user satisfaction, users’ understanding of the AI system, the effect of curiosity on the search for explanations, user trust, and performance.
- [4] used Explicitness, Faithfulness and Stability for evaluation.
Other methods seem to compute explanations of explanations...
[1] Doshi-Velez, F., & Kim, B. (2017). A roadmap for a rigorous science of interpretability. Stat, 1050, 28. [2] Gunning, D. (2017). Explainable artificial intelligence (xai). Defense Advanced Research Projects Agency (DARPA), Nd Web. [3] Hoffman, R. R., Mueller, S. T., Klein, G., & Litman, J. (2018). Metrics for explainable AI: Challenges and prospects. ArXiv preprint arXiv:1812.04608. [4] Melis, D. A., & Jaakkola, T. (2018). Towards robust interpretability with self-explaining neural networks. In Advances in neural information processing systems (pp.7786–7795).
While reading one of the papers, I came upon the XAI desiderata from Hansen, L. K., & Rieger, L. (2019). Interpretability in intelligent systems–a new concept?. In Explainable AI: Interpreting, Explaining and Visualizing Deep Learning (pp. 41-49). Springer, Cham.
It lists the following "requirements" :
D1. Fidelity: the explanation must be a reasonable representation of what the system actually does. D2. Understandability: Involves multiple usability factors including terminology, user competencies, levels of abstraction and interactivity. D3. Sufficiency: Should be able to explain function and terminology and be detailed enough to justify decision. D4. Low Construction overhead: The explanation should not dominate the cost of designing AI. D5. Efficiency: The explanation system should not slow down the AI significantly.
It could have been great to evaluate for each work in the survey if it complies with these requirements.