search

holistic-ai / holisticai

Apache License 2.0
65 stars 10 forks source link

Explainability metrics #78

Closed Kleyt0n closed 8 months ago

Kleyt0n commented 10 months ago
rosepolle commented 10 months ago

Also, for lime, am I right in thinking that the contrast metrics are not included here? Is it "replaced" with the importance spread for features? I guess for each feature, you get their range and calculate the entropy of the ranks compared to if it was the same rank in each sample, which give you some sort of contrast metric?

Do we have demos of the effectiveness of these metrics? i.e. what are expected "good" values? Or can we only figure that out by comparing to a benchmark explainable model?

Finally, how does this metric compare with the contrast one in global? Do we have "equivalence"?