AI4LIFE-GROUP / OpenXAI

OpenXAI : Towards a Transparent Evaluation of Model Explanations
https://open-xai.github.io/
MIT License
227 stars 37 forks source link

`evaluator.evaluate` uses only single sample #29

Open enkeejunior1 opened 7 months ago

enkeejunior1 commented 7 months ago

Hi, and thank you for your awesome repository!

I recently tried using this library to validate some attribution methods, specifically by utilizing evaluator.evaluate.

However, I discovered that it only uses an attribution map from a single sample. link

        attrA = self.gt_feature_importances.detach().numpy().reshape(1, -1)
        attrB = self.explanation_x_f.detach().numpy().reshape(1, -1)

Is this behavior expected? The comment mentions evaluator.evaluate using n x m samples, so I am confused.