dilyabareeva / quanda

A toolkit for quantitative evaluation of data attribution methods.
https://quanda.readthedocs.io
MIT License
33 stars 0 forks source link

When to randomize model in model randomization test? #89

Open gumityolcu opened 4 months ago

gumityolcu commented 4 months ago

In the randomization test, we need to randomize the model and look at the correlation between randomized and unrandomized explanations. We have a correlation value for each test sample. We take the average, but do we randomize independently for each sample, or do we randomize once at the beginning? Does it matter? The source paper is not clear.