Question about the implementation of saliency score.

lancopku / label-words-are-anchors

Repository for Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learning

MIT License

151 stars 13 forks source link

Question about the implementation of saliency score. #3

Closed ToheartZhang closed 10 months ago

ToheartZhang commented 10 months ago

Thanks for your great work! I notice that the gradient is first summed across the head dimension, and then applied to absolute calculation, which may not be consistent with Formula (1) in the paper. Is it intended?

https://github.com/lancopku/label-words-are-anchors/blob/eba87afbd5115f7e70df31de9821254e3d74421f/icl/analysis/attentioner_for_attribution.py#L94-L99

leanwang326 commented 10 months ago

sorry for this, this is a typo in our paper, and I will update it in arxiv. But we also tried first to apply abs and then use grad.sum(1), the results are similar. (so this choice is just arbitary)

ToheartZhang commented 10 months ago

Thanks for the explanation!