search

jeanne-s / Patchscopes

Implementation of the Patchscopes paper (arXiv:2401.06102)
1 stars 0 forks source link

Unable to reproduce paper results #1

Open ArgusK17 opened 6 months ago

ArgusK17 commented 6 months ago

Can you reproduce the results of the paper? I tried using my homemade patching tool but failed. Not even the case in Figure 1.

jeanne-s commented 6 months ago

Hi! Same for me but I'm not confident in the quality of my code honestly. Can I have a look at your tool ?

ArgusK17 commented 6 months ago

Sorry about the delay. Here is my tool. The core patching code is from another paper of Mor Geva, In-context learning create task vectores. I didn't use transformer_lens because it's too resource-intensive.

jeanne-s commented 6 months ago

Sorry for the delay ! I plan on rewriting my code using pytorch hooks instead of transformer_lens, I'll let you know what are the results but I guess I'll get the same thing as you. Did you consider writing to the authors ?

ArgusK17 commented 6 months ago

I want to be more confident in my results (that's why I search and find this repo). If you get the same results, perhaps we could write an email to the authors. I generally trust their work, as their previous projects have all been reproducible. I don't know why this one isn't. Maybe there are some tricks they omitted in the article.

jeanne-s commented 5 months ago

Hi ! Did you see they published the code ? Did you have time to look at it ?

ArgusK17 commented 5 months ago

Thanks for reminding! I'll check it out.

ArgusK17 commented 5 months ago

I've checked their code. Sadly, they do not provide the code for Figure 1, only for later experiments. And none of later experiments are like that demo.

As I reread their paper, I feel that their experimental setting is a little cheating. Take section 4.3, for example. It seems to me like taking a hidden activation (for example a h at layer 4 with token input 'Paris'), and using ICL in the target prompt to explain this hidden activation (for example, the result is 'the capital of France'). This does not necessarily imply that the result's information exists in the hidden activation. It may just be added by the scope itself after the patching is done.

Generally speaking, the results of their experiments can be reproduced, but I question their value.