search

ArthurConmy / Automatic-Circuit-Discovery

MIT License
156 stars 30 forks source link

IOI task setup #73

Open javiferran opened 1 year ago

javiferran commented 1 year ago

Hi! Very nice work, and thanks for sharing the code. In the paper you mention that, for the IOI experiment, you did not split the computational graph into the query, key and value calculations for each head (Appendix E). I see the default behavior is to consider them split, and I was wondering if there is a straightforward way to modify it so that I can reproduce your results. Setting tl_model.set_use_split_qkv_input(False) makes assertion in verify_model_setup() flag.

Thanks in advance.

Javier

ArthurConmy commented 1 year ago

Hey @javiferran , thanks for reaching out!

See https://github.com/ArthurConmy/Automatic-Circuit-Discovery/pull/76 . Hopefully TransformerLens will allow the changes soon!

I have some runs on at the moment and the different seed seems to mean my results look a bit better with a slightly lower threshold (e.g 0.04... here https://api.wandb.ai/links/remix_school-of-rock/asj1u03n ).

EDIT: I also believe that in the paper the figure used the remove_redundant flag which deletes nodes that don't have any important incoming connections. This leads to better looking output graphs but which often miss e.g negative heads

javiferran commented 12 months ago

Hi @ArthurConmy! Thanks a lot for this!