search

DavidUdell / sparse_circuit_discovery

Circuit discovery in GPT-2 small, using sparse autoencoding
MIT License
6 stars 1 forks source link

Remove the topologically incorrect edges in `grad_graph.py` #110

Closed DavidUdell closed 2 weeks ago

DavidUdell commented 2 weeks ago

E.g., there is no attn_to_res edge; attn only maps to mlp. Keep only the subset of the 36 possibilities for edges (36 with error terms) that topologically match GPT-2.

DavidUdell commented 2 weeks ago

res_to_mlp edges actually needed to be preserved for double-counting correction, then skipped at graphing time. Done.