search

DavidUdell / sparse_circuit_discovery

Circuit discovery in GPT-2 small, using sparse autoencoding
MIT License
7 stars 1 forks source link

Through-looping building on prior graphed nodes, not on independently set nodes. #18

Closed DavidUdell closed 10 months ago

DavidUdell commented 10 months ago

Pickle the pygraphviz graph and build on it in-place, if it already exists.

DavidUdell commented 10 months ago

Actually, I think the way to do it is to specify layer.feature features in the YAML list. Then, the single run will iterate over those.

DavidUdell commented 10 months ago

I think I got recursive calls, expanding on a prior graph, down. The other half of this is that I want to be able to tee up recursive calls across a large layers slice--that's next, then.

DavidUdell commented 10 months ago

I had written much of the directed_graph scripts with this in mind. I just need to build out that shell of a structure, so that the directed_graph scripts will step, pairwise, down through an arbitrary passed layer list.

DavidUdell commented 10 months ago

Nearly done, I think!

It'd be nice to have runs through several layers take the graphed dims as the dims that will be continued, rather than sampling dims for each layer independently. That would make the progressive graphing a unified operation, instead of basically a set of independent operations that only link up by change.

DavidUdell commented 10 months ago

Feature done, but extensive debugging is probably needed after all these changes.