search

DavidUdell / sparse_circuit_discovery

Circuit discovery in GPT-2 small, using sparse autoencoding
MIT License
7 stars 1 forks source link

Reduce RAM load in `contexts.py` #45

Closed DavidUdell closed 8 months ago

DavidUdell commented 9 months ago

The RAM load in contexts.py likes to kill the process. I successfully got this off the GPUs in the past, but there should be some way of cutting the tensors down to maximally manageable sizes, so that they don't accumulate and bottleneck token scaling.

DavidUdell commented 8 months ago

I don't think there's an easy improvement, actually, over in d_graphs, and contexts.py isn't a bottleneck.