Single token prompts seem to cause misbehavior again, at the caching dim activations stage

DavidUdell / sparse_circuit_discovery

Circuit discovery in GPT-2 small, using sparse autoencoding

MIT License

7 stars 1 forks source link

Single token prompts seem to cause misbehavior again, at the caching dim activations stage #119

Closed DavidUdell closed 1 month ago

DavidUdell commented 1 month ago

The fix from last time this happened apparently holds for the residuals case. It just fails at the first mlp or attn case it encounters. So, repeat that fix, whatever that was, for the other two cases in collect_acts.py, I think.