Custom hookpoints - Githubissues

EleutherAI / sae

Sparse autoencoders

MIT License

333 stars 46 forks source link

Custom hookpoints #12

Closed norabelrose closed 3 months ago

norabelrose commented 3 months ago

This PR is a minor refactor of the training code to support training SAEs on the outputs of arbitrary modules, not just the residual stream output.

Other minor changes:

Removing attn_implementation entirely since HuggingFace seems to do the right thing by default
Add pre-commit and Ruff formatting to the project
Remove jaxtyping