jbloomAus / SAELens

Training Sparse Autoencoders on Language Models
https://jbloomaus.github.io/SAELens/
MIT License
490 stars 127 forks source link

chore: updating training docs with tips / jumprelu #366

Closed chanind closed 2 weeks ago

chanind commented 2 weeks ago

Description

This PR updates the docs with some more tips for SAE training, adds jumprelu SAEs to the docs, and fixes some minor errors in the docs (missing params, inconsistent citation, etc...)

Type of change

Please delete options that are not relevant.

Checklist:

codecov[bot] commented 2 weeks ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 66.80%. Comparing base (998c277) to head (52f2dfb). Report is 2 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #366 +/- ## ======================================= Coverage 66.80% 66.80% ======================================= Files 25 25 Lines 3389 3389 Branches 434 434 ======================================= Hits 2264 2264 Misses 1005 1005 Partials 120 120 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

chanind commented 2 weeks ago

Merging as this is just a docs update which is pretty low-stakes. @jbloomAus if you have thoughts on this after merging we can fix in a follow-up PR.