search

ai-safety-foundation / sparse_autoencoder

Sparse Autoencoder for Mechanistic Interpretability
https://ai-safety-foundation.github.io/sparse_autoencoder/
MIT License
185 stars 39 forks source link

Fix the linear encoder's initialisation #121

Closed alan-cooney closed 11 months ago

alan-cooney commented 11 months ago

As we are using regular rather than leaky ReLu, this should not have an a parameter set.