jbloomAus / SAELens

Training Sparse Autoencoders on Language Models
https://jbloomaus.github.io/SAELens/
MIT License
380 stars 104 forks source link

[Question] Llama-3-8B-it-res model not showing up in the pretrained_saes.yaml file #302

Open CallterC opened 2 hours ago

CallterC commented 2 hours ago

Questions

Dear authors of the SAE Lens, Thanks for your amazing work. I saw on the Neuronpedia and Huggingface that the Llama-3-8B-IT SAE published by Julius Han is currently available. However, upon checking the pretrained_saes.yaml files, I don't see this SAE shows up in the file. I am wondering if it is possible to add this SAE to the list of supported SAE by SAE_lens.

CallterC commented 2 hours ago

Here is the huggingface library: https://huggingface.co/Juliushanhanhan/llama-3-8b-it-res It states that SAE do support this model, but I still got "Release Juliushanhanhan/llama-3-8b-it-res not found in pretrained SAEs directory."