[Proposal] Download tracking, code snippets, and filtering on the Hugging Face Hub

[osanseviero]

Proposal

Hey there! It would be great to have better support for SAELens on the HF Hub and provide

• Automatic code snippets via a "Use in SAELens" button
• Download counts to know how many people are using the model
• A filter/tag that when clicked people can easily find all SAELens repos

The repo to add this is open source (huggingface.js). See here an example PR to add a new library

Checklist

• [ ] I have checked that there is no similar issue in the repo (required)

[jbloomAus]

@chanind would be curious for your thoughts on this / whether this is something you might have capacity to do :)

[chanind]

This looks like a cool idea! I guess we just need to open a PR like specified in that example to add sae-lens to the list of libraries? Seems doable!

I'm worried about how to specify the snippet though - it looks like we know the model ID, but to load a SAE you need to specify the sae_id, not just the model_id, which I don't think the snippet would have a way of knowing. Also we rename models for use with the SAE.from_pretrained() function. I think we'd need to make it so that if the user passes in a huggingface repo_id to SAE.from_pretrained(), we fallback to pulling from that huggingface repo? Then the snippet could look something like the following:

from sae_lens import SAE

sae = SAE.from_pretrained("blah/my_sae", <fill in sae ID>)[0]

thoughts?

[osanseviero]

Yes, it's just opening a PR as the example. Yes, I think that would work well :+1:

Maybe with these small changes?

from sae_lens import SAE

sae = SAE.from_pretrained("blah/my_sae", sae_id=<fill in sae ID>)

[xenova]

Hey all! 👋 Now that v3.14.0 is out, which means Gemma Scope works with .from_pretrained, it would be great to extend this to any SAELens model on the Hugging Face Hub.

This would entail:

• Moving away from a "single global .yaml file" (https://github.com/jbloomAus/SAELens/blob/main/sae_lens/pretrained_saes.yaml) to a per-model config, possibly specified in the root directory of each repo as config.yaml
• Allowing the user to specify the model_id instead of release.
• Adding functionality to save a per-model config file. e.g., sae.save_pretrained() or sae.save_config() could be good.

and would mean that:

• Whenever a new model comes out, you do not need to update the global yaml file
• Users can publish their own models without requiring a PR to this repo.

• Easier + auto-generated code snippets on the HF hub, like: .

  We will add this to huggingface.js as a follow-up to a previous PR (https://github.com/huggingface/huggingface.js/pull/826) which added SAELens as a supported library. This will also ensure download counts are correctly tracked for the model, like:

  

Let me know what you think @jbloomAus!

[jbloomAus]

I'm excited about this. @chanind will follow up with you to discuss.