Support secrets

[andreasjansson]

Allow passing secrets to a model at boot time.

One use case:

• A model takes as input an audio URL
• That URL exists on Freesound, but in order to download it you need an API key
• The Cog model keeps the API key in memory and authenticates with Freesound to download the audio from the user's URL

[alastair]

btw, Freesound has a small s :)

[bfirsh]

This level of pedantry is very pleasing. Welcome to the community, @alastair. I think you'll fit in just fine around here. 😆

[nateraw]

Would love to see this feature prioritized :)

Some more real examples where folks would need this:

• I have a private model/files on the Hugging Face Hub needed by my model. To download at setup, I'd need my HF API token
• I have a model that does X and Y, then calls OpenAI API to leverage GPT-4. Requires OpenAI token
• I have a private model stored in cloud storage that I'd like to download securely at setup time. Requires auth creds at runtime
• I want to log my results of a training on Replicate to WandB. Requires my WandB API Key at training time.

We've seen folks do all sorts of things to get around this missing feature, almost all of which are security risks.

CC @zeke since we were chatting about this

[nateraw]

There's also a product-oriented question of how this feature is surfaced up...

• are the secrets set on a per-model basis? i.e. I push a model of mine and attach an API key for myself. this is what I assumed at first, most straightforward
• or, are they set on a per-user-per-model basis? i.e. I share a model that needs an API key. I set it one time for myself, and I can happily use the model. If others want to use it, they have to set their own secrets (once) and they can also happily use it, but with their own credentials. this is more complicated but perhaps nice