Setting up model credentials programmatically?

[sahuguet]

At my firm, we are proxying all requests to LLM service providers; users are assigned keys we rotate keys frequently; these are then translated into OpenAI/Anthropic/etc. keys on the backend.

Having keys hardcoded in the .env file is not practical.

Is there a way of programmatically setting up keys for the various models inside the script itself?

[pelikhan]

Currently not aside from dynamically setting the environment variables when launching the CLI. The .env file is only used to populate the process.env object.

One could think of registering a function that reads secrets?

readSecret(id: string) => promise

We kind of have a similar support for the Azure providers where we fetch a token on demand.

[sahuguet]

And what variable would that function need to update to override the value from .env?

[pelikhan]

I've added a documentation section about skipping the '.env' file all together (https://microsoft.github.io/genaiscript/getting-started/configuration/#no-env-file). I wonder if this would be enough to unblock as you "just" need to have a wrapper that sets up the environment variables and launch genaiscript. (genaiscript optionally loads the .env file)

[sahuguet]

I ended up writing a zx script that fetches the credentials and update the .env file.

A more elegant option (for me) would be to be able to override these variables in the script section, e.g. using process.env.XXXX to retrieve the values.

script({
    OPENAI_API_KEY: process.env.OPENAI_API_KEY,   // or any arbitrary function
   ...
})

[pelikhan]

If you simply need to use process.env variable, you already get this for free since this is where GenAIScript loads the secrets (after optionally applying the .env file). So in your case, remove any .env file and set the process environment variables when lauching genaiscript. This is what I tried to document in the new docs.

(Looking for a better option for a javascript-based config)