Field for API key missing when using custom openai compatible API

[sammcj]

When you select to use a custom openai compatible API endpoint the API Key field disappears.

This makes using an openai compatible API impossible unless you have one that doesn't have an API key.

[nichochar]

Indeed we were not sure what the right abstraction was here. Do most of these services require only one API key? Is it always using the same header or do we need to give people "multiple custom header" edit access.

Because this question wasn't obvious, we so far don't allow any changes.. This allows you to use local models through ollama or other local inference utilities, but does not let you use alternative online providers.

It seems like your provider is AWS bedrock. What does it need for authentication? Simply the API key ?

[sammcj]

Most OpenAI compatible APIs require an API key and they're all the same thankfully - simply a a standard bearer token in the header.

For example:

Completions

curl http://localhost:8000/v1/completions -H "Content-Type: application/json" -H "Authorization: Bearer abcdefhijqlmnop123" -d '{
  "model": "facebook/opt-125m",
  "prompt": "tell me a joke",
  "max_tokens": 512,
  "temperature": 0,
}'

Listing models

curl https://openai-compat.local/v1/models -H "Content-Type: application/json" -H "Authorization: Bearer abcdefhijqlmnop123"

etc...

[nichochar]

Alright in that case we can simply add an optional API key field to the settings page. Do you want to take a stab at it by any chance?

[Lokendra-sinh]

Hey @nichochar I'm back. Ready to work on this :)

[nichochar]

Awesome, if you check this PR I recently made it shows most of the work needed to get this working (to add a new field to the config, you'll have to use the pnpm run migrate and pnpm run generate commands, as described in the README, and there's a bug where drizzle adds an annoying comment in the generated sql file, just delete that.