vercel / ai

Build AI-powered applications with React, Svelte, Vue, and Solid
https://sdk.vercel.ai/docs
Other
10.31k stars 1.55k forks source link

Custom provider name when extending the openai provider #3209

Open williamlmao opened 1 month ago

williamlmao commented 1 month ago

Feature Description

Right now if you create a groq provider via the openai provider, the model.provider will still say openai.

Instead, you should be able to provider a name or customName property when extending the openai provider.

Use Case

Logging the provider

Additional context

No response

jeremyphilemon commented 1 month ago

Hey @williamlmao, thanks for opening an issue!

You can create a provider registry and specify your own provider name and model name aliases that you wish to log.

Update: Just tried this and it does look like the original provider's name gets logged instead

const registry = createProviderRegistry({
  groq: createOpenAI({
    baseUrl: "...",
    apiKey: process.env.OPENAI_API_KEY,
  }),
});

console.log(registry.languageModel("groq:gpt-4o").provider);

cc @lgrammel

lgrammel commented 1 month ago

https://github.com/vercel/ai/pull/3253

williamlmao commented 2 weeks ago

Thanks @jeremyphilemon! Sorry it took me a minute to get around to testing this.

Could we reopen this one please? I am on version 3.4.33 and the following code will still log openai.chat as the provider.

const originalTogether = createOpenAI({
  apiKey: process.env.TOGETHER_API_KEY ?? '',
  baseURL: 'https://api.together.xyz/v1',
});

export const together = customProvider({
  languageModels: {
    'together:llama-3-8b': originalTogether(
      'meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo'
    ),
    'together:llama-3-70b': originalTogether(
      'meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo'
    ),
  },
  fallbackProvider: originalTogether,
});

export const registry = createProviderRegistry({
  together,
});

console.log(registry.languageModel('together:llama-3-8b').provider);

Also just some feedback on the DX here - it still feels a bit awkward to need to go to registry.languageModel. What I was expecting was something more like this:

export const together = customProvider({
  languageModels: {
    'together:llama-3-8b': originalTogether(
      'meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo'
    ),
    'together:llama-3-70b': originalTogether(
      'meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo'
    ),
  },
  fallbackProvider: originalTogether,
});

// model names registered under languageModels would be type safe here
together('together:llama-3-8b') 

This would bring the custom providers in line with officially supported providers