TheFireCo / genkit-plugins

Community Plugins for Genkit (OpenAI, Groq, Anthropic, Cohere, etc)
https://thefireco.github.io/genkit-plugins/
Apache License 2.0
101 stars 21 forks source link

Support for Sonnet 3.5 October Version #154

Open danwritecode opened 1 month ago

danwritecode commented 1 month ago

Is your feature request related to a problem? Please describe. Currently the anthropic plugin is supporting version: claude-3-5-sonnet-20240620. It should be updated to support the latest October version.

Describe the solution you'd like I'd like to be able to use the latest sonnet model.

Additional context I'm curious how the "versions" are intended to function. In the genkit-ai library that the anthropic plugin uses, it references a modelRef definition which has this ModelInfoSchmea which contains a version. Presumably the version is there so we can pass in the version name directly instead of the generic name? But doing this does not work.

IE - passing model: anthropic/claude-3-5-sonnet-20240620 for example, does not work with dotprompt.

https://github.com/firebase/genkit/blob/fe24a4e36fad88f1fb094748904e9bc40f7c2ff5/js/ai/src/model.ts#L126

export const ModelInfoSchema = z.object({
  /** Acceptable names for this model (e.g. different versions). */
  versions: z.array(z.string()).optional(),
danwritecode commented 1 week ago

Just adding on here that support for haiku 3.5 is also needed.

I want to point out that it's pretty critical that model versions with no API differences need to be supported quickly. For models from oai and anthropic, I'd expect support within days at most. Testing new models is a mission critical thing that can't wait weeks.

After reviewing the source code further, it seems like the "versions" array in the modelRef is actually not doing anything. My proposal is to change the models to the versions themselves, ie:

export const SUPPORTED_CLAUDE_MODELS: Record<string, ModelReference<typeof AnthropicConfigSchema>> = {
  'claude-3-5-sonnet-20240620': claude35Sonnet20240620,
  'claude-3-5-sonnet-20241022': claude35Sonnet20241022,
  ...
};

More than happy to PR this if approach is agreeable.