search

TheFireCo / genkit-plugins

Community Plugins for Genkit (OpenAI, Groq, Anthropic, Cohere, etc)
https://thefireco.github.io/genkit-plugins/
Apache License 2.0
97 stars 18 forks source link

Error when running Anthropic plugin and models #149

Open nechmads opened 1 month ago

nechmads commented 1 month ago

Affected plugin(s)

Describe the bug When running generation using dotprompt and the Anthropic plugin, I get the error: roles must alternate between \"user\" and \"assistant\", but found multiple \"user\" roles in a row. Same code works fine when running with OpenAI plugin or Gemini models.

To Reproduce Run generation with Claude models.

Expected behavior Generation should succeed

Screenshots

Screenshot 2024-10-07 at 6 34 50 PM

Plugin(s) version: 0.10.1

Genkit version: 0.5.13

nechmads commented 1 month ago

If I add a {{ role "user" }} somewhere in the prompt, the error changed to "role system doesn't map to an Anthropic role.".

Seems it all comes down to the {{ role "system" }} clause when used in prompt templates. The plugin doesn't translate it well to the Claude API.

MichaelDoyle commented 1 month ago

Hey @nechmads can you share an example prompt and/or code. I think I see what you are likely running into - but want to verify. Vertex AI also supports Anthropic, so I'd like to verify it in both places if possible.