zed-industries / zed

Code at the speed of thought – Zed is a high-performance, multiplayer code editor from the creators of Atom and Tree-sitter.
https://zed.dev
Other
49.95k stars 3.06k forks source link

Problems with using Claude 3.5 sonnet in the Assistant Panel #20633

Open goodlux opened 1 day ago

goodlux commented 1 day ago

Check for existing issues

Describe the bug / provide steps to reproduce it

Hi - I'm trying to use Claude 3.5 Sonnet in the Assistant panel. I have both an API key with Anthropic, I also have paid for credits on Zed.dev website.

My ANTHROPIC_API_KEY is set in .env file, and I can use this with other libraries like liteLLM, etc without issue.

For some reason, I'm not able to connect to Claude 3.5 though Zed. I've tried various settings,

Environment

Screenshot 2024-11-13 at 2 33 38 PM

If applicable, add mockups / screenshots to help explain present your vision of the feature

In this screenshot, the tab says Claude 2, the model is reporting Claude 3, and the dropdown says Claude 3.5.

Screenshot 2024-11-13 at 2 34 41 PM

In this screenshot, you can see my settings.json file. I have tried many iterations (copying directly from Zed website, changing model numbers, opening and closing Zed, etc etc.

Screenshot 2024-11-13 at 2 37 46 PM

Sometimes I get Claude 2, sometimes 3. Selecting different models from the dropdown has mixed results, but never Claude 3.5 Sonnet .... in other applications, 3.5 is always very straightforward about his model number and date. This is with completely fresh settings, selecting Claude 3.5 from the dropdown, then restarting (i get Claude 3 sonnet this time instead of 3.5)

Screenshot 2024-11-13 at 2 52 51 PM

Logfile from this last open is attached.

If applicable, attach your Zed.log file to this issue.

No response

goodlux commented 1 day ago

tried attaching log, but it was too big and couldn't post.

goodlux commented 17 hours ago

Update. I've spent hours trying to debug this, but was not able to even get Claude 3.5 in any version or format. I have Zed.dev, anthropic, and Github copilot_chat .... paying for all of these but can't get a recent anthropic model. I took the whole debugging thread and fed in into the Claude website for a summary, here goes:

Comprehensive Update on Claude Integration Issues

  1. Model Name & Version Inconsistencies Current Names Found in Zed:

Default settings reference: "claude-3-5-sonnet" (incorrect/outdated) Settings.json auto-populates: "claude-3-5-sonnet-latest" when using zed.dev provider Dropdown shows multiple variations of Claude 3.5 Sonnet without clear versioning

Correct Model Names (per Anthropic's documentation):

Claude 3.5 Sonnet: claude-3-5-sonnet-20241022 (or alias claude-3-5-sonnet-latest) Claude 3 Sonnet: claude-3-sonnet-20240229 Claude 3 Haiku: claude-3-haiku-20240307 Claude 3 Opus: claude-3-opus-20240229

  1. Provider Configuration Issues

Direct Anthropic API connections result in 400 Bad Request errors Zed.dev provider works but connects to Claude 3 instead of 3.5 API URL configuration doesn't seem to affect connection behavior

  1. Context and Persistence Problems

Model selection persists across contexts unexpectedly Settings changes require new contexts to take effect Sometimes changes don't take effect even with new contexts Got stuck on GPT-4 despite Claude being configured as default

  1. UI/Settings Discrepancies

Dropdown shows multiple versions of same model Settings.json changes don't always reflect in actual connections Default provider settings seem to override custom configurations Model selection in dropdown doesn't always match active connection

  1. Documentation Gaps

Unclear how to properly configure direct Anthropic connection No documentation about context persistence behavior Ambiguous model naming in dropdown vs configuration

Bug Impact

When connection fails, appears to fall back to Claude 3 Sonnet without indicating the fallback Multiple entries in dropdown don't clearly indicate which version string is being used The zed.dev provider seems to use different model strings than direct Anthropic connections Settings and UI show inconsistent model naming patterns across different parts of the interface Version mismatch appears to be contributing to the 400 Bad Request errors when attempting direct Anthropic API connections

Recommendations

Update default settings to reference current model versions Fix direct Anthropic API connection issues Make model switching behavior more predictable Clarify relationship between settings.json and UI selections Document context persistence behavior Clean up duplicate model entries in dropdown