Feature Request: Custom OpenAPI/Claude Endpoint Support

saoudrizwan / claude-dev

Autonomous coding agent right in your IDE, capable of creating/editing files, executing commands, and more with your permission every step of the way.

https://marketplace.visualstudio.com/items?itemName=saoudrizwan.claude-dev

MIT License

4.08k stars 387 forks source link

Feature Request: Custom OpenAPI/Claude Endpoint Support #76

Closed btiger closed 1 week ago

btiger commented 1 month ago

Can you add the ability for users to configure and use custom OpenAPI/Claude endpoints in the application, allowing for integration with third-party OpenAPI/Claude services? some of the 3rd party services have no rate limitation.

scarr05 commented 1 month ago

I second this feature, something like LiteLLM akin to OpenwebUI supports would give the user the flexibility to use any LLM endpoint (supported by LiteLLM) they choose.

Claude 3.1 Sonnet works fantastic through Amazon Bedrock proxied through LiteLLM which i use for multiple VSCode ext and through OpenwebUI.

edit : i've seen you do have bedrock support, although LiteLLM could take some of the heavy lifting for switching models

BartNetJS commented 1 month ago

In version v1.1.13 i noted "Added option to choose other Claude models (+ GPT-4o, DeepSeek, and Mistral if you use OpenRouter)" Do i understand it correct that I need an OpenRouter account (and credits) to use gpt-4o?

mkearl commented 1 month ago

That is correct

BartNetJS commented 1 month ago

will there be an option in the future to use openai api keys directly?

lucacri commented 1 month ago

Ollama support (or even better, LiteLLM) would be fantastic

lei1402 commented 1 month ago

hi, can this be deployed and used in an intranet environment?

lei1402 commented 1 month ago

Can this support models run by ollama?

Batyan45 commented 1 month ago

+1 for this request, it will be useful

xvweirong commented 3 weeks ago

+1 same requirement, please allow to config custom base url for Claude/OpenAPI endpoints.

danividalg commented 3 weeks ago

It could be great if it could be possible to use ollama, deepseek and or openai compatible API :) Perhaps it could be needed a little bit modifications on prompts to be able to guide the model to reply as expected.

saoudrizwan commented 1 week ago

Ollama support is on the roadmap, it's a bit more complicated than other providers since there's no knowing what the model is/stats which is a big part of the UI. Closing as dupe of https://github.com/saoudrizwan/claude-dev/issues/30#issuecomment-2325005847

saoudrizwan commented 1 week ago

v1.5.21 now supports OpenAI compatible APIs and Ollama!

Please keep in mind that claude dev uses complex prompts so it may not function nearly as well as claude 3.5 sonnet. I added a new error message in case he makes 3 mistakes in a row (mistakes mean invalid tool calls or no tool call whatsoever), so this should help you guide less capable models in the right direction.

saoudrizwan commented 6 days ago

You can now set a custom base URL for the Anthropic API in v.1.5.33: