search

saoudrizwan / claude-dev

Autonomous coding agent right in your IDE, capable of creating/editing files, executing commands, and more with your permission every step of the way.
https://marketplace.visualstudio.com/items?itemName=saoudrizwan.claude-dev
MIT License
4.04k stars 381 forks source link

OpenRouter as Groq Llama 3 #122

Closed kiranmaya closed 1 week ago

kiranmaya commented 3 weeks ago

OpenAi and Groq Api and LM Studio API paths are similar, to get llama 3.1 model in this extension, make the following changes, and use groq key as openai apikey.

file : openRouter.ts: baseURL: "https://api.groq.com/openai/v1",

file : api.ts: export const openRouterDefaultModelId: OpenRouterModelId = "llama3-groq-70b-8192-tool-use-preview"

export const openRouterModels = { "llama3-groq-70b-8192-tool-use-preview": { maxTokens: 8192, supportsImages: false, supportsPromptCache:false, inputPrice: 0.0, outputPrice: 0.0, },"mixtral-8x7b-32768": { maxTokens: 8192, supportsImages: false, supportsPromptCache:false, inputPrice: 0.0, outputPrice: 0.0, },

in setting panel, custom instructions

try this prompt wrap your tools calls like this <tool_call>your content here</tool_call> avoid <tool_response> <function_call> <function> for tool calling The tool calls are working fine, but the model is fragile compared to the anthropic/claude-3.5-sonnet , with an 8k maxtokens limit.

saoudrizwan commented 1 week ago

Closing as duplicate of https://github.com/saoudrizwan/claude-dev/issues/30