Added Groq Support

[fire17]

Describe the changes you have made:

Groq's official python api now fits well into oi flow, no errors. Though final answers are halucinated rather than actual output. Seems to plan, write code, but not execute yet.

Reference any relevant issues:

Trying to get groq/mixtral to work #1237 aka groq is not working with litellm out of the box --model groq/mixtral-8x7b-32768 throws errors

Pre-Submission Checklist (optional but appreciated):

• [x] I have included relevant documentation updates (stored in /docs)
• [x] I have read docs/CONTRIBUTING.md
• [-] I have read docs/ROADMAP.md (not fully yet, but no mention of groq)

OS Tests (optional but appreciated):

• [x] Tested Through Github Codespaces

[fire17]

@KillianLucas please open a branch groq so i can make a PR to it instead of main atleast until tests are added, and it is fully production ready

as it is now, the errors are comming from litellm's side, groq is too good to not include, hope you feel the same i can see it being the default for a lot of people. it is already default for me in all my other pipelines

Thanks a lot and all the best! 😇

[fire17]

Current state of PR:

• groq api integration now fits well into oi flow, no errors.
  • Final answers are still halucinated rather than using actual output.
  • Seems to plan, write code, but not execute yet. - Probably just need to match it to oi expected json output
• groq.mdx docs added

Todos:

• match mixtral output to oi expected output,
  • do the same for llama3 (8b/70b)
• Add tests, make production-ready

[Cobular]

Damn I came to add this fork, beat me by an hour! Nice going!

[fire17]

haha thanks @Cobular , as stated this pr doest make code execute yet, just swaps the completion apis correctly, so if yours does, you might as well

One important thing.... there is a way to make it work Now just not with --model yet

With techfren's help :heart: The key is to use api_base url --api_base "https://api.groq.com/openai/v1" --api_key $GROQ_API_KEY --model "mixtral-8x7b-32768"

This is working

export GROQ_API_KEY='<your-key-here>'
poetry run interpreter --api_base "https://api.groq.com/openai/v1" --api_key $GROQ_API_KEY  --model "mixtral-8x7b-32768" --context_window 32000

This is NOT working

export GROQ_API_KEY='<your-key-here>'
poetry run interpreter  --model "groq/mixtral-8x7b-32768" --context_window 32000

[CyanideByte]

https://github.com/BerriAI/litellm/pull/3176 Potentially already being done by litellm

[KillianLucas]

NICE. Love Groq, great work on this @fire17. As @CyanideByte mentioned I think we should push this into LiteLLM (they abstract away the Groq interaction so it's = to an OpenAI client.)

And it looks like it works with the latest LiteLLM! interpreter --model groq/llama3-70b-8192 runs OI with groq, and can execute code, if I also pass in my api_key.

In that case, it would be great if we could merge this PR with just the documentation then. I'll make that change then merge if that's okay with you. If there's anything else to include from the PR let me know, we can reopen.

[fire17]

I'll make that change then merge if that's okay with you. If there's anything else to include from the PR let me know, we can reopen.

For sure! @KillianLucas I've just added the --api_base workaround to the docs incase anyone still running into issues. Welcome to take the doc :)

All the best!

Ps checkout my new PR #1259