search

jupyterlab / jupyter-ai

A generative AI extension for JupyterLab
https://jupyter-ai.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
3.16k stars 316 forks source link

Groq Support #739

Open robertritz opened 5 months ago

robertritz commented 5 months ago

Groq has tremendous inference speeds (280 tokens per second for Llama 3 70B and 877 tokens per second for Llama 8B0. It would be amazing to get support for this in Jupyter AI.

welcome[bot] commented 5 months ago

Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! :hugs:
If you haven't done so already, check out Jupyter's Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more effectively. welcome You can meet the other Jovyans by joining our Discourse forum. There is also an intro thread there where you can stop by and say Hi! :wave:
Welcome to the Jupyter community! :tada:

nick-youngblut commented 4 months ago

Also, groq has a generous free pricing tier