search

zhudotexe / kani

kani (カニ) is a highly hackable microframework for chat-based language models with tool use/function calling. (NLP-OSS @ EMNLP 2023)
https://kani.readthedocs.io
MIT License
557 stars 30 forks source link

Support for Groq #42

Open sorcerer2073 opened 3 months ago

sorcerer2073 commented 3 months ago

I was just wondering if anyone has been able to make it work with groq?

zhudotexe commented 3 months ago

I haven't used Groq personally, but as a cloud-based provider it shouldn't be too difficult to implement an engine for it. You'll want to look at the docs for Implementing an Engine and an existing engine implementation (e.g. LlamaCppEngine as a less-complex example of using a Python library as an engine provider).

If you do implement a Groq engine it would make for a great extension package or PR!