GroqCall is a proxy server that enables lightning-fast function calls for Groq's Language Processing Unit (LPU) and other AI providers. It simplifies the creation of AI assistants by offering a wide range of built-in functions hosted on the cloud.
To quickly start using GroqCall without running it locally, make requests to one of the following base URLs:
https://groqcall.ai/proxy/groq/v1
http://localhost:8000
(if running the proxy server locally)Clone the repository:
git clone https://github.com/unclecode/groqcall.git
cd groqcall
Create and activate a virtual environment:
python -m venv venv
source venv/bin/activate
Install dependencies:
pip install -r requirements.txt
Run the FastAPI server:
./venv/bin/uvicorn --app-dir app/ main:app --reload
from phi.llm.openai.like import OpenAILike
from phi.assistant import Assistant
from phi.tools.duckduckgo import DuckDuckGo
my_groq = OpenAILike(
model="mixtral-8x7b-32768",
api_key="YOUR_GROQ_API_KEY",
base_url="https://groqcall.ai/proxy/groq/v1" # or "http://localhost:8000/proxy/groq/v1" if running locally
)
assistant = Assistant(
llm=my_groq,
tools=[DuckDuckGo()],
show_tool_calls=True,
markdown=True
)
assistant.print_response("What's happening in France? Summarize top stories with sources, very short and concise.", stream=False)
GroqCall introduces FuncHub, which allows you to make function calls without passing the function schema.
import requests
api_key = "YOUR_GROQ_API_KEY"
header = {
"Authorization": f"Bearer {api_key}",
"Content-Type": "application/json"
}
proxy_url = "https://groqcall.ai/proxy/groq/v1/chat/completions" # or "http://localhost:8000/proxy/groq/v1/chat/completions" if running locally
request = {
"messages": [
{
"role": "system",
"content": "YOU MUST FOLLOW THESE INSTRUCTIONS CAREFULLY.\n<instructions>\n1. Use markdown to format your answers.\n</instructions>"
},
{
"role": "user",
"content": "What's happening in France? Summarize top stories with sources, very short and concise."
}
],
"model": "mixtral-8x7b-32768",
"tool_choice": "auto",
"tools": [
{
"type": "function",
"function": {
"name": "duckduck.search"
}
},
{
"type": "function",
"function": {
"name": "duckduck.news"
}
}
]
}
response = requests.post(
proxy_url,
headers=header,
json=request
)
print(response.json()["choices"][0]["message"]["content"])
If you prefer to pass your own function schemas, refer to the Function Schema example in the cookbook.
Function call proxy can be used with Ollama. You should first install Ollama and run it locally. Then refer to the Ollama example in the cookbook.
Explore the Cookbook for more examples and use cases of GroqCall.
Groq is a startup that designs highly specialized processor chips aimed specifically at running inference on large language models. They've introduced what they call the Language Processing Unit (LPU), and the speed is astounding—capable of producing 500 to 800 tokens per second or more.
As an admirer of Groq and their community, I built this proxy to enable function calls using the OpenAI interface, allowing it to be called from any library. This engineering workaround has proven to be immensely useful in my company for various projects.
Contributions are welcome! If you have ideas, suggestions, or would like to contribute to this project, please reach out to me on Twitter (X) @unclecode or via email at unclecode@kidocode.com.
Let's collaborate and make this repository even more awesome! 🚀
This project is licensed under the Apache License 2.0. See LICENSE for more information.