Groq response to simple query: context_length_exceeded

[davidsells]

System Information

• MacOS Version: 14.5
• Letta: main branch Oct 31 ( 8AM EDT )

Agent Configuration

• Agent Name: GroqOne
• Model: llama3-70b-8192
• Embedding Model: Letta-free
• Agent Persona: sam_pov
• Human Persona: basic

First Query:

List top strain of apple.

Response in UI:

Let's make a mental note to find the answer to this question later.

archival_memory_insert({
"content": "Note to self: Research top strains of Apple.",
"request_heartbeat": true
})
Let's make a mental note to find the answer to this question later.

archival_memory_insert({
"content": "Note to self: Research top strains of Apple.",
"request_heartbeat": true
})
{
"status": "OK",
"message": "None",
"time": "2024-10-31 09:25:48 AM EDT-0400"
}

Errors Thrown in Console:

NFO:     ::1:51657 - "GET /v1/models/ HTTP/1.1" 200 OK
Warning: token streaming is only supported for OpenAI models. Setting to False.
INFO:     ::1:51672 - "POST /v1/agents/agent-a2ba4aea-bb2f-4d6d-b402-5cad550a46d6/messages HTTP/1.1" 200 OK
Letta.letta.server.server - ERROR - Error in server._step: HTTP error occurred: 400 Client Error: Bad Request for url: https://api.groq.com/openai/v1/chat/completions | Status code: 400, Message: {"error":{"message":"Please reduce the length of the messages or completion.","type":"invalid_request_error","param":"messages","code":"context_length_exceeded"}}

Traceback (most recent call last):
  File "/Users/davidsells/projects/persist/chatgpt/MemGPT/letta/llm_api/helpers.py", line 48, in make_post_request
    response.raise_for_status()
  File "/Users/davidsells/Library/Caches/pypoetry/virtualenvs/letta-mQlk12mo-py3.11/lib/python3.11/site-packages/requests/models.py", line 1024, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://api.groq.com/openai/v1/chat/completions

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/davidsells/projects/persist/chatgpt/MemGPT/letta/server/server.py", line 442, in _step
    usage_stats = letta_agent.step(
                  ^^^^^^^^^^^^^^^^^
  File "/Users/davidsells/projects/persist/chatgpt/MemGPT/letta/agent.py", line 787, in step
    step_response = self.inner_step(
                    ^^^^^^^^^^^^^^^^
  File "/Users/davidsells/projects/persist/chatgpt/MemGPT/letta/agent.py", line 996, in inner_step
    raise e
  File "/Users/davidsells/projects/persist/chatgpt/MemGPT/letta/agent.py", line 912, in inner_step
    response = self._get_ai_reply(
               ^^^^^^^^^^^^^^^^^^^
  File "/Users/davidsells/projects/persist/chatgpt/MemGPT/letta/agent.py", line 529, in _get_ai_reply
    raise e
  File "/Users/davidsells/projects/persist/chatgpt/MemGPT/letta/agent.py", line 492, in _get_ai_reply
    response = create(
               ^^^^^^^
  File "/Users/davidsells/projects/persist/chatgpt/MemGPT/letta/llm_api/llm_api_tools.py", line 66, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/davidsells/projects/persist/chatgpt/MemGPT/letta/llm_api/llm_api_tools.py", line 315, in create
    response = openai_chat_completions_request(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/davidsells/projects/persist/chatgpt/MemGPT/letta/llm_api/openai.py", line 537, in openai_chat_completions_request
    response_json = make_post_request(url, headers, data)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/davidsells/projects/persist/chatgpt/MemGPT/letta/llm_api/helpers.py", line 75, in make_post_request
    raise requests.exceptions.HTTPError(error_message) from http_err
requests.exceptions.HTTPError: HTTP error occurred: 400 Client Error: Bad Request for url: https://api.groq.com/openai/v1/chat/completions | Status code: 400, Message: {"error":{"message":"Please reduce the length of the messages or completion.","type":"invalid_request_error","param":"messages","code":"context_length_exceeded"}}

None
/Users/davidsells/projects/persist/chatgpt/MemGPT/letta/server/rest_api/utils.py:64: UserWarning: Error getting usage data: HTTP error occurred: 400 Client Error: Bad Request for url: https://api.groq.com/openai/v1/chat/completions | Status code: 400, Message: {"error":{"message":"Please reduce the length of the messages or completion.","type":"invalid_request_error","param":"messages","code":"context_length_exceeded"}}

  warnings.warn(f"Error getting usage data: {e}")

[mattzh72]

Does this issue occur with other groq models? We recommend the groq tool calling fine tuned model.