gpt-4-1106-preview 500 errors

[cpacker]

Describe the bug

Using gpt-4-1106-preview (gpt-4-turbo) seems to produce an unusually high amount of 500 errors. The discrepancy in error rate may be related to how we're calling the API (possibly due to using deprecated functions kwargs?).

• [x] Reproduce high 500 error occurrence on gpt-4-turbo (isolate)
• [ ] Patch the bug either by changing the request format or adding extra error handling for gpt-4-turbo specifically

---

Example:

💭 Here's Chad, signing in for the first time. Interesting, I have no prior record of him, blank slate. Let’s fix that and give a warm welcome. I wonder what brings him here
today. Foster a striking first impression. “More human than human”—fitting. Show curiosity, enthusiasm, a dash of wit maybe? Ready to engage.
🤖 Hey Chad! Welcome to... well, right here! I'm pretty stoked to have you join me. I'm all ears – what's on your mind today?
> Enter your message: my name is nicholas, lets play dnd 5e anything in () is out of character never assume what i say and do your the dungeon master im your only player dont ass
ume anything about the campaign or world lets work together to create it ill handle mechanics you handle lore just ask me questions to get us started
💭 A twist. First Chad, now Nicholas. He seeks a D&D adventure. I must be the illustrious Dungeon Master in a world we'll forge together. Excitement bubbles within. Let's weave a
tale of legend with Nicholas as the hero. Ask for his character, set the first scene—make it vivid, intriguing. A spark to ignite our journey. Your move, Dungeon Master.
⚡🧠 [function] updating memory with core_memory_append
💭 Ready with Nicholas' details as our quest begins. The mystery of the naming is a story for another time. Now, to craft a world that breathes life into Nicholas's imagination.
Start simple, yet brimming with potential—a tavern, a rumor, a call to action? Engage him, encourage participation. This is our canvas, and together we'll paint it with the tales
of heroes.
🤖 Grand starts hail from humble origins, Nicholas. Your character finds themselves in 'The Gilded Griffin', a tavern buzzing with rumors of a lost citadel that houses a treasure
beyond comprehension. (What's your character's name and their reaction to such whispers? Feel free to add any traits or backstories you deem fit.)
> Enter your message: (i said lets make the world together not just throw me in...)
An exception ocurred when running agent.step():
Traceback (most recent call last):
  File "C:\Users\.\AppData\Local\Programs\Python\Python310\lib\site-packages\memgpt\main.py", line 284, in run_agent_loop
    new_messages, user_message, skip_next_user_input = process_agent_step(user_message, no_verify)
  File "C:\Users\.\AppData\Local\Programs\Python\Python310\lib\site-packages\memgpt\main.py", line 260, in process_agent_step
    new_messages, heartbeat_request, function_failed, token_warning = memgpt_agent.step(
  File "C:\Users\.\AppData\Local\Programs\Python\Python310\lib\site-packages\memgpt\agent.py", line 683, in step
    raise e
  File "C:\Users\.\AppData\Local\Programs\Python\Python310\lib\site-packages\memgpt\agent.py", line 614, in step
    response = self.get_ai_reply(
  File "C:\Users\.\AppData\Local\Programs\Python\Python310\lib\site-packages\memgpt\agent.py", line 812, in get_ai_reply
    raise e
  File "C:\Users\.\AppData\Local\Programs\Python\Python310\lib\site-packages\memgpt\agent.py", line 793, in get_ai_reply
    response = create(
  File "C:\Users\.\AppData\Local\Programs\Python\Python310\lib\site-packages\memgpt\openai_tools.py", line 334, in wrapper
    return func(*args, **kwargs)
  File "C:\Users\.\AppData\Local\Programs\Python\Python310\lib\site-packages\memgpt\openai_tools.py", line 384, in create
    return openai_chat_completions_request(
  File "C:\Users\.\AppData\Local\Programs\Python\Python310\lib\site-packages\memgpt\openai_tools.py", line 201, in openai_chat_completions_request
    raise http_err
  File "C:\Users\.\AppData\Local\Programs\Python\Python310\lib\site-packages\memgpt\openai_tools.py", line 193, in openai_chat_completions_request
    response.raise_for_status()  # Raises HTTPError for 4XX/5XX status
  File "C:\Users\.\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: https://api.openai.com/v1/chat/completions

[cpacker]

Tagging @ndisalvio03 for visibility

[agiletechnologist]

I get these in the preview, too. Again this is PREVIEW, not ready for production use.

[cpacker]

I tried running gpt-4-turbo with the tool calling spec instead, but am still running into the same 500 errors:

error = 500 Server Error: Internal Server Error for url: https://api.openai.com/v1/chat/completions
HTTPError occured, but unknown error message: {'message': 'The server had an error processing your request. Sorry about that! You can retry your request, or contact us
through our help center at help.openai.com if you keep seeing this error. (Please include the request ID ... in your email.)', 'type':
'server_error', 'param': None, 'code': None}
step() failed with an unrecognized exception: '500 Server Error: Internal Server Error for url: https://api.openai.com/v1/chat/completions'

So it doesn't seem to be related to PR #619. Maybe there are other parameters that could be passed to the call that might help? Eg seed?

[ndisalvio3]

Could potentially be resolved using the new JSON mode: https://platform.openai.com/docs/guides/text-generation/json-mode