Assistant API Streaming: events are non-conformant to AssistantStreamEvent

[Kr1sh1]

Confirm this is a Node library issue and not an underlying OpenAI API issue

• [ ] This is an issue with the Node library

Describe the bug

I'm seeing a strange issue that exists only in our deployed environment on Twilio Functions, but works perfectly fine when I test locally. This makes debugging the issue more difficult. I'm confident environment variables such as API keys and Assistant ID are set correctly.

I'm trying to use the streaming assistants api as seen in the code snippet. On our deployed environment where the issue exists, the event variable does not conform to the type AssistantStreamEvent. Instead it is just a single character from some part of the system prompt of the assistant. There are multiple events, and it ends up printing a few consecutive characters from the system prompt from an arbitrary location. On one occasion the characters printed from multiple events also spelled out data:[DONE].

There is no issue when using the Assistants API without streaming.

I'm not sure if this is a problem with the Node library or the OpenAI API, or some other environmental configuration issue. The issue has existed since the library update to introduce streaming support, but I also tested with the latest v4.32.2.

I checked later the status of the run and thread that were created and they seemed fine, the thread had new messages from the assistant appended correctly and the run was in a completed state.

To Reproduce

Run the code in the snippet after plugging in an api key and assistant id. Issue was not reproducible in our local testing environment.

Code snippets

import OpenAI from "openai";

async function a() {
  const openai = new OpenAI({ apiKey: "" })
  const threadID = (await openai.beta.threads.create()).id;
  await openai.beta.threads.messages.create(threadID, {
    role: "user",
    content: "Hi"
  });
  const assistant_id = ""
  const assistantStream = await openai.beta.threads.runs.create(threadID, { assistant_id, stream: true });
  for await (const event of assistantStream) {
    console.log("Event: ", event)            // Single character from system prompt
    console.log("Event:Data:", event.data)   // null
    console.log("Event:Event:", event.event) // null
  }
}

OS

Twilio Functions

Node version

v18

Library version

openai v4.32.2

[pstern-sl]

Hi @Kr1sh1, thanks for your detailed notes! I think this could be an issue in the SDK based on your description so would be great to try and narrow things down a bit further.

A couple questions/thoughts: 1) Are you also experiencing the issue with chat completions? (this will help confirm if the issue is in upstream logic or specific to the new assistant code). 2) Just wanted to double check you tested against v4.32.2? This release improved the stream processing and addressed some known issues that caused behavior that sounds similar to what you described, I appreciate you mentioning you tested against that, just confirming. 3) Can you provide a bit more information on the differences between the environments where this is working and where it is not working? I am not familiar with Twilio functions. 4) Could you also give our streaming helpers a try? (using the createAndStream method an event handlers, see example below)

  const run = openai.beta.threads.runs
    .createAndStream(threadId, {
      assistant_id: assistantId,
    })
    //Subscribe to streaming events and log them
    .on('event', (event) => console.log(event));
  const result = await run.finalRun();

[Kr1sh1]

Hi! I'm sorry, maybe I accidentally tested against the previous version before. Just now I forced the package version to be the latest and it is working brilliantly.

[pstern-sl]

@Kr1sh1 So glad to hear that!