How to use streaming in Azure OpenAI Assistants API ?

[BeanHsiang]

Please provide us with the following information:

This issue is for a: (mark with an x)

- [x] bug report -> please search issues before submitting
- [x] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

1. pip install openai
2. input the codes below

   
   from dotenv import load_dotenv
   from typing_extensions import override
   from openai import AzureOpenAI, AssistantEventHandler

load_dotenv()

api_key = os.environ.get("AZURE_OPENAI_API_KEY") api_version = os.environ.get("OPENAI_API_VERSION") azure_endpoint = os.environ.get("AZURE_OPENAI_ENDPOINT") assistant_id = os.environ.get("AZURE_OPENAI_ENDPOINT")

class EventHandler(AssistantEventHandler): @override def on_text_created(self, text) -> None: print(f"\nassistant > ", end="", flush=True)

@override
def on_text_delta(self, delta, snapshot):
    print(delta.value, end="", flush=True)

def on_tool_call_created(self, tool_call):
    print(f"\nassistant > {tool_call.type}\n", flush=True)

def on_tool_call_delta(self, delta, snapshot):
    if delta.type == 'code_interpreter':
        if delta.code_interpreter.input:
            print(delta.code_interpreter.input, end="", flush=True)
        if delta.code_interpreter.outputs:
            print(f"\n\noutput >", flush=True)
            for output in delta.code_interpreter.outputs:
                if output.type == "logs":
                    print(f"\n{output.logs}", flush=True)

client = AzureOpenAI(api_key=api_key, api_version=api_version, azure_endpoint=azure_endpoint)

thread = client.beta.threads.create( messages=[] )

client.beta.threads.messages.create( thread_id=thread.id, role="user", content="here are some messages..." )

with client.beta.threads.runs.stream( thread_id=thread.id, assistant_id=assistant_id, event_handler=EventHandler() ) as stream: stream.until_done()

### Any log messages given by the failure
>
```bash
Traceback (most recent call last):
  File "/Users/admin/project/aoai-assistant-demo/main.py", line 47, in <module>
    with client.beta.threads.runs.stream(
            thread_id=thread.id,
            assistant_id=assistant_id,
            event_handler=EventHandler()
        ) as stream:
  File "/Users/admin/project/aoai-assistant-demo/.venv/lib/python3.12/site-packages/openai/lib/streaming/_assistants.py", line 444, in __enter__
    self.__stream = self.__api_request()
                    ^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/project/aoai-assistant-demo/.venv/lib/python3.12/site-packages/openai/_base_client.py", line 1213, in post
    return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/project/aoai-assistant-demo/.venv/lib/python3.12/site-packages/openai/_base_client.py", line 902, in request
    return self._request(
           ^^^^^^^^^^^^^^
  File "/Users/admin/project/aoai-assistant-demo/.venv/lib/python3.12/site-packages/openai/_base_client.py", line 993, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.BadRequestError: Error code: 400 - {'error': {'message': "Unknown parameter: 'stream'.", 'type': 'invalid_request_error', 'param': None, 'code': None}}

Expected/desired behavior

Expect the codes to work fine.

OS and Version?

macOS Sonoma 14.0

Versions

python 3.12 openai 1.16.2

Mention any other details that might be useful

---

[Yuki-Matsuyama-a]

You may need to define an assistant. https://platform.openai.com/docs/assistants/overview?lang=python&context=with-streaming

[hayescode]

I am also unable to use AzureOpenAI client and the EventHandler. It runs using 1.17.0 but nothing prints.

[Adlef]

So am I. I suspect an issue with Azure OpenAI API version, not including the necessary structures to use the new stream feature on assistant from OpenAI. EventHandler() has no impact.

[sitebay]

Streaming works using AsyncAzureOpenAI(api_version="2024-04-01-preview")