Stream as True returns Nothing due to a bug in _streaming.py SSEDecoder decode function

[xuechendi]

Confirm this is an issue with the Python library and not an underlying OpenAI API

• [X] This is an issue with the Python library

Describe the bug

decode function returns "None" when input is valid, Which leads to a 'None' output for streaming

To Reproduce

1. set stream as True in client.chat.completions.create
2. when waiting for response iterator, 'None' returned instead of Completion Chunks

Code snippets

Below function always return None when input argument 'line' is not None.

def decode(self, line: str) -> ServerSentEvent | None:
        # See: https://html.spec.whatwg.org/multipage/server-sent-events.html#event-stream-interpretation  # noqa: E501

        if not line:
            if not self._event and not self._data and not self._last_event_id and self._retry is None:
                return None

            sse = ServerSentEvent(
                event=self._event,
                data="\n".join(self._data),
                id=self._last_event_id,
                retry=self._retry,
            )

            # NOTE: as per the SSE spec, do not reset last_event_id.
            self._event = None
            self._data = []
            self._retry = None

            return sse

        if line.startswith(":"):
            return None

        fieldname, _, value = line.partition(":")

        if value.startswith(" "):
            value = value[1:]

        if fieldname == "event":
            self._event = value
        elif fieldname == "data":
            self._data.append(value)
        elif fieldname == "id":
            if "\0" in value:
                pass
            else:
                self._last_event_id = value
        elif fieldname == "retry":
            try:
                self._retry = int(value)
            except (TypeError, ValueError):
                pass
        else:
            pass  # Field is ignored.

        return None

OS

Linux

Python version

python 3.9

Library version

openai 1.13.2

[rattrayalex]

Thanks for the report, we'll look into this.

[xuechendi]

Thanks for the report, we'll look into this.

Thank, @rattrayalex , I fixed by this PR if it makes sense to you https://github.com/openai/openai-python/pull/1173

[RobertCraigie]

@xuechendi we intentionally do not yield an event until an empty line is received as events can span multiple data lines, the first example in the spec showcases this: https://html.spec.whatwg.org/multipage/server-sent-events.html

How did you run into this? Are you using a proxy server? If you are I suspect the proxy is not relaying the SSE content properly.

[xuechendi]

@RobertCraigie , I see! Thanks for the information. I am using a tricky self-hosted LLM. I'll double check with my LLM serving.

[shobrook]

Still seeing occasional None return types for streamed chat responses in the latest version of the SDK. Probably not caused by the same bug, but is anyone else still experiencing this? Should always return an iterator and never None –– perhaps an unhandled 500 exception in the SDK?

[RobertCraigie]

@shobrook can you open an issue with an example repro?