Unexpected error occurs sporadically when using gr.ChatInterface with text streaming

jaredsu commented 7 months ago

Describe the bug

The error occurs sporadically when text is being printed out to the chatbot. The error happens when I run the app locally and when I host it using Nginx. The error occurs more frequently when I use the hosted version of the app. No error gets printed to my python console even when I use the debug=True. I looked in the js and provided a screenshot of the error that is displayed.

Have you searched existing issues? 🔎

[X] I have searched and found no existing issues

Reproduction

from openai import OpenAI
import gradio as gr

api_key = "APIKEY"  # Replace with your key
client = OpenAI(api_key=api_key)

def predict(message, history):
    history_openai_format = []
    for human, assistant in history:
        history_openai_format.append({"role": "user", "content": human })
        history_openai_format.append({"role": "assistant", "content":assistant})
    history_openai_format.append({"role": "user", "content": message})

    response = client.chat.completions.create(model='gpt-3.5-turbo',
    messages= history_openai_format,
    temperature=1.0,
    stream=True)

    partial_message = ""
    for chunk in response:
        if chunk.choices[0].delta.content is not None:
              partial_message = partial_message + chunk.choices[0].delta.content
              yield partial_message

gr.ChatInterface(predict).launch(server_name='127.0.0.1',debug=True,show_api=False,share=False)

Screenshot

Logs

No response

System Info

Gradio Environment Information:
------------------------------
Operating System: Linux
gradio version: 4.25.0
gradio_client version: 0.15.0

------------------------------------------------
gradio dependencies in your environment:

aiofiles: 23.2.1
altair: 5.2.0
fastapi: 0.110.1
ffmpy: 0.3.2
gradio-client==0.15.0 is not installed.
httpx: 0.27.0
huggingface-hub: 0.21.4
importlib-resources: 6.4.0
jinja2: 3.1.3
markupsafe: 2.1.5
matplotlib: 3.8.3
numpy: 1.22.0
orjson: 3.9.15
packaging: 23.2
pandas: 1.5.3
pillow: 10.2.0
pydantic: 2.6.3
pydub: 0.25.1
python-multipart: 0.0.9
pyyaml: 6.0.1
ruff: 0.3.5
semantic-version: 2.10.0
tomlkit==0.12.0 is not installed.
typer: 0.9.4
typing-extensions: 4.10.0
uvicorn: 0.29.0
authlib; extra == 'oauth' is not installed.
itsdangerous; extra == 'oauth' is not installed.

gradio_client dependencies in your environment:

fsspec: 2024.2.0
httpx: 0.27.0
huggingface-hub: 0.21.4
packaging: 23.2
typing-extensions: 4.10.0
websockets: 11.0.3