Using gr.State cause deadlock

[lenow55]

Describe the bug

When i use gr.State to hold information about user_id, concurency work bad. I can run three generators simultaneously but no more. And it does not depend on the asynchrony of the function.

When i remove state, concurency work as expected.

Have you searched existing issues? 🔎

• [X] I have searched and found no existing issues

Reproduction

import asyncio
import time
import uuid

import gradio as gr

async def async_bot():
    history = [{"role": "assistant", "content": "Help me."}]
    for i in range(10000):
        await asyncio.sleep(0.05)
        history[-1]["content"] += "hello " * i
        yield history

def bot():
    history = [{"role": "assistant", "content": "Help me."}]
    for i in range(10000):
        time.sleep(0.05)
        history[-1]["content"] += "hello " * i
        yield history

with gr.Blocks(css="footer{display:none !important}") as demo:
    output = gr.HTML(elem_id="output_html")
    conversation_id = gr.State(uuid.uuid4)
    chatbot = gr.Chatbot(type="messages")
    button = gr.Button("Start generation")
    _ = button.click(async_bot, None, chatbot)

_ = demo.queue(default_concurrency_limit=4).launch()

Screenshot

No response

Logs

No response

System Info

Gradio Environment Information:
------------------------------
Operating System: Linux
gradio version: 5.7.1
gradio_client version: 1.5.0

------------------------------------------------
gradio dependencies in your environment:

aiofiles: 23.2.1
anyio: 4.6.2.post1
audioop-lts is not installed.
fastapi: 0.115.6
ffmpy: 0.4.0
gradio-client==1.5.0 is not installed.
httpx: 0.28.0
huggingface-hub: 0.26.3
jinja2: 3.1.4
markupsafe: 2.1.5
numpy: 2.1.3
orjson: 3.10.12
packaging: 24.2
pandas: 2.2.3
pillow: 11.0.0
pydantic: 2.10.3
pydub: 0.25.1
python-multipart==0.0.12 is not installed.
pyyaml: 6.0.2
ruff: 0.8.1
safehttpx: 0.1.6
semantic-version: 2.10.0
starlette: 0.41.3
tomlkit==0.12.0 is not installed.
typer: 0.15.0
typing-extensions: 4.12.2
urllib3: 2.2.3
uvicorn: 0.32.1
authlib; extra == 'oauth' is not installed.
itsdangerous; extra == 'oauth' is not installed.

gradio_client dependencies in your environment:

fsspec: 2024.10.0
httpx: 0.28.0
huggingface-hub: 0.26.3
packaging: 24.2
typing-extensions: 4.12.2
websockets: 12.0

Severity

I can work around it

[lenow55]

Threading no state

https://github.com/user-attachments/assets/4689c99e-2128-4566-b17f-4e5c442edfc5

[lenow55]

https://github.com/user-attachments/assets/1692c50e-478e-444f-86e9-7a249c48b198

https://github.com/user-attachments/assets/4b4a86bf-2b0b-45d5-afde-8a800a03d257

[lenow55]

https://github.com/user-attachments/assets/4a88682f-0e86-4ab8-a00d-e2a86f8ece72

[freddyaboulton]

Hi @lenow55 - Gradio establishes a persistent connection HTTP connection to know when you leave the page and clean up state variables. Browsers limit pages to establishing around ~4 connections at a time so that's what's happening. If you try across different browsers you won't see the behavior you see now.

[lenow55]

Hi @lenow55 - Gradio establishes a persistent connection HTTP connection to know when you leave the page and clean up state variables. Browsers limit pages to establishing around ~4 connections at a time so that's what's happening. If you try across different browsers you won't see the behavior you see now.

Hi @freddyaboulton - But why can gradio generate without state more than 4 connections (please check threading_no_state.mp4 carefully), oposite, with state gradio can't generate more than 3 connections (please check threading_without_state.mp4 carefully)?

[freddyaboulton]

The connection (GET /heartbeat/) is only established if there are gr.State variables since it's used to delete state variables from memory when users close the page

[lenow55]

The connection (GET /heartbeat/) is only established if there are gr.State variables since it's used to delete state variables from memory when users close the page

Ok, i see. If i will use api for open multiple stream connection, gradio works as expected?

In company load testing our gradio service was carried out. In that cause api requests were used and service managed three connections also.

[freddyaboulton]

If i will use api for open multiple stream connection, gradio works as expected?

Yes. The limit is imposed by browsers. Connecting via API (e.g. gradio_client) should be fine.