Cancelled generators do only get closed after gradio exits.

[potamides]

Describe the bug

Resources inside generators are properly freed when the generator is exhausted. But if we cancel the generator before that happens, the resource is held until gradio exits. I would expect that the generator should get garbage collected and closed eventually (freeing resources), but it seems that there are still references to the generator object somewhere so this doesn't happen. In the provided minimum example this is not so problematic, but in my actual application tensors are not freed which will slowly lead to a Cuda OOM.

Have you searched existing issues? 🔎

• [X] I have searched and found no existing issues

Reproduction

from tempfile import TemporaryDirectory
import gradio as gr

def create_tmpdir():
    with TemporaryDirectory() as tmpdir:
        print(tmpdir)
        while True:
            yield

with gr.Blocks() as demo:
    run_btn = gr.Button()
    stop_btn = gr.Button("Stop")

    event = run_btn.click(create_tmpdir)
    stop_btn.click(None, cancels=event) # tmpdir persists until we exit gradio
    demo.queue().launch()

Screenshot

No response

Logs

No response

System Info

Gradio Environment Information:
------------------------------
Operating System: Linux
gradio version: 4.36.0
gradio_client version: 1.0.1

------------------------------------------------
gradio dependencies in your environment:

aiofiles: 23.2.1
altair: 5.3.0
fastapi: 0.111.0
ffmpy: 0.3.2
gradio-client==1.0.1 is not installed.
httpx: 0.27.0
huggingface-hub: 0.23.3
importlib-resources: 6.4.0
jinja2: 3.1.4
markupsafe: 2.1.5
matplotlib: 3.9.0
numpy: 1.26.4
orjson: 3.10.3
packaging: 24.0
pandas: 2.2.2
pillow: 10.3.0
pydantic: 2.7.3
pydub: 0.25.1
python-multipart: 0.0.9
pyyaml: 6.0.1
ruff: 0.4.8
semantic-version: 2.10.0
tomlkit==0.12.0 is not installed.
typer: 0.12.3
typing-extensions: 4.12.2
urllib3: 2.2.1
uvicorn: 0.30.1
authlib; extra == 'oauth' is not installed.
itsdangerous; extra == 'oauth' is not installed.

gradio_client dependencies in your environment:

fsspec: 2024.6.0
httpx: 0.27.0
huggingface-hub: 0.23.3
packaging: 24.0
typing-extensions: 4.12.2
websockets: 11.0.3

Severity

Blocking usage of gradio

[potamides]

To clarify more, I think the behavior I would expect would be to call the .close() method of the generator in question immediately after it is cancelled.

[freddyaboulton]

Thanks for filing @potamides ! Would you like to open a PR?

[potamides]

Sure, if someone can give me some pointers where to look at I would be happy to do that. For starters, I tried adding something like app.iterators[event_id].iterator.close() to the following locations, but this didn't seem to work:

https://github.com/gradio-app/gradio/blob/d429690611eacd7b19cf4609332b0c12c93a1591/gradio/queueing.py#L653-L655

https://github.com/gradio-app/gradio/blob/d429690611eacd7b19cf4609332b0c12c93a1591/gradio/routes.py#L826-L828