Reloading a gradio app while running a function throws errors

irgolic commented 3 months ago

Describe the bug

If one is running gradio with reloading (i.e., gradio demo.py instead of python demo.py), runs a function (e.g., presses a "Submit" button), and then changes the file, unexpected behavior occurs.

Specifically:

the app displays Connection errored out. and stops updating the frontend,
the function keeps running,
any time the processing yields a value, it prints a "Returned component not specified as output of function" error.

This happens even if key is set on components, which is counter-intuitive – it feels like it should have enough information to complete the interaction.

From the error I conjecture that the function is still running in the context of the old demo, though the components have been updated in-place to the new demo's, leading to a "Returned component not specified as output of function" error.

The ideal fix would be to preserve execution of the function, and map old output components to new ones under the hood via their key attribute. If the referenced output does not exist anymore, throw a "Returned component not specified as output of function" error as it does now.

Have you searched existing issues? 🔎

[X] I have searched and found no existing issues

Reproduction

import asyncio

import gradio as gr

with gr.Blocks() as demo:
    submit = gr.Button("Submit")

    out = gr.Textbox(key="1")
    outputs = [out]

    # out2 = gr.Textbox(key="2")
    # outputs = [out, out2]

    async def process():
        await asyncio.sleep(5)
        for out_ in outputs:
            yield {out_: 'hi'}

    submit.click(
        process,
        outputs=outputs,
    )

demo.launch()

Create a file with the contents above called script.py
Run gradio script.py
Press the Submit button
Within 5 seconds, uncomment the two lines starting at out2, and save the file

Screenshot

No response

Logs

❯ gradio test.py
Watching: '/Users/rafael/' '/Users/rafael/'

Running on local URL:  http://127.0.0.1:7861

To create a public link, set `share=True` in `launch()`.
Changes detected in: /Users/rafael/script.py
Changes detected in: /Users/rafael/script.py
Traceback (most recent call last):
  File "/opt/homebrew/Caskroom/miniconda/base/envs/my_env/lib/python3.11/site-packages/gradio/queueing.py", line 541, in process_events
    response = await route_utils.call_process_api(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/my_env/lib/python3.11/site-packages/gradio/route_utils.py", line 276, in call_process_api
    output = await app.get_blocks().process_api(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/my_env/lib/python3.11/site-packages/gradio/blocks.py", line 1938, in process_api
    data = await self.postprocess_data(block_fn, result["prediction"], state)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/my_env/lib/python3.11/site-packages/gradio/blocks.py", line 1702, in postprocess_data
    predictions = convert_component_dict_to_list(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/my_env/lib/python3.11/site-packages/gradio/blocks.py", line 638, in convert_component_dict_to_list
    raise ValueError(
ValueError: Returned component <gradio.components.textbox.Textbox object at 0x13a09ab50> not specified as output of function.

System Info

Gradio Environment Information:
------------------------------
Operating System: Darwin
gradio version: 4.37.2
gradio_client version: 1.0.2

------------------------------------------------
gradio dependencies in your environment:

aiofiles: 23.2.1
altair: 5.3.0
fastapi: 0.111.0
ffmpy: 0.3.2
gradio-client==1.0.2 is not installed.
httpx: 0.27.0
huggingface-hub: 0.23.2
importlib-resources: 6.4.0
jinja2: 3.1.4
markupsafe: 2.1.5
matplotlib: 3.9.0
numpy: 1.26.4
orjson: 3.10.6
packaging: 24.0
pandas: 2.2.2
pillow: 10.3.0
pydantic: 2.7.2
pydub: 0.25.1
python-multipart: 0.0.9
pyyaml: 6.0.1
ruff: 0.3.7
semantic-version: 2.10.0
tomlkit==0.12.0 is not installed.
typer: 0.12.3
typing-extensions: 4.12.1
urllib3: 2.2.1
uvicorn: 0.30.1
authlib; extra == 'oauth' is not installed.
itsdangerous; extra == 'oauth' is not installed.

gradio_client dependencies in your environment:

fsspec: 2024.5.0
httpx: 0.27.0
huggingface-hub: 0.23.2
packaging: 24.0
typing-extensions: 4.12.1
websockets: 11.0.3

Severity

I can work around it

irgolic commented 3 months ago

For completeness, when streaming data, the race condition often occurs at a different point.

In the above example, it happens when mapping components to their IDs https://github.com/gradio-app/gradio/blob/main/gradio/blocks.py#L1671

In the below example, it happens afterward, when trying to dereference the old components' IDs https://github.com/gradio-app/gradio/blob/main/gradio/blocks.py#L1711

Following the same instructions with this script:

import asyncio

import gradio as gr

with gr.Blocks() as demo:
    submit = gr.Button("Submit")

    out = gr.Textbox(key="1")
    outputs = [out]

    # out2 = gr.Textbox(key="2")
    # outputs = [out, out2]

    async def process():
        for i in range(100000):
            await asyncio.sleep(0.01)
            yield {out: i}

    submit.click(
        process,
        outputs=outputs,
    )

demo.launch()

Commonly produces this error (but can produce the other error too):

❯ gradio test.py
Watching: '/Users/rafael' '/Users/rafael'

Running on local URL:  http://127.0.0.1:7861

To create a public link, set `share=True` in `launch()`.
Changes detected in: /Users/rafael/test.py
Traceback (most recent call last):
  File "/opt/homebrew/Caskroom/miniconda/base/envs/my_env/lib/python3.11/site-packages/gradio/queueing.py", line 580, in process_events
    response = await route_utils.call_process_api(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/my_env/lib/python3.11/site-packages/gradio/route_utils.py", line 276, in call_process_api
    output = await app.get_blocks().process_api(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/my_env/lib/python3.11/site-packages/gradio/blocks.py", line 1938, in process_api
    data = await self.postprocess_data(block_fn, result["prediction"], state)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/my_env/lib/python3.11/site-packages/gradio/blocks.py", line 1759, in postprocess_data
    if block._id in state:
       ^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/my_env/lib/python3.11/site-packages/gradio/state_holder.py", line 104, in __contains__
    block = self.blocks_config.blocks[key]
            ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^
KeyError: 2

freddyaboulton commented 3 months ago

The ideal fix would be to preserve execution of the function, and map old output components to new ones under the hood via their key attribute. If the referenced output does not exist anymore, throw a "Returned component not specified as output of function" error as it does now.

This makes sense. Would you be interested in opening a PR for this @irgolic ?

irgolic commented 3 months ago

I can give it a shot. There should be a lock between the reloading thread and a function's outputs being processed. Any thoughts on where that should be managed?

gradio-app / gradio