Using RunPod proxy, the speed of the Gradio app is very slow

[FurkanGozukara]

Describe the bug

I am running gradio 4.7.1 and using this web app : https://github.com/PixArt-alpha/PixArt-alpha/blob/master/scripts/app.py

When using Gradio share or running locally it is working great

When I use

The gradio starts but when I do inference this is the error

ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/workspace/PixArt-alpha/venv/lib/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py", line 408, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/workspace/PixArt-alpha/venv/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
  File "/workspace/PixArt-alpha/venv/lib/python3.10/site-packages/fastapi/applications.py", line 1106, in __call__
    await super().__call__(scope, receive, send)
  File "/workspace/PixArt-alpha/venv/lib/python3.10/site-packages/starlette/applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/workspace/PixArt-alpha/venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/workspace/PixArt-alpha/venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/workspace/PixArt-alpha/venv/lib/python3.10/site-packages/starlette/middleware/cors.py", line 91, in __call__
    await self.simple_response(scope, receive, send, request_headers=headers)
  File "/workspace/PixArt-alpha/venv/lib/python3.10/site-packages/starlette/middleware/cors.py", line 146, in simple_response
    await self.app(scope, receive, send)
  File "/workspace/PixArt-alpha/venv/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/workspace/PixArt-alpha/venv/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/workspace/PixArt-alpha/venv/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__
    raise e
  File "/workspace/PixArt-alpha/venv/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__
    await self.app(scope, receive, send)
  File "/workspace/PixArt-alpha/venv/lib/python3.10/site-packages/starlette/routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "/workspace/PixArt-alpha/venv/lib/python3.10/site-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/workspace/PixArt-alpha/venv/lib/python3.10/site-packages/starlette/routing.py", line 66, in app
    response = await func(request)
  File "/workspace/PixArt-alpha/venv/lib/python3.10/site-packages/fastapi/routing.py", line 274, in app
    raw_response = await run_endpoint_function(
  File "/workspace/PixArt-alpha/venv/lib/python3.10/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
    return await dependant.call(**values)
  File "/workspace/PixArt-alpha/venv/lib/python3.10/site-packages/gradio/routes.py", line 668, in queue_data
    blocks._queue.attach_data(body)
  File "/workspace/PixArt-alpha/venv/lib/python3.10/site-packages/gradio/queueing.py", line 161, in attach_data
    raise ValueError("Event not found", event_id)
ValueError: ('Event not found', 'bd936e1771ab4cafb985146ff9556260')

root@81bd59c6210f:/workspace/PixArt-alpha/venv# cd /workspace/PixArt-alpha
export HF_HOME="/workspace"
source venv/bin/activate
(venv) root@81bd59c6210f:/workspace/PixArt-alpha# pip freeze
accelerate==0.24.1
aiofiles==23.2.1
aiohttp==3.9.1
aiosignal==1.3.1
altair==5.1.2
annotated-types==0.6.0
anyio==3.7.1
async-timeout==4.0.3
attrs==23.1.0
beautifulsoup4==4.12.2
bitsandbytes==0.41.2.post2
certifi==2023.11.17
charset-normalizer==3.3.2
click==8.1.7
cmake==3.27.7
colorama==0.4.6
coloredlogs==15.0.1
contourpy==1.2.0
cycler==0.12.1
datasets==2.15.0
diffusers @ git+https://github.com/huggingface/diffusers@50a749e90990932d49556ee54a333278559722f3
dill==0.3.7
exceptiongroup==1.2.0
fastapi==0.104.1
ffmpy==0.3.1
filelock==3.13.1
fonttools==4.45.1
frozenlist==1.4.0
fsspec==2023.10.0
ftfy==6.1.3
gradio==4.7.1
gradio_client==0.7.0
h11==0.14.0
httpcore==1.0.2
httpx==0.25.2
huggingface-hub==0.19.4
humanfriendly==10.0
idna==3.6
importlib-metadata==6.8.0
importlib-resources==6.1.1
Jinja2==3.1.2
jsonschema==4.20.0
jsonschema-specifications==2023.11.1
kiwisolver==1.4.5
lit==17.0.5
markdown-it-py==3.0.0
MarkupSafe==2.1.3
matplotlib==3.8.2
mdurl==0.1.2
mpmath==1.3.0
multidict==6.0.4
multiprocess==0.70.15
networkx==3.2.1
numpy==1.26.2
optimum==1.14.1
orjson==3.9.10
packaging==23.2
pandas==2.1.3
Pillow==10.1.0
protobuf==4.25.1
psutil==5.9.6
pyarrow==14.0.1
pyarrow-hotfix==0.6
pydantic==2.5.2
pydantic_core==2.14.5
pydub==0.25.1
Pygments==2.17.2
pyparsing==3.1.1
python-dateutil==2.8.2
python-multipart==0.0.6
pytz==2023.3.post1
PyYAML==6.0.1
referencing==0.31.0
regex==2023.10.3
requests==2.31.0
rich==13.7.0
rpds-py==0.13.1
safetensors==0.4.1
semantic-version==2.10.0
sentencepiece==0.1.99
shellingham==1.5.4
six==1.16.0
sniffio==1.3.0
soupsieve==2.5
starlette==0.27.0
sympy==1.12
tokenizers==0.15.0
tomlkit==0.12.0
toolz==0.12.0
torch==2.0.1+cu118
tqdm==4.66.1
transformers==4.35.2
triton==2.0.0
typer==0.9.0
typing_extensions==4.8.0
tzdata==2023.3
urllib3==2.1.0
uvicorn==0.24.0.post1
wcwidth==0.2.12
websockets==11.0.3
xxhash==3.4.1
yarl==1.9.3
zipp==3.17.0
(venv) root@81bd59c6210f:/workspace/PixArt-alpha#

Severity

Blocking usage of gradio

Example working on local

[JindaHuang]

Hi @FurkanGozukara I faced the same issue while running Gradio project in docker container. Have you resolved this issue?

[FurkanGozukara]

Hi @FurkanGozukara I faced the same issue while running Gradio project in docker container. Have you resolved this issue?

nope i couldn't. only share option working

[abidlabs]

@FurkanGozukara @JindaHuang We may have a fix for you here: https://github.com/gradio-app/gradio/pull/6556

Could you try installing gradio from this branch? You can do it by doing:

pip install https://gradio-builds.s3.amazonaws.com/756e3431d65172df986a7e335dce8136206a293a/gradio-4.7.1-py3-none-any.whl

[FurkanGozukara]

@FurkanGozukara @JindaHuang We may have a fix for you here: #6556

Could you try installing gradio from this branch? You can do it by doing:

pip install https://gradio-builds.s3.amazonaws.com/756e3431d65172df986a7e335dce8136206a293a/gradio-4.7.1-py3-none-any.whl

sorry for delay i just tested

it works but interface update speed is very slow

[abidlabs]

Ok I've renamed the issue, we'll take a look into this when we get a chance. If there are any other details you can provide about why it might be slow, that would be helpful!

[FurkanGozukara]

Ok I've renamed the issue, we'll take a look into this when we get a chance. If there are any other details you can provide about why it might be slow, that would be helpful!

I noticed slowness from update frequency of the step progress %. Normally it updates % of steps after each step but it was like 5 6x slower. So instead of like every second it was like once 5 6 seconds I don't know sadly why could be.

[abidlabs]

Hi @FurkanGozukara can you please check if the latest version of gradio (4.21) still has this issue? We made some fixes to how Gradio works behind proxies which may have solved this issue

[FurkanGozukara]

Hi @FurkanGozukara can you please check if the latest version of gradio (4.21) still has this issue? We made some fixes to how Gradio works behind proxies which may have solved this issue

it works but i got this error like after 100 seconds of working any ideas?

I made it even like this and still didnt fix

if args.port is not None:  # Check if the --port argument is provided
    block.queue().launch(server_name=server_ip, server_port=args.port, share=args.share, inbrowser=True)
else:
    block.queue().launch(server_name=server_ip, share=args.share, inbrowser=True)

The app works perfectly fine. The gradio timesout

[pngwn]

524 is a Cloudflare specific timeout I think. Weird.

[FurkanGozukara]

524 is a Cloudflare specific timeout I think. Weird.

Gradio live share working

In past I had this error and queue() was solving but this time didnt

[abidlabs]

@FurkanGozukara could you see if this happens with any Gradio app with a long running predict function? A simple repro would be helpful here as we look into it

[FurkanGozukara]

@FurkanGozukara could you see if this happens with any Gradio app with a long running predict function? A simple repro would be helpful here as we look into it

i can start a machine and give you gradio live share link and runpod proxy link when is a good time?

[abidlabs]

Hi @FurkanGozukara, apologies for the late follow up. We haven't had a chance to look into this issue, but the Gradio codebase has changed quite significantly since this issue was created, particularly with the release of Gradio 5. Could you let us know if this is still an issue in the latest version of Gradio (pip install --upgrade gradio)? Thanks!