Open shaojun opened 5 months ago
Hi @shaojun are you installing Gradio on multiple replicas? Or can you tell us more about how this Gradio app is deployed?
Hi @shaojun are you installing Gradio on multiple replicas? Or can you tell us more about how this Gradio app is deployed?
I just run with vscode debug as started with single python file, in my laptop. I don't know what is multiple replicas.
This is super strange, I would not have expected this error. Can you try a fresh installation of gradio using pip
instead of conda
cc @aliabid94 if you have any ideas
I do install the gradio by pip though i am using miniconda. My ubuntu has a http and https proxy set, not sure it's related or not.
Ah yeah is it possible for you test without the proxy set?
I have unset https_proxy
and unset http_proxy
, and then re-run and refresh the app web page, it works.
so its confirmed caused by http proxy? but it does work in old version of gradio.
you can try export no_proxy="localhost,127.0.0.1"
before running the gradio app
I am also encountering this issue, running Gradio on Kubernetes with Ray Serve and a single replica:
(ProxyActor pid=7460) await app(scope, receive, sender)
(ProxyActor pid=7460) File "/opt/venv/lib/python3.10/site-packages/starlette/routing.py", line 72, in app
(ProxyActor pid=7460) response = await func(request)
(ProxyActor pid=7460) File "/opt/venv/lib/python3.10/site-packages/fastapi/routing.py", line 278, in app
(ProxyActor pid=7460) raw_response = await run_endpoint_function(
(ProxyActor pid=7460) File "/opt/venv/lib/python3.10/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
(ProxyActor pid=7460) return await dependant.call(**values)
(ProxyActor pid=7460) File "/opt/venv/lib/python3.10/site-packages/gradio/routes.py", line 747, in queue_join
(ProxyActor pid=7460) return await queue_join_helper(body, request, username)
(ProxyActor pid=7460) File "/opt/venv/lib/python3.10/site-packages/gradio/routes.py", line 765, in queue_join_helper
(ProxyActor pid=7460) success, event_id = await blocks._queue.push(
(ProxyActor pid=7460) File "/opt/venv/lib/python3.10/site-packages/gradio/queueing.py", line 225, in push
(ProxyActor pid=7460) raise KeyError(
(ProxyActor pid=7460) KeyError: 'Event not found in queue. If you are deploying this Gradio app with multiple replicas, please enable stickiness to ensure that all requests from the same user are routed to the same instance.'
Here is the serve config.yaml
:
proxy_location: EveryNode
http_options:
host: 0.0.0.0
port: 8000
applications:
- name: ChatBotGroup
route_prefix: /generate
import_path: demo:bot
runtime_env: {
"working_dir": "file:///root/app/app.zip",
"env_vars": {
"HF_HUB_ENABLE_HF_TRANSFER": "1",
}
}
deployments:
- name: ChatBot
user_config:
repo_id: "TheBloke/dolphin-2.5-mixtral-8x7b-GGUF"
model_file: "dolphin-2.5-mixtral-8x7b.Q4_K_M.gguf"
tensor_split: [0.6, 1, 1, 1]
ray_actor_options:
num_gpus: 4.0
num_replicas: 2
- name: ChatClient
route_prefix: /
import_path: demo:app
runtime_env: {
"working_dir": "file:///root/app/app.zip",
"env_vars": {
"CONCURRENCY_LIMIT": "2",
"TITLE": "Chat - Dolphin 2.5 Mixtral 8x7b",
"CHAT_LABEL": "dolphin-2.5-mixtral-8x7b.Q4_K_M.gguf",
}
}
deployments:
- name: ChatIngress
num_replicas: 1
Downgrading with pip install gradio==4.24.0
fixed the issue.
Downgrading with pip install gradio==4.24.0 fixed the issue.
This didn't work for me, but
you can try export no_proxy="localhost,127.0.0.1" before running the gradio app
This worked.
None of the above suggestion are working for me.
I have the same error, deploying gradio 4.27.0
on docker compose under FastAPI 0.110.3
. I can access the Gradio chat example fine under localhost:8000/gradio
on the host, but once I try to send a message, the same error occurs: KeyError: 'Event not found in queue. If you are deploying this Gradio app with multiple replicas, please enable stickiness to ensure that all requests from the same user are routed to the same instance.'
I run FastAPI using uvicorn: uvicorn gradio_app:app --host 0.0.0.0
In gradio_app.py
:
from fastapi import FastAPI, Request
import gradio as gr
import requests
import datetime
CUSTOM_PATH = "/gradio"
app = FastAPI()
def greet(name):
return "Hello " + name + "!"
io = gr.Interface(fn=greet, inputs="textbox", outputs="textbox")
gradio_app = gr.routes.App.create_app(io)
app.mount(CUSTOM_PATH, gradio_app)
I'm setting the following environment variables in the dockerfile (from python:3-slim
):
ENV GRADIO_SERVER_NAME="0.0.0.0"
ENV COMMANDLINE_ARGS="--no-gradio-queue"
I can provide a full minimal example if needed.
you can try
export no_proxy="localhost,127.0.0.1"
before running the gradio app
Thanks, this works for me!
@shaojun (or anybody else who faced this issue), if you can provide more details about the proxy you're using and how it's configured, we might be able to provide a solution for this.
@shaojun (or anybody else who faced this issue), if you can provide more details about the proxy you're using and how it's configured, we might be able to provide a solution for this. (或遇到此问题的任何其他人),如果您可以提供有关您正在使用的代理及其配置方式的更多详细信息,我们也许能够为此提供解决方案。
I encountered a KeyError: 'Event not found in queue'
when running a Python program that uses Gradio on an Ubuntu 22.04 server, connected via SSH from a Windows machine(Same LAN). This setup involves a reverse proxy using the following environment variables:
export http_proxy=http://<windows-ip>:<port>
export https_proxy=http://<windows-ip>:<port>
where
Steps to Reproduce Set up a reverse proxy using the above environment variables. Run a Python program with Gradio on the server.
Error Message
KeyError: 'Event not found in queue. If you are deploying this Gradio app with multiple replicas, please enable stickiness to ensure that all requests from the same user are routed to the same instance.'
Temporary Solution Adding export no_proxy="localhost,127.0.0.1" to the environment variables before running the Gradio app resolves the issue.
Which reverse proxy library /software are you using?
I'm using clash for windows(CFW), can this help?
Thanks @iwannabewater I think that should be enough for us to repro this. I'll take a look!
I have
unset https_proxy
andunset http_proxy
, and then re-run and refresh the app web page, it works. so its confirmed caused by http proxy? but it does work in old version of gradio.
I have updated to gradio 4.31.1 and still see this error with http_proxy
and https_proxy
set.
And by unset http_proxy
and unset https_proxy
can fix this issue.
Describe the bug
followed exactly the official tutorial, the web page can show correctly, but if I entered sth in textbox, and click submit, the exception show:
can get to work by downgrade to
4.8.0
:Have you searched existing issues? 🔎
Reproduction
Screenshot
No response
Logs
No response
System Info
Severity
Blocking usage of gradio