WebSocket Connection Error with Gunicorn and Multiple Workers

huoxiao-C commented 4 days ago

After mounting Chainlit to the FastAPI application, if the Gunicorn command is used to start the application and multiple processes are specified (using the -w option), an error will occur. The startup command is: gunicorn -w 2 -k uvicorn.workers.UvicornWorker main:app --bind 0.0.0.0:3010

bug description

If the method based on FastAPI provided on the Chainlit official website (https://docs.chainlit.io/deploy/api) is used to mount Chainlit, and then the application is started using the Gunicorn command with multiple processes specified, such as: gunicorn -w 2 -k uvicorn.workers.UvicornWorker main:app --bind 0.0.0.0:3010, an error will be encountered on the web UI stating "Unable to connect to server".

Additionally, the console will display the following errors:

[2024-09-24 11:03:23 +0800] [10116] [INFO] connection closed [2024-09-24 11:03:24 +0800] [10116] [INFO] ('127.0.0.1', 63124) - "WebSocket /chat/ws/socket.io/?EI0=4&transport=websocket&sid=TVH1xbI1HXjNDocwAAAK" 403 [2024-09-24 11:03:24 +0800] [10116] [INFO] connection rejected (403 Forbidden) [2024-09-24 11:03:24 +0800] [10116] [INFO] connection closed [2024-09-24 11:03:25 +0800] [10116] [INFO] ('127.0.0.1', 63129) - "WebSocket /chat/ws/socket.io/?EI0=4&transport=websocket&sid=i_R_YcNkA9SIqMBuAAAc" 403 [2024-09-24 11:03:25 +0800] [10116] [INFO] connection rejected (403 Forbidden) [2024-09-24 11:03:25 +0800] [10116] [INFO] connection closed

Chainlit mounting code:

` app = FastAPI() @app.get("/") def read_root(): return {"message": "Hello, World!"}

mount_chainlit(app=app, target="src/chainlit_app.py", path="/chat"). `

software information

chainlit：1.1.402
python：3.8.5
Browser：google chrome
OS：windows or linux

dosubot[bot] commented 4 days ago

Hello, @huoxiao-C! I'm here to help you with any bugs, questions, or contributions you have regarding Chainlit. Let's tackle this issue together!

I found a similar closed issue that might be relevant to your problem: Error Message "Could not reach the server" on mounting to FastAPI. The issue was resolved by pull request #1337, which addressed WebSocket connection issues when submounting Chainlit.

The specific changes introduced in pull request #1337 include modifications to the mount_chainlit function in backend/chainlit/utils.py. These changes ensure that the environment variables CHAINLIT_ROOT_PATH and CHAINLIT_SUBMOUNT are set correctly, and the Chainlit app is properly mounted to the specified path. Here is the relevant code snippet:

def mount_chainlit(app: FastAPI, target: str, path="/chainlit"):
    os.environ["CHAINLIT_ROOT_PATH"] = path
    os.environ["CHAINLIT_SUBMOUNT"] = "true"
    from chainlit.config import config, load_module
    from chainlit.server import app as chainlit_app

    config.run.root_path = path
    config.run.debug = os.environ.get("CHAINLIT_DEBUG", False)

    check_file(target)
    # Load the module provided by the user
    config.run.module_name = target
    load_module(config.run.module_name)

    ensure_jwt_secret()

    app.mount(path, chainlit_app)

These changes ensure that the WebSocket connections work correctly when Chainlit is submounted ^[1]^[2].

Additionally, there are specific configurations required in Gunicorn to support WebSocket connections for Chainlit. The relevant settings are managed through Uvicorn, which is used to run the Chainlit server. Here are the key configurations:

WebSocket Protocol: The ws_protocol environment variable can be set to specify the WebSocket protocol. By default, it is set to "auto".
```
ws_protocol = os.environ.get("UVICORN_WS_PROTOCOL", "auto")
```

Per-Message Deflate: The ws_per_message_deflate environment variable can be set to enable or disable per-message deflate. By default, it is enabled.

ws_per_message_deflate_env = os.environ.get("UVICORN_WS_PER_MESSAGE_DEFLATE", "true")
ws_per_message_deflate = ws_per_message_deflate_env.lower() in ["true", "1", "yes"]

These settings are then passed to the Uvicorn configuration when starting the server:

config = uvicorn.Config(
    app,
    host=host,
    port=port,
    ws=ws_protocol,
    log_level=log_level,
    ws_per_message_deflate=ws_per_message_deflate,
    ssl_keyfile=ssl_keyfile,
    ssl_certfile=ssl_certfile,
)

To summarize, you need to ensure that the UVICORN_WS_PROTOCOL and UVICORN_WS_PER_MESSAGE_DEFLATE environment variables are set appropriately to support WebSocket connections in Chainlit ^[3].

If you continue to experience issues, please verify that you are using the latest version of Chainlit and that the environment variables are correctly set.

To continue talking to Dosu, mention @dosu.

dokterbob commented 4 days ago

@dosu Could you suggest a more concise title for the issue?

@huoxiao-C Do let us know whether the bot helped you out!

dosubot[bot] commented 4 days ago

A more concise title for the issue could be:

"WebSocket Connection Error with Gunicorn and Multiple Workers" ^[1].

To continue talking to Dosu, mention @dosu.

Chainlit / chainlit