Gradio Client - Repeated WebSocket Connection Rejection (HTTP 403)

[bibo7086]

I encountered an issue while using the Gradio client to connect to a Gradio application running at http://localhost:7860/. The client initially reports successful loading of the API, but attempts to establish a WebSocket connection are rejected by the server with an HTTP 403 (Forbidden) status code.

Error Message:

Loaded as API: http://localhost:7860/ ✔
GR job failed: server rejected WebSocket connection: HTTP 403
File "/path/to/miniconda3/envs/gradioclient/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
File "/path/to/miniconda3/envs/gradioclient/lib/python3.10/site-packages/gradio_client/client.py", line 798, in _inner
    predictions = _predict(*data)
File "/path/to/miniconda3/envs/gradioclient/lib/python3.10/site-packages/gradio_client/client.py", line 827, in _predict
    result = utils.synchronize_async(self._ws_fn, data, hash_data, helper)
File "/path/to/miniconda3/envs/gradioclient/lib/python3.10/site-packages/gradio_client/utils.py", line 540, in synchronize_async
    return fsspec.asyn.sync(fsspec.asyn.get_loop(), func, *args, **kwargs)  # type: ignore
File "/path/to/miniconda3/envs/gradioclient/lib/python3.10/site-packages/fsspec/asyn.py", line 103, in sync
    raise return_result
File "/path/to/miniconda3/envs/gradioclient/lib/python3.10/site-packages/fsspec/asyn.py", line 56, in _runner
    result[0] = await coro
File "/path/to/miniconda3/envs/gradioclient/lib/python3.10/site-packages/gradio_client/client.py", line 1055, in _ws_fn
    async with websockets.connect(  # type: ignore
File "/path/to/miniconda3/envs/gradioclient/lib/python3.10/site-packages/websockets/legacy/client.py", line 637, in __aenter__
    return await self
File "/path/to/miniconda3/envs/gradioclient/lib/python3.10/site-packages/websockets/legacy/client.py", line 655, in __await_impl_timeout__
    return await self.__await_impl__()
File "/path/to/miniconda3/envs/gradioclient/lib/python3.10/site-packages/websockets/legacy/client.py", line 662, in __await_impl__
    await protocol.handshake(
File "/path/to/miniconda3/envs/gradioclient/lib/python3.10/site-packages/websockets/legacy/client.py", line 329, in handshake
    raise InvalidStatusCode(status_code, response_headers)

Loaded as API: http://localhost:7860/ ✔
GR job failed again: server rejected WebSocket connection: HTTP 403
    ... (same stack trace as above)

Steps to Reproduce:

1. python generate.py --base_model=TheBloke/Mistral-7B-Instruct-v0.2-GGUF --prompt_type=mistral --max_seq_len=4096
2. Run the Gradio client code.

   
   import time
   import os
   import sys

from gradio_utils.grclient import GradioClient

self-contained example used for readme, to be copied to README_CLIENT.md if changed, setting local_server = True at first

The grclient.py file can be copied from h2ogpt repo and used with local gradio_client for example use

if local_server: client = GradioClient("http://0.0.0.0:7860") else: h2ogpt_key = os.getenv('H2OGPT_KEY') or os.getenv('H2OGPT_H2OGPT_KEY') if h2ogpt_key is None: sys.exit("API key not found. Exiting.")

if you have API key for public instance:

client = GradioClient("https://gpt.h2o.ai", h2ogpt_key=h2ogpt_key)

print(client.question("Who are you?"))

* **Environment:**
    * Operating System: Ubuntu 22.04.4 LTS
    * gradio: 4.26.0
    * gradio_client:  0.6.1

 * **Note:**
I suspect this is due to internal network restrictions and using 127.0.0.1 should ideally work. Unfortunately it doesn't.