Initial WebSocket Error on First WhisperLive Deployment and Model Download

[mrtoorich]

The first time WhisperLive is deployed and a client request is made, the model is downloaded. However, at this time, there is always a WebSocket error. But subsequent client requests all proceed normally.

python3 run_server.py
config.json: 100%|████████████████████████████████████████████████████████████████████████████████████████████████| 2.39k/2.39k [00:00<00:00, 16.6MB/s]
preprocessor_config.json: 100%|███████████████████████████████████████████████████████████████████████████████████████| 340/340 [00:00<00:00, 2.30MB/s]
vocabulary.json: 100%|████████████████████████████████████████████████████████████████████████████████████████████| 1.07M/1.07M [00:00<00:00, 1.65MB/s]
tokenizer.json: 100%|█████████████████████████████████████████████████████████████████████████████████████████████| 2.48M/2.48M [00:00<00:00, 3.08MB/s]
model.bin: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████| 3.09G/3.09G [04:18<00:00, 11.9MB/s]
ERROR:websockets.server:connection handler failed
Traceback (most recent call last):████████████████████████████████████████████████████████████████████████████████| 3.09G/3.09G [04:18<00:00, 11.5MB/s]
  File "/home/ubuntu/.local/lib/python3.10/site-packages/websockets/sync/server.py", line 499, in conn_handler
    handler(connection)
  File "/home/ubuntu/WhisperLive/whisper_live/server.py", line 99, in recv_audio
    client = ServeClient(
  File "/home/ubuntu/WhisperLive/whisper_live/server.py", line 255, in __init__
    self.websocket.send(
  File "/home/ubuntu/.local/lib/python3.10/site-packages/websockets/sync/connection.py", line 284, in send
    with self.send_context():
  File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__
    return next(self.gen)
  File "/home/ubuntu/.local/lib/python3.10/site-packages/websockets/sync/connection.py", line 724, in send_context
    raise self.protocol.close_exc from original_exc
websockets.exceptions.ConnectionClosedError: no close frame received or sent

[mrtoorich]

Could anyone provide some hints? ❤️ I can submit a PR with the hints.

[makaveli10]

Not sure, i cannot reproduce this. Can you provide more details when this issue happens?

[mrtoorich]

Ok, I will provide more details within 24 hours. (I'm busy with my work now. 🤣 )

[mrtoorich]

Setup the server

on Ubuntu server 22.04

# Install NVIDIA drivers
sudo apt update
sudo DEBIAN_FRONTEND=noninteractive apt -y upgrade
sudo apt -y autoremove nvidia* --purge
ubuntu-drivers devices
sudo DEBIAN_FRONTEND=noninteractive ubuntu-drivers autoinstall
sudo apt install nvidia-driver-535
sudo reboot
nvidia-smi

# Install CUDA drivers
sudo apt -y install nvidia-cuda-toolkit
nvcc --version

# Install cuDNN
# Download: https://developer.nvidia.com/rdp/cudnn-download
# Upload: scp ~/Documents/AI/cudnn-local-repo-ubuntu2204-8.9.7.29_1.0-1_amd64.deb ubuntu@xxx:~
sudo apt install ./cudnn-local-repo-ubuntu2204-8.9.7.29_1.0-1_amd64.deb
sudo cp /var/cudnn-local-repo-ubuntu2204-8.9.7.29/cudnn-local-8AE81B24-keyring.gpg /usr/share/keyrings/
sudo apt update
sudo apt install libcudnn8 libcudnn8-dev libcudnn8-samples

# Install WhisperLive
git clone https://github.com/collabora/WhisperLive.git
cd WhisperLive
sudo bash setup.sh
pip install whisper-live
python3 run_server.py

Setup and run a WebSocket client

with the following initial message:

{
  "uid": "12e6b69d-94e0-4cdf-ada9-c5a3288f68c6",
  "multilingual": true,
  "language": "zh",
  "task": "transcribe",
  "model_size": "large-v3",
  "initial_prompt": "这是一段普通话对话，对话人有华晨、孙悟空。",
  "vad_parameters": { "threshold": 0.9 },
}

Run the server and the client, it connected, and the client send its initial message to the server, and then the server started to download large v3 model, after download finished, the error messages above shown.

[makaveli10]

@hcljsq this is something that i cant reproduce but i am curious this happens only on the first run?

[mrtoorich]

Yes, it only happens on the first run, after received the initial message that the server didn't have the model downloaded.

[makaveli10]

If you are using a docker container, can you share the tag? maybe it will be easier to reproduce.