Error occurred during backend connection.

[Epresin]

(whisper-playground) E:\whisper\whisper-playground>cd backend

(whisper-playground) E:\whisper\whisper-playground\backend>python server.py
The torchaudio backend is switched to 'soundfile'. Note that 'sox_io' is not supported on Windows.
The torchaudio backend is switched to 'soundfile'. Note that 'sox_io' is not supported on Windows.
======== Running on http://0.0.0.0:8000 ========
(Press CTRL+C to quit)
INFO:root:Client connected, initializing stream...
INFO:root:Stream configuration received: {'beamSize': '1', 'transcribeTimeout': '5', 'transcriptionMethod': 'real-time', 'model': 'small', 'language': 'english'}
WARNING:faster_whisper:An error occured while synchronizing the model guillaumekln/faster-whisper-small.en from the Hugging Face Hub:
Connection error, and we cannot find the requested files in the disk cache. Please try again or make sure your Internet connection is on.
WARNING:faster_whisper:Trying to load the model directly from the local cache, if it exists.
Exception in thread Thread-1:
Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\envs\whisper-playground\lib\threading.py", line 932, in _bootstrap_inner
    self.run()
  File "C:\ProgramData\Anaconda3\envs\whisper-playground\lib\threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "C:\ProgramData\Anaconda3\envs\whisper-playground\lib\asyncio\runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "C:\ProgramData\Anaconda3\envs\whisper-playground\lib\asyncio\base_events.py", line 616, in run_until_complete
    return future.result()
  File "E:\whisper\whisper-playground\backend\client_manager.py", line 15, in create_new_client
    new_client = initialize_client(sid, sio, config)
  File "E:\whisper\whisper-playground\backend\clients\utils.py", line 56, in initialize_client
    transcriber = WhisperTranscriber(model_name=whisper_model.value, language_code=language_code, beam_size=beam_size)
  File "E:\whisper\whisper-playground\backend\transcription\whisper_transcriber.py", line 29, in __init__
    self.model = WhisperModel(self.get_full_model_name(model_name, language_code), device=device,
  File "C:\ProgramData\Anaconda3\envs\whisper-playground\lib\site-packages\faster_whisper\transcribe.py", line 124, in __init__
    self.model = ctranslate2.models.Whisper(
RuntimeError: Unable to open file 'model.bin' in model 'C:\Users\admin\.cache\huggingface\hub\models--guillaumekln--faster-whisper-small.en\snapshots\4e49ce629e3fa4c3da596c602b212cb026910443'

"The web page is using the default settings with the 'small' model in English for real-time usage. However, when connecting to the backend, an error occurs. The backend seems to be using a previous version of the automatically downloaded model."

[ethanzrd]

Are you sure that you were connected to the internet? If there's an existing full installation (one that hadn't cut off) on your disk, this shouldn't be happening. Could you try with another model you haven't downloaded yet?

[Epresin]

(whisper) D:\whisper\whisper-playground>cd backend

(whisper) D:\whisper\whisper-playground\backend>python server.py
The torchaudio backend is switched to 'soundfile'. Note that 'sox_io' is not supported on Windows.
The torchaudio backend is switched to 'soundfile'. Note that 'sox_io' is not supported on Windows.
======== Running on http://0.0.0.0:8000 ========
(Press CTRL+C to quit)
INFO:root:Client connected, initializing stream...
INFO:root:Stream configuration received: {'model': 'small', 'transcribeTimeout': '5', 'language': 'english', 'transcriptionMethod': 'real-time', 'beamSize': '1'}
Downloading (…)26910443/config.json: 100%|████████████████████████████████████████| 2.66k/2.66k [00:00<00:00, 1.54MB/s]
Downloading (…)10443/tokenizer.json: 100%|████████████████████████████████████████| 2.13M/2.13M [00:00<00:00, 2.16MB/s]
Downloading (…)10443/vocabulary.txt: 100%|███████████████████████████████████████████| 422k/422k [00:00<00:00, 698kB/s]
Downloading model.bin: 100%|████████████████████████████████████████████████████████| 484M/484M [00:13<00:00, 36.5MB/s]]
Exception in thread Thread-1:██████████████████████████████████████████████████████▊| 482M/484M [00:13<00:00, 21.4MB/s]]
Traceback (most recent call last):
  File "C:\ProgramData\anaconda3\envs\whisper\lib\threading.py", line 980, in _bootstrap_inner
    self.run()
  File "C:\ProgramData\anaconda3\envs\whisper\lib\threading.py", line 917, in run
    self._target(*self._args, **self._kwargs)
  File "C:\ProgramData\anaconda3\envs\whisper\lib\asyncio\runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "C:\ProgramData\anaconda3\envs\whisper\lib\asyncio\base_events.py", line 647, in run_until_complete
    return future.result()
  File "D:\whisper\whisper-playground\backend\client_manager.py", line 15, in create_new_client
    new_client = initialize_client(sid, sio, config)
  File "D:\whisper\whisper-playground\backend\clients\utils.py", line 58, in initialize_client
    new_client = client_class(sid, socket, transcriber, transcription_timeout)
  File "D:\whisper\whisper-playground\backend\clients\RealTimeClient.py", line 19, in __init__
    self.diarization_pipeline = OnlineSpeakerDiarization(self.pipeline_config)
  File "C:\ProgramData\anaconda3\envs\whisper\lib\site-packages\diart\blocks\diarization.py", line 44, in __init__
    self.segmentation = SpeakerSegmentation(self._config.segmentation, self._config.device)
  File "C:\ProgramData\anaconda3\envs\whisper\lib\site-packages\diart\blocks\segmentation.py", line 17, in __init__
    self.model.to(self.device)
  File "C:\ProgramData\anaconda3\envs\whisper\lib\site-packages\diart\models.py", line 38, in to
    self.load()
  File "C:\ProgramData\anaconda3\envs\whisper\lib\site-packages\diart\models.py", line 35, in load
    self.model = self.get_model()
  File "C:\ProgramData\anaconda3\envs\whisper\lib\site-packages\diart\models.py", line 20, in __call__
    return pyannote_loader.get_model(self.model_info, self.hf_token)
  File "C:\ProgramData\anaconda3\envs\whisper\lib\site-packages\pyannote\audio\pipelines\utils\getter.py", line 75, in get_model
    model = Model.from_pretrained(
  File "C:\ProgramData\anaconda3\envs\whisper\lib\site-packages\pyannote\audio\core\model.py", line 786, in from_pretrained
    path_for_pl = hf_hub_download(
  File "C:\ProgramData\anaconda3\envs\whisper\lib\site-packages\huggingface_hub\utils\_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
  File "C:\ProgramData\anaconda3\envs\whisper\lib\site-packages\huggingface_hub\file_download.py", line 1181, in hf_hub_download
    headers = build_hf_headers(
  File "C:\ProgramData\anaconda3\envs\whisper\lib\site-packages\huggingface_hub\utils\_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
  File "C:\ProgramData\anaconda3\envs\whisper\lib\site-packages\huggingface_hub\utils\_headers.py", line 121, in build_hf_headers
    token_to_send = get_token_to_send(token)
  File "C:\ProgramData\anaconda3\envs\whisper\lib\site-packages\huggingface_hub\utils\_headers.py", line 153, in get_token_to_send
    raise LocalTokenNotFoundError(
huggingface_hub.utils._headers.LocalTokenNotFoundError: Token is required (`token=True`), but no token found. You need to provide a token or be logged in to Hugging Face with `huggingface-cli login` or `huggingface_hub.login`. See https://huggingface.co/settings/tokens.

I tried clearing the model files in the .cache\huggingface， and it redownloaded the model. I compared the model.bin file in the cache with the model.bin file from the official website and found that their MD5 checksums are the same. This time it displayed different error messages， Is it due to network issues on my end?

[ethanzrd]

I clearly forgot to include the important step of accepting the terms and conditions of these models, could you review the additional steps in the branch attached to this issue fix/huggingface and let me know if the issue is resolved? The previous issue you had is indeed resolved.

[Epresin]

According to the method you provided, it is up and running. The waveform of the sound can also be viewed on the webpage, but a few seconds after starting, the backend crashes and outputs the following content:

(whisper) D:\whisper\whisper-playground> cd backend

(whisper) D:\whisper\whisper-playground\backend>python server.py
The torchaudio backend is switched to 'soundfile'. Note that 'sox_io' is not supported on Windows.
The torchaudio backend is switched to 'soundfile'. Note that 'sox_io' is not supported on Windows.
======== Running on http://0.0.0.0:8000 ========
(Press CTRL+C to quit)
INFO:root:Client connected, initializing stream...
INFO:root:Stream configuration received: {'beamSize': '1', 'transcribeTimeout': '5', 'language': 'english', 'model': 'small', 'transcriptionMethod': 'real-time'}
Downloading pytorch_model.bin: 100%|██████████████████████████████████████████████| 17.7M/17.7M [00:12<00:00, 1.46MB/s]
Downloading (…)lve/main/config.yaml: 100%|█████████████████████████████████████████████| 318/318 [00:00<00:00, 159kB/s]
Downloading pytorch_model.bin: 100%|██████████████████████████████████████████████| 96.4M/96.4M [00:22<00:00, 4.37MB/s]
Downloading (…)lve/main/config.yaml: 100%|█████████████████████████████████████████| 2.00k/2.00k [00:00<00:00, 664kB/s]
INFO:root:Real-time transcription thread started
INFO:root:New chunks handler started
INFO:root:Stream start signaled to client
INFO:root:Transcription number 1 started
INFO:faster_whisper:Processing audio with duration 00:05.000
Could not load library cudnn_ops_infer64_8.dll. Error code 126
Please make sure cudnn_ops_infer64_8.dll is in your library path!

It seems to be due to a lack of CUDA. I will install it later and see if the issue gets resolved.

What CUDA version is the project using? I have tried CUDA versions 10.2 and 12, as well as cuDNN, but I am still encountering this error.

[ethanzrd]

I recommend you review this issue: Error code 126 - faster-whisper, it includes some fixes!

Edit: If you're struggling, let me know!