m-bain / whisperX

WhisperX: Automatic Speech Recognition with Word-level Timestamps (& Diarization)
BSD 2-Clause "Simplified" License
12.6k stars 1.33k forks source link

Always crashes at alignment stage #198

Open Troceleng opened 1 year ago

Troceleng commented 1 year ago

I really like the quality in the text previews, but I cannot get any results out of this, as every time it gets to the alignment stage, it crashes, giving me this error:

Performing alignment... ╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮ │ C:\Program Files\Python39\lib\runpy.py:197 in _run_module_as_main │ │ │ │ 194 │ main_globals = sys.modules["main"].dict │ │ 195 │ if alter_argv: │ │ 196 │ │ sys.argv[0] = mod_spec.origin │ │ ❱ 197 │ return _run_code(code, main_globals, None, │ │ 198 │ │ │ │ │ "main", mod_spec) │ │ 199 │ │ 200 def run_module(mod_name, init_globals=None, │ │ │ │ C:\Program Files\Python39\lib\runpy.py:87 in _run_code │ │ │ │ 84 │ │ │ │ │ loader = loader, │ │ 85 │ │ │ │ │ package = pkg_name, │ │ 86 │ │ │ │ │ spec = mod_spec) │ │ ❱ 87 │ exec(code, run_globals) │ │ 88 │ return run_globals │ │ 89 │ │ 90 def _run_module_code(code, init_globals=None, │ │ │ │ C:\Program Files\Python39\Scripts\whisperx.exe__main.py:7 in │ │ │ │ [Errno 2] No such file or directory: 'C:\Program │ │ Files\Python39\Scripts\whisperx.exe\main.py' │ │ │ │ C:\Program Files\Python39\lib\site-packages\whisperx\transcribe.py:213 in cli │ │ │ │ 210 │ │ │ 211 │ # >> Write │ │ 212 │ for result, audio_path in results: │ │ ❱ 213 │ │ writer(result, audio_path) │ │ 214 │ │ │ │ 215 │ │ # cleanup │ │ 216 │ │ if input_audio_path != audio_path: │ │ │ │ C:\Program Files\Python39\lib\site-packages\whisperx\utils.py:308 in write_all │ │ │ │ 305 │ │ │ │ 306 │ │ def write_all(result: dict, file: TextIO): │ │ 307 │ │ │ for writer in all_writers: │ │ ❱ 308 │ │ │ │ writer(result, file) │ │ 309 │ │ │ │ 310 │ │ return write_all │ │ 311 │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ TypeError: call__() missing 1 required positional argument: 'options'

m-bain commented 1 year ago

Sorry the v2 release was not updated. If you reinstalll now it should work pip install git+https://github.com/m-bain/whisperx.git --upgrade

Troceleng commented 1 year ago

I just did this upgrade and still got this error:

Performing alignment... ╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮ │ C:\Program Files\Python39\lib\runpy.py:197 in _run_module_as_main │ │ │ │ 194 │ main_globals = sys.modules["main"].dict │ │ 195 │ if alter_argv: │ │ 196 │ │ sys.argv[0] = mod_spec.origin │ │ ❱ 197 │ return _run_code(code, main_globals, None, │ │ 198 │ │ │ │ │ "main", mod_spec) │ │ 199 │ │ 200 def run_module(mod_name, init_globals=None, │ │ │ │ C:\Program Files\Python39\lib\runpy.py:87 in _run_code │ │ │ │ 84 │ │ │ │ │ loader = loader, │ │ 85 │ │ │ │ │ package = pkg_name, │ │ 86 │ │ │ │ │ spec = mod_spec) │ │ ❱ 87 │ exec(code, run_globals) │ │ 88 │ return run_globals │ │ 89 │ │ 90 def _run_module_code(code, init_globals=None, │ │ │ │ C:\Program Files\Python39\Scripts\whisperx.exe__main.py:7 in │ │ │ │ [Errno 2] No such file or directory: 'C:\Program │ │ Files\Python39\Scripts\whisperx.exe\main.py' │ │ │ │ C:\Users\Troc\AppData\Roaming\Python\Python39\site-packages\whisperx\transcribe.py:213 in cli │ │ │ │ 210 │ │ │ 211 │ # >> Write │ │ 212 │ for result, audio_path in results: │ │ ❱ 213 │ │ writer(result, audio_path) │ │ 214 │ │ │ │ 215 │ │ # cleanup │ │ 216 │ │ if input_audio_path != audio_path: │ │ │ │ C:\Users\Troc\AppData\Roaming\Python\Python39\site-packages\whisperx\utils.py:308 in write_all │ │ │ │ 305 │ │ │ │ 306 │ │ def write_all(result: dict, file: TextIO): │ │ 307 │ │ │ for writer in all_writers: │ │ ❱ 308 │ │ │ │ writer(result, file) │ │ 309 │ │ │ │ 310 │ │ return write_all │ │ 311 │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ TypeError: call__() missing 1 required positional argument: 'options'

m-bain commented 1 year ago

mm double check you are using the right whisper version pip install openai-whisper==20230314

Troceleng commented 1 year ago

I tried to uninstall whisper and reinstall with that command, but I still get the same error. When reinstalling, it gives me a bunch of "already satisfied" though, so maybe that has to do with it?

Troceleng commented 1 year ago

If I uninstall everything Python from my Windows, which exact commands do I need to run in order to get whisperx to work?

gobbledygoober commented 1 year ago

I believe I have encountered this on Linux as well. Everything was installed from on May 6, 2023 on a fresh Ubuntu 22.04.2:

Console log:

[00:18.000 --> 00:20.000]  <Last two>
[00:20.000 --> 00:21.000]  <Transcriptions here>
>>Performing alignment...
Failed to align segment (""): no characters in this segment found in model dictionary, resorting to original...
Traceback (most recent call last):
  File "/home/user/.local/bin/whisperx", line 8, in <module>
    sys.exit(cli())
  File "/home/user/.local/lib/python3.10/site-packages/whisperx/transcribe.py", line 213, in cli
    writer(result, audio_path)
  File "/home/user/.local/lib/python3.10/site-packages/whisperx/utils.py", line 308, in write_all
    writer(result, file)
TypeError: ResultWriter.__call__() missing 1 required positional argument: 'options'
user@host:~/whisperX$ 

This is how whisperx was executed:

whisperx --model tiny.en --task translate --language en --diarize --hf_token <read_token_here> --threads 30 --output_dir output input.mp3

OS information:

user@host:~/whisperX$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.2 LTS
Release:        22.04
Codename:       jammy

user@host:~/whisperX$ uname -a
Linux host 5.15.0-71-generic #78-Ubuntu SMP Tue Apr 18 09:00:29 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

And package information:

user@host:~/whisperX$ apt list ffmpeg python3                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          
Listing... Done                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
ffmpeg/jammy-updates,jammy-security,now 7:4.4.2-0ubuntu0.22.04.1 amd64 [installed]                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            
python3/jammy-updates,now 3.10.6-1~22.04 amd64 [installed,automatic]                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          

user@host:~/whisperX$ pip list                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         
Package                   Version                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
------------------------- ---------------------                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
absl-py                   1.4.0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
aiohttp                   3.8.4                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
aiosignal                 1.3.1                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
alembic                   1.10.4                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
antlr4-python3-runtime    4.9.3                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
asteroid-filterbanks      0.4.0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
async-timeout             4.0.2                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
attrs                     21.2.0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
audioread                 3.0.0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
Automat                   20.2.0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
Babel                     2.8.0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
backports.cached-property 1.0.2                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
bcrypt                    3.2.0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
blinker                   1.4                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 
cachetools                5.3.0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
certifi                   2020.6.20                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           
cffi                      1.15.1                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
chardet                   4.0.0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
charset-normalizer        3.1.0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
click                     8.0.3                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
cloud-init                23.1.2                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
cmaes                     0.9.1                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
cmake                     3.26.3                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
colorama                  0.4.4                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
colorlog                  6.7.0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
command-not-found         0.3                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 
configobj                 5.0.6                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
constantly                15.1.0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
contourpy                 1.0.7                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
cryptography              3.4.8                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
cycler                    0.11.0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
dbus-python               1.2.18                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
decorator                 5.1.1                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
distro                    1.7.0                                                                                                                                                            
distro-info               1.1build1                                                                                                                                                        
docopt                    0.6.2                                                                                                                                                            
einops                    0.3.2                                                                                                                                                            
ffmpeg-python             0.2.0                                                                                                                                                            
filelock                  3.12.0                                                                                                                                                           
fonttools                 4.39.3                                                                                                                                                           
frozenlist                1.3.3                                                                                                                                                            
fsspec                    2023.4.0                                                                                                                                                         
future                    0.18.3                                                                                                                                                           
google-auth               2.17.3                                                                                                                                                           
google-auth-oauthlib      1.0.0                                                                                                                                                            
greenlet                  2.0.2                                                                                                                                                            
grpcio                    1.54.0                                                                                                                                                           
hmmlearn                  0.2.8.post35+g822894e                                                                                                                                            
httplib2                  0.20.2                                                                                                                                                           
huggingface-hub           0.14.1                                                                                                                                                           
hyperlink                 21.0.0                                                                                                                                                           
HyperPyYAML               1.2.0                                                                                                                                                            
idna                      3.3                                                                                                                                                              
importlib-metadata        4.6.4                                                                                                                                                            
incremental               21.3.0                                                                                                                                                           
jeepney                   0.7.1                                                                                                                                                            
Jinja2                    3.0.3                                                                                                                                                            
joblib                    1.2.0                                                                                                                                                            
jsonpatch                 1.32                                                                                                                                                             
jsonpointer               2.0                                                                                                                                                              
jsonschema                3.2.0                                                                                                                                                            
julius                    0.2.7                                                                                                                                                            
keyring                   23.5.0                                                                                                                                                           
kiwisolver                1.4.4                                                                                                                                                            
launchpadlib              1.10.16                                                                                                                                                          
lazr.restfulclient        0.14.4                                                                                                                                                           
lazr.uri                  1.0.6                                                                                                                                                            
librosa                   0.9.2                                                                                                                                                            
lit                       16.0.3                                                                                                                                                           
llvmlite                  0.40.0                                                                                                                                                           
Mako                      1.2.4                                                                                                                                                            
Markdown                  3.4.3                                                                                                                                                            
markdown-it-py            2.2.0                                                                                                                                                            
MarkupSafe                2.1.2                                                                                                                                                            
matplotlib                3.7.1                                                                                                                                                            
mdurl                     0.1.2                                                                                                                                                            
more-itertools            8.10.0                                                                                                                                                           
mpmath                    1.3.0                                                                                                                                                            
multidict                 6.0.4                                                                                                                                                            
netifaces                 0.11.0                                                                                                                                                           
networkx                  2.8.8                                                                                                                                                            
numba                     0.57.0                                                                                                                                                           
numpy                     1.24.3                                                                                                                                                           
nvidia-cublas-cu11        11.10.3.66                                                                                                                                                       
nvidia-cuda-cupti-cu11    11.7.101                                                                                                                                                         
nvidia-cuda-nvrtc-cu11    11.7.99                                                                                                                                                          
nvidia-cuda-runtime-cu11  11.7.99                                                                                                                                                          
nvidia-cudnn-cu11         8.5.0.96                                                                                                                                                         
nvidia-cufft-cu11         10.9.0.58                                                                                                                                                        
nvidia-curand-cu11        10.2.10.91                                                                                                                                                       
nvidia-cusolver-cu11      11.4.0.1                                                                                                                                                         
nvidia-cusparse-cu11      11.7.4.91                                                                                                                                                        
nvidia-nccl-cu11          2.14.3                                                                                                                                                           
nvidia-nvtx-cu11          11.7.91                                                                                                                                                          
oauthlib                  3.2.0                                                                                                                                                            
omegaconf                 2.3.0                                                                                                                                                            
openai-whisper            20230314                                                                                                                                                         
optuna                    3.1.1                                                                                                                                                            
packaging                 23.1                                                                                                                                                             
pandas                    2.0.1                                                                                                                                                            
pexpect                   4.8.0                                                                                                                                                            
Pillow                    9.5.0                                                                                                                                                            
pip                       23.1.2                                                                                                                                                           
platformdirs              3.5.0                                                                                                                                                            
pooch                     1.7.0                                                                                                                                                            
primePy                   1.3                                                                                                                                                              
protobuf                  3.20.1                                                                                                                                                           
ptyprocess                0.7.0                                                                                                                                                            
pyannote.audio            2.1.1                                                                                                                                                            
pyannote.core             4.5                                                                                                                                                              
pyannote.database         4.1.3                                                                                                                                                            
pyannote.metrics          3.2.1                                                                                                                                                            
pyannote.pipeline         2.3                                                                                                                                                              
pyasn1                    0.4.8                                                                                                                                                            
pyasn1-modules            0.2.1                                                                                                                                                            
pycparser                 2.21                                                                                                                                                             
pyDeprecate               0.3.2                                                                                                                                                            
Pygments                  2.15.1                                                                                                                                                           
PyGObject                 3.42.1                                                                                                                                                           
PyHamcrest                2.0.2                                                                                                                                                            
PyJWT                     2.3.0                                                                                                                                                            
pyOpenSSL                 21.0.0                                                                                                                                                           
pyparsing                 2.4.7                                                                                                                                                            
pyrsistent                0.18.1                                                                                                                                                           
pyserial                  3.5                                                                                                                                                              
python-apt                2.4.0+ubuntu1                                                                                                                                                    
python-dateutil           2.8.2                                                                                                                                                            
python-debian             0.1.43ubuntu1                                                                                                                                                    
python-magic              0.4.24                                                                                                                                                           
pytorch-lightning         1.6.5                                                                                                                                                            
pytorch-metric-learning   1.7.3                                                                                                                                                            
pytz                      2022.1                                                                                                                                                           
PyYAML                    5.4.1                                                                                                                                                            
regex                     2023.5.5                                                                                                                                                         
requests                  2.30.0                                                                                                                                                           
requests-oauthlib         1.3.1                                                                                                                                                            
resampy                   0.4.2                                                                                                                                                            
rich                      13.3.5                                                                                                                                                           
rsa                       4.9                                                                                                                                                              
ruamel.yaml               0.17.24                                                                                                                                                          
ruamel.yaml.clib          0.2.7                                                                                                                                                            
scikit-learn              1.2.2                                                                                                                                                            
scipy                     1.10.1                                                                                                                                                           
SecretStorage             3.3.1                                                                                                                                                            
semver                    2.13.0                                                                                                                                                           
sentencepiece             0.1.99                                                                                                                                                           
service-identity          18.1.0                                                                                                                                                           
setuptools                59.6.0                                                                                                                                                           
shellingham               1.5.0.post1                                                                                                                                                      
simplejson                3.19.1                                                                                                                                                           
singledispatchmethod      1.0                                                                                                                                                              
six                       1.16.0                                                                                                                                                           
sortedcontainers          2.4.0                                                                                                                                                            
sos                       4.4                                                                                                                                                              
SoundFile                 0.10.3.post1                                                                                                                                                     
speechbrain               0.5.14                                                                                                                                                           
SQLAlchemy                2.0.12                                                                                                                                                           
ssh-import-id             5.11                                                                                                                                                             
sympy                     1.11.1                                                                                                                                                           
systemd-python            234                                                                                                                                                              
tabulate                  0.9.0                                                                                                                                                            
tensorboard               2.13.0                                                                                                                                                                                                                                                                                                          
tensorboard-data-server   0.7.0
threadpoolctl             3.1.0
tiktoken                  0.3.3
tokenizers                0.13.3
torch                     1.11.0
torch-audiomentations     0.11.0
torch-pitch-shift         1.2.4
torchaudio                0.11.0
torchmetrics              0.11.4
torchvision               0.12.0
tqdm                      4.65.0
transformers              4.28.1
triton                    2.0.0
Twisted                   22.1.0
typer                     0.9.0
typing_extensions         4.5.0
tzdata                    2023.3
ubuntu-advantage-tools    8001
ubuntu-drivers-common     0.0.0
ufw                       0.36.1
unattended-upgrades       0.1
urllib3                   1.26.5
wadllib                   1.3.6
Werkzeug                  2.3.3
wheel                     0.37.1
whisperx                  2.0.1
xkit                      0.0.0
yarl                      1.9.2
zipp                      1.0.0
zope.interface            5.4.0
gobbledygoober commented 1 year ago

I started removing options to see if the problem still occurred. In case it is helpful, it still occurs even if I remove the diarize and hf_token options, or if I set the output_format.

Edit: Just noticed that v3 was merged -/ updating now and will retry shortly.

gobbledygoober commented 1 year ago

@Troceleng The issue went away for me using v3 as long as I added --compute_type int8 per #229

Curious if the problem went away for you too.

Troceleng commented 1 year ago

Curious if the problem went away for you too.

whisperx: error: unrecognized arguments: --compute_type int8

gobbledygoober commented 1 year ago

Curious if the problem went away for you too.

whisperx: error: unrecognized arguments: --compute_type int8

Have you upgraded from v2 to v3?