search

SYSTRAN / faster-whisper

Faster Whisper transcription with CTranslate2
MIT License
11.6k stars 962 forks source link

can't get it to use GPU #941

Open M2ATrail opened 2 months ago

M2ATrail commented 2 months ago

so I've been trying to use whisperX but can't get it to work, so decided to test with faster whisper since it is built on that. but I still can't get it working, it only uses CPU. I've also downloaded the 11 and 12 files from here and pasted them in the bin https://github.com/Purfview/whisper-standalone-win/releases/tag/libs image

# $ conda create --name <env> --file <this file>
# platform: win-64
aiohttp=3.9.5=pypi_0
aiosignal=1.3.1=pypi_0
alembic=1.13.2=pypi_0
antlr4-python3-runtime=4.9.3=pypi_0
asteroid-filterbanks=0.4.0=pypi_0
asttokens=2.4.1=pyhd8ed1ab_0
async-timeout=4.0.3=pypi_0
attrs=23.2.0=pypi_0
audioread=3.0.1=pypi_0
av=12.3.0=pypi_0
blas=1.0=mkl
brotli-python=1.1.0=py310h00ffb61_1
bzip2=1.0.8=h2466b09_7
ca-certificates=2024.7.4=h56e8100_0
certifi=2024.7.4=pyhd8ed1ab_0
cffi=1.16.0=py310h8d17308_0
charset-normalizer=3.3.2=pyhd8ed1ab_0
click=8.1.7=pypi_0
colorama=0.4.6=pyhd8ed1ab_0
coloredlogs=15.0.1=pypi_0
colorlog=6.8.2=pypi_0
comm=0.2.2=pyhd8ed1ab_0
contourpy=1.2.1=pypi_0
ctranslate2=4.3.1=pypi_0
cuda-cccl=12.5.39=0
cuda-cccl_win-64=12.5.39=0
cuda-cudart=12.1.105=0
cuda-cudart-dev=12.1.105=0
cuda-cupti=12.1.105=0
cuda-libraries=12.1.0=0
cuda-libraries-dev=12.1.0=0
cuda-nvrtc=12.1.105=0
cuda-nvrtc-dev=12.1.105=0
cuda-nvtx=12.1.105=0
cuda-opencl=12.5.39=0
cuda-opencl-dev=12.5.39=0
cuda-profiler-api=12.5.39=0
cuda-runtime=12.1.0=0
cuda-version=12.5=3
cycler=0.12.1=pypi_0
debugpy=1.8.2=py310h9e98ed7_0
decorator=5.1.1=pyhd8ed1ab_0
docopt=0.6.2=pypi_0
einops=0.8.0=pypi_0
exceptiongroup=1.2.2=pyhd8ed1ab_0
executing=2.0.1=pyhd8ed1ab_0
faster-whisper=1.0.3=pypi_0
filelock=3.15.4=pyhd8ed1ab_0
flatbuffers=24.3.25=pypi_0
fonttools=4.53.1=pypi_0
freetype=2.12.1=hdaf720e_2
frozenlist=1.4.1=pypi_0
fsspec=2024.6.1=pypi_0
greenlet=3.0.3=pypi_0
h2=4.1.0=pyhd8ed1ab_0
hpack=4.0.0=pyh9f0ad1d_0
huggingface-hub=0.24.2=pypi_0
humanfriendly=10.0=pypi_0
hyperframe=6.0.1=pyhd8ed1ab_0
hyperpyyaml=1.2.2=pypi_0
idna=3.7=pyhd8ed1ab_0
importlib-metadata=8.2.0=pyha770c72_0
importlib_metadata=8.2.0=hd8ed1ab_0
intel-openmp=2024.2.0=h57928b3_980
ipykernel=6.29.5=pyh4bbf305_0
ipython=8.26.0=pyh7428d3b_0
jedi=0.19.1=pyhd8ed1ab_0
jinja2=3.1.4=pyhd8ed1ab_0
joblib=1.4.2=pypi_0
julius=0.2.7=pypi_0
jupyter_client=8.6.2=pyhd8ed1ab_0
jupyter_core=5.7.2=py310h5588dad_0
kiwisolver=1.4.5=pypi_0
krb5=1.21.3=hdf4eb48_0
lazy-loader=0.4=pypi_0
lcms2=2.16=h67d730c_0
lerc=4.0.0=h63175ca_0
libblas=3.9.0=1_h8933c1f_netlib
libcblas=3.9.0=5_hd5c7e75_netlib
libcublas=12.1.0.26=0
libcublas-dev=12.1.0.26=0
libcufft=11.0.2.4=0
libcufft-dev=11.0.2.4=0
libcurand=10.3.6.82=0
libcurand-dev=10.3.6.82=0
libcusolver=11.4.4.55=0
libcusolver-dev=11.4.4.55=0
libcusparse=12.0.2.55=0
libcusparse-dev=12.0.2.55=0
libdeflate=1.20=hcfcfb64_0
libffi=3.4.2=h8ffe710_5
libhwloc=2.11.1=default_h8125262_1000
libiconv=1.17=hcfcfb64_2
libjpeg-turbo=3.0.0=hcfcfb64_1
liblapack=3.9.0=5_hd5c7e75_netlib
libnpp=12.0.2.50=0
libnpp-dev=12.0.2.50=0
libnvjitlink=12.1.105=0
libnvjitlink-dev=12.1.105=0
libnvjpeg=12.1.1.14=0
libnvjpeg-dev=12.1.1.14=0
libpng=1.6.43=h19919ed_0
librosa=0.10.2.post1=pypi_0
libsodium=1.0.18=h8d14728_1
libsqlite=3.46.0=h2466b09_0
libtiff=4.6.0=hddb2be6_3
libuv=1.48.0=hcfcfb64_0
libwebp-base=1.4.0=hcfcfb64_0
libxcb=1.16=hcd874cb_0
libxml2=2.12.7=h0f24e4e_4
libzlib=1.3.1=h2466b09_1
lightning=2.3.3=pypi_0
lightning-utilities=0.11.6=pypi_0
llvmlite=0.43.0=pypi_0
m2w64-gcc-libgfortran=5.3.0=6
m2w64-gcc-libs=5.3.0=7
m2w64-gcc-libs-core=5.3.0=7
m2w64-gmp=6.1.0=2
m2w64-libwinpthread-git=5.0.0.4634.697f757=2
mako=1.3.5=pypi_0
markdown-it-py=3.0.0=pypi_0
markupsafe=2.1.5=py310h8d17308_0
matplotlib=3.9.1=pypi_0
matplotlib-inline=0.1.7=pyhd8ed1ab_0
mdurl=0.1.2=pypi_0
mkl=2023.1.0=h6a75c08_48682
mpmath=1.3.0=pyhd8ed1ab_0
msgpack=1.0.8=pypi_0
msys2-conda-epoch=20160418=1
multidict=6.0.5=pypi_0
nest-asyncio=1.6.0=pyhd8ed1ab_0
networkx=3.3=pyhd8ed1ab_1
nltk=3.8.1=pypi_0
numba=0.60.0=pypi_0
numpy=1.26.4=pypi_0
omegaconf=2.3.0=pypi_0
onnxruntime=1.18.1=pypi_0
openjpeg=2.5.2=h3d672ee_0
openssl=3.3.1=h2466b09_2
optuna=3.6.1=pypi_0
packaging=24.1=pyhd8ed1ab_0
pandas=2.2.2=pypi_0
parso=0.8.4=pyhd8ed1ab_0
pickleshare=0.7.5=py_1003
pillow=10.4.0=py310h3e38d90_0
pip=24.0=pyhd8ed1ab_0
platformdirs=4.2.2=pyhd8ed1ab_0
pooch=1.8.2=pypi_0
primepy=1.3=pypi_0
prompt-toolkit=3.0.47=pyha770c72_0
protobuf=5.27.2=pypi_0
psutil=6.0.0=py310ha8f682b_0
pthread-stubs=0.4=hcd874cb_1001
pthreads-win32=2.9.1=hfa6e2cd_3
pure_eval=0.2.3=pyhd8ed1ab_0
pyannote-audio=3.1.1=pypi_0
pyannote-core=5.0.0=pypi_0
pyannote-database=5.1.0=pypi_0
pyannote-metrics=3.2.1=pypi_0
pyannote-pipeline=3.0.1=pypi_0
pycparser=2.22=pyhd8ed1ab_0
pygments=2.18.0=pyhd8ed1ab_0
pyparsing=3.1.2=pypi_0
pyreadline3=3.4.1=pypi_0
pysocks=1.7.1=pyh0701188_6
python=3.10.14=h4de0772_0_cpython
python-dateutil=2.9.0.post0=pypi_0
python_abi=3.10=4_cp310
pytorch=2.4.0=py3.10_cuda12.1_cudnn9_0
pytorch-cuda=12.1=hde6ce7c_5
pytorch-lightning=2.3.3=pypi_0
pytorch-metric-learning=2.6.0=pypi_0
pytorch-mutex=1.0=cuda
pytz=2024.1=pypi_0
pywin32=306=py310h00ffb61_2
pyyaml=6.0.1=py310h8d17308_1
pyzmq=26.0.3=py310h656833d_0
regex=2024.7.24=pypi_0
requests=2.32.3=pyhd8ed1ab_0
rich=13.7.1=pypi_0
ruamel-yaml=0.18.6=pypi_0
ruamel-yaml-clib=0.2.8=pypi_0
safetensors=0.4.3=pypi_0
scikit-learn=1.5.1=pypi_0
scipy=1.14.0=pypi_0
semver=3.0.2=pypi_0
sentencepiece=0.2.0=pypi_0
setuptools=71.0.4=pyhd8ed1ab_0
shellingham=1.5.4=pypi_0
six=1.16.0=pyh6c4a22f_0
sortedcontainers=2.4.0=pypi_0
soundfile=0.12.1=pypi_0
soxr=0.4.0=pypi_0
speechbrain=1.0.0=pypi_0
sqlalchemy=2.0.31=pypi_0
stack_data=0.6.2=pyhd8ed1ab_0
sympy=1.13.0=pyh04b8f61_3
tabulate=0.9.0=pypi_0
tbb=2021.12.0=hc790b64_3
tensorboardx=2.6.2.2=pypi_0
threadpoolctl=3.5.0=pypi_0
tk=8.6.13=h5226925_1
tokenizers=0.19.1=pypi_0
torch-audiomentations=0.11.1=pypi_0
torch-pitch-shift=1.2.4=pypi_0
torchaudio=2.4.0=pypi_0
torchmetrics=1.4.0.post0=pypi_0
torchvision=0.19.0=pypi_0
tornado=6.4.1=py310ha8f682b_0
tqdm=4.66.4=pypi_0
traitlets=5.14.3=pyhd8ed1ab_0
transformers=4.43.3=pypi_0
typer=0.12.3=pypi_0
typing_extensions=4.12.2=pyha770c72_0
tzdata=2024.1=pypi_0
ucrt=10.0.22621.0=h57928b3_0
urllib3=2.2.2=pyhd8ed1ab_1
vc=14.3=h8a93ad2_20
vc14_runtime=14.40.33810=ha82c5b3_20
vs2015_runtime=14.40.33810=h3bf8584_20
wcwidth=0.2.13=pyhd8ed1ab_0
wheel=0.43.0=pyhd8ed1ab_1
whisperx=3.1.1=dev_0
win_inet_pton=1.1.0=pyhd8ed1ab_6
xorg-libxau=1.0.11=hcd874cb_0
xorg-libxdmcp=1.1.3=hcd874cb_0
xz=5.2.6=h8d14728_0
yaml=0.2.5=h8ffe710_2
yarl=1.9.4=pypi_0
zeromq=4.3.5=he1f189c_4
zipp=3.19.2=pyhd8ed1ab_0
zstandard=0.23.0=py310he5e10e1_0
zstd=1.5.6=h0ea2cb4_0
YaserHabib commented 2 months ago

I did solve it by coping all the dll files in (cuBLAS.and.cuDNN_CUDA11_win_v2.7z) to C:\Windows\System32 folder.

M2ATrail commented 2 months ago

@YaserHabib Thanks for reply, the files are already in my PATH though, so should be accessible. and before I copied them, it would throw an error about not finding the files, so it certainly can access it. Torch also commands says it is available too.

waan1 commented 1 month ago

Fist simple example: from faster_whisper import WhisperModel

model_size = "large-v3"

Run on GPU with FP16

model = WhisperModel(model_size, device="cuda", compute_type="float16")

Gives me an error:

Could not load library libcudnn_ops_infer.so.8. Error: libcudnn_ops_infer.so.8: cannot open shared object file: No such file or directory zsh: IOT instruction (core dumped) python test1.py

I have cuDNN 9.2.1.18-1 on manjaro linux. As I understand it wants version 8. When I try to install version 8 with manjaro package manager, it warns me that it will uninstall CUDA 12 and replace it with CUDA 11 that I certainly do not want to do.

If there is something I'm doing wrong, please advise. Otherwise it looks like faster-whisper is not for my server.

DeoLeung commented 1 month ago

same issue using torch 2.4.0 image which use cudnn9, how to make faster whisper compatible with cudnn9? I tried to install from source and still reading cudnn8