Error In ffmpeg on docker container

[Texugooftomorrow2]

Recently whern I try to do a docker-compose up -d it gets stuck on the middle of the process, by running the container log it displays:

whisper has been successfully installed. Traceback (most recent call last): File "/subgen/./subgen.py", line 43, in import ffmpeg ModuleNotFoundError: No module named 'ffmpeg'

And gets stuck here.

[McCloudS]

Look further up in your log and see if the ffmpeg module is installing. Either it's failing to install it, or it's looking in the wrong place for it.

I may rebuild the images soon to have them include ffmpeg instead of downloading them via the script. I originally did that to make them smaller to upload and download, but that doesn't matter as much for something that's only downloading once.

[Texugooftomorrow2]

here is the full log output from a clean run from zero:

Installing numpy... Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (1.26.3) WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv numpy has been successfully installed. Installing stable-ts... Requirement already satisfied: stable-ts in /usr/local/lib/python3.10/dist-packages (2.15.1) Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from stable-ts) (1.26.3) Requirement already satisfied: openai-whisper==20231117 in /usr/local/lib/python3.10/dist-packages (from stable-ts) (20231117) Requirement already satisfied: torch in /usr/local/lib/python3.10/dist-packages (from stable-ts) (2.1.2) Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from stable-ts) (4.66.1) Requirement already satisfied: torchaudio in /usr/local/lib/python3.10/dist-packages (from stable-ts) (2.1.2) Requirement already satisfied: numba in /usr/local/lib/python3.10/dist-packages (from openai-whisper==20231117->stable-ts) (0.58.1) Requirement already satisfied: triton<3,>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from openai-whisper==20231117->stable-ts) (2.1.0) Requirement already satisfied: more-itertools in /usr/local/lib/python3.10/dist-packages (from openai-whisper==20231117->stable-ts) (10.2.0) Requirement already satisfied: tiktoken in /usr/local/lib/python3.10/dist-packages (from openai-whisper==20231117->stable-ts) (0.5.2) Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from torch->stable-ts) (2023.12.2) Requirement already satisfied: nvidia-cuda-cupti-cu12==12.1.105 in /usr/local/lib/python3.10/dist-packages (from torch->stable-ts) (12.1.105) Requirement already satisfied: nvidia-cublas-cu12==12.1.3.1 in /usr/local/lib/python3.10/dist-packages (from torch->stable-ts) (12.1.3.1) Requirement already satisfied: nvidia-curand-cu12==10.3.2.106 in /usr/local/lib/python3.10/dist-packages (from torch->stable-ts) (10.3.2.106) Requirement already satisfied: nvidia-cusparse-cu12==12.1.0.106 in /usr/local/lib/python3.10/dist-packages (from torch->stable-ts) (12.1.0.106) Requirement already satisfied: nvidia-nccl-cu12==2.18.1 in /usr/local/lib/python3.10/dist-packages (from torch->stable-ts) (2.18.1) Requirement already satisfied: nvidia-nvtx-cu12==12.1.105 in /usr/local/lib/python3.10/dist-packages (from torch->stable-ts) (12.1.105) Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.1.105 in /usr/local/lib/python3.10/dist-packages (from torch->stable-ts) (12.1.105) Requirement already satisfied: nvidia-cudnn-cu12==8.9.2.26 in /usr/local/lib/python3.10/dist-packages (from torch->stable-ts) (8.9.2.26) Requirement already satisfied: nvidia-cuda-runtime-cu12==12.1.105 in /usr/local/lib/python3.10/dist-packages (from torch->stable-ts) (12.1.105) Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch->stable-ts) (1.12) Requirement already satisfied: nvidia-cusolver-cu12==11.4.5.107 in /usr/local/lib/python3.10/dist-packages (from torch->stable-ts) (11.4.5.107) Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch->stable-ts) (3.1.3) Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch->stable-ts) (4.9.0) Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch->stable-ts) (3.2.1) Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch->stable-ts) (3.13.1) Requirement already satisfied: nvidia-cufft-cu12==11.0.2.54 in /usr/local/lib/python3.10/dist-packages (from torch->stable-ts) (11.0.2.54) Requirement already satisfied: nvidia-nvjitlink-cu12 in /usr/local/lib/python3.10/dist-packages (from nvidia-cusolver-cu12==11.4.5.107->torch->stable-ts) (12.3.101) Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch->stable-ts) (2.1.4) Requirement already satisfied: llvmlite<0.42,>=0.41.0dev0 in /usr/local/lib/python3.10/dist-packages (from numba->openai-whisper==20231117->stable-ts) (0.41.1) Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch->stable-ts) (1.3.0) Requirement already satisfied: regex>=2022.1.18 in /usr/local/lib/python3.10/dist-packages (from tiktoken->openai-whisper==20231117->stable-ts) (2023.12.25) Requirement already satisfied: requests>=2.26.0 in /usr/local/lib/python3.10/dist-packages (from tiktoken->openai-whisper==20231117->stable-ts) (2.31.0) Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.26.0->tiktoken->openai-whisper==20231117->stable-ts) (2.1.0) Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests>=2.26.0->tiktoken->openai-whisper==20231117->stable-ts) (3.6) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests>=2.26.0->tiktoken->openai-whisper==20231117->stable-ts) (2023.11.17) Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.26.0->tiktoken->openai-whisper==20231117->stable-ts) (3.3.2) WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv stable-ts has been successfully installed. Installing fastapi... Requirement already satisfied: fastapi in /usr/local/lib/python3.10/dist-packages (0.109.0) Requirement already satisfied: pydantic!=1.8,!=1.8.1,!=2.0.0,!=2.0.1,!=2.1.0,<3.0.0,>=1.7.4 in /usr/local/lib/python3.10/dist-packages (from fastapi) (2.5.3) Requirement already satisfied: typing-extensions>=4.8.0 in /usr/local/lib/python3.10/dist-packages (from fastapi) (4.9.0) Requirement already satisfied: starlette<0.36.0,>=0.35.0 in /usr/local/lib/python3.10/dist-packages (from fastapi) (0.35.1) Requirement already satisfied: pydantic-core==2.14.6 in /usr/local/lib/python3.10/dist-packages (from pydantic!=1.8,!=1.8.1,!=2.0.0,!=2.0.1,!=2.1.0,<3.0.0,>=1.7.4->fastapi) (2.14.6) Requirement already satisfied: annotated-types>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from pydantic!=1.8,!=1.8.1,!=2.0.0,!=2.0.1,!=2.1.0,<3.0.0,>=1.7.4->fastapi) (0.6.0) Requirement already satisfied: anyio<5,>=3.4.0 in /usr/local/lib/python3.10/dist-packages (from starlette<0.36.0,>=0.35.0->fastapi) (4.2.0) Requirement already satisfied: exceptiongroup>=1.0.2 in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.4.0->starlette<0.36.0,>=0.35.0->fastapi) (1.2.0) Requirement already satisfied: sniffio>=1.1 in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.4.0->starlette<0.36.0,>=0.35.0->fastapi) (1.3.0) Requirement already satisfied: idna>=2.8 in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.4.0->starlette<0.36.0,>=0.35.0->fastapi) (3.6) WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv fastapi has been successfully installed. Installing requests... Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (2.31.0) Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests) (3.3.2) Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests) (3.6) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests) (2023.11.17) Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests) (2.1.0) WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv requests has been successfully installed. Installing faster-whisper... Requirement already satisfied: faster-whisper in /usr/local/lib/python3.10/dist-packages (0.10.0) Requirement already satisfied: ctranslate2<4,>=3.22 in /usr/local/lib/python3.10/dist-packages (from faster-whisper) (3.24.0) Requirement already satisfied: tokenizers<0.16,>=0.13 in /usr/local/lib/python3.10/dist-packages (from faster-whisper) (0.15.1) Requirement already satisfied: huggingface-hub>=0.13 in /usr/local/lib/python3.10/dist-packages (from faster-whisper) (0.20.3) Requirement already satisfied: av==10.* in /usr/local/lib/python3.10/dist-packages (from faster-whisper) (10.0.0) Requirement already satisfied: onnxruntime<2,>=1.14 in /usr/local/lib/python3.10/dist-packages (from faster-whisper) (1.16.3) Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from ctranslate2<4,>=3.22->faster-whisper) (1.26.3) Requirement already satisfied: setuptools in /usr/lib/python3/dist-packages (from ctranslate2<4,>=3.22->faster-whisper) (59.6.0) Requirement already satisfied: pyyaml<7,>=5.3 in /usr/local/lib/python3.10/dist-packages (from ctranslate2<4,>=3.22->faster-whisper) (6.0.1) Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.13->faster-whisper) (4.9.0) Requirement already satisfied: tqdm>=4.42.1 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.13->faster-whisper) (4.66.1) Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.13->faster-whisper) (2023.12.2) Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.13->faster-whisper) (3.13.1) Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.13->faster-whisper) (2.31.0) Requirement already satisfied: packaging>=20.9 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.13->faster-whisper) (23.2) Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from onnxruntime<2,>=1.14->faster-whisper) (1.12) Requirement already satisfied: protobuf in /usr/local/lib/python3.10/dist-packages (from onnxruntime<2,>=1.14->faster-whisper) (4.25.2) Requirement already satisfied: flatbuffers in /usr/local/lib/python3.10/dist-packages (from onnxruntime<2,>=1.14->faster-whisper) (23.5.26) Requirement already satisfied: coloredlogs in /usr/local/lib/python3.10/dist-packages (from onnxruntime<2,>=1.14->faster-whisper) (15.0.1) Requirement already satisfied: humanfriendly>=9.1 in /usr/local/lib/python3.10/dist-packages (from coloredlogs->onnxruntime<2,>=1.14->faster-whisper) (10.0) Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub>=0.13->faster-whisper) (2.1.0) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub>=0.13->faster-whisper) (2023.11.17) Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub>=0.13->faster-whisper) (3.3.2) Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub>=0.13->faster-whisper) (3.6) Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->onnxruntime<2,>=1.14->faster-whisper) (1.3.0) WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv faster-whisper has been successfully installed. Installing uvicorn... Requirement already satisfied: uvicorn in /usr/local/lib/python3.10/dist-packages (0.27.0) Requirement already satisfied: typing-extensions>=4.0 in /usr/local/lib/python3.10/dist-packages (from uvicorn) (4.9.0) Requirement already satisfied: h11>=0.8 in /usr/local/lib/python3.10/dist-packages (from uvicorn) (0.14.0) Requirement already satisfied: click>=7.0 in /usr/local/lib/python3.10/dist-packages (from uvicorn) (8.1.7) WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv uvicorn has been successfully installed. Installing python-multipart... Requirement already satisfied: python-multipart in /usr/local/lib/python3.10/dist-packages (0.0.6) WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv python-multipart has been successfully installed. Installing whisper... Requirement already satisfied: whisper in /usr/local/lib/python3.10/dist-packages (1.1.10) Requirement already satisfied: six in /usr/local/lib/python3.10/dist-packages (from whisper) (1.16.0) WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv whisper has been successfully installed. Traceback (most recent call last): File "/subgen/./subgen.py", line 43, in import ffmpeg ModuleNotFoundError: No module named 'ffmpeg'

[McCloudS]

I misremembered. ffmpeg is installed in the docker image @ https://github.com/McCloudS/subgen/blob/f02cb3d8709869245c6398c7aeed8b2db26f4809/Dockerfile#L5C1-L5C68. How old is your build? Did you try a whole new pull? Did you use the volume mount "/usr/local/lib/python3.10/dist-packages"? You could remove it if so.

The ffmpeg python module is installed from faster-whisper inside the av module. Which shows installed correctly in your log : "Requirement already satisfied: av==10.* in /usr/local/lib/python3.10/dist-packages (from faster-whisper) (10.0.0)"

[Texugooftomorrow2]

tried with a new pull and after clearing docker build cache same results, using the default cpu version of the image, I have not put the volume mount "/usr/local/lib/python3.10/dist-packages" in the compose.

Currently testing with the volume to see if it works. Edit: same error

[2600box]

Hello, I noticed the same error just now. All fresh, never tried this before. I added ffmpeg-python to subgen.py and was able to pass the error.

I then ran into a GPU related error, but that is probably unrelated... CUDA failed with error CUDA driver version is insufficient for CUDA runtime version

[2600box]

Apologies if you were working on more changes, but I saw the latest commits and gave it a spin and found a different error:

[+] Running 2/2
 ✔ Network subgen_default  Created                                                                                 0.0s
 ✔ Container subgen        Created                                                                                18.1s
Attaching to subgen
subgen  |
subgen  | ==========
subgen  | == CUDA ==
subgen  | ==========
subgen  |
subgen  | CUDA Version 11.8.0
subgen  |
subgen  | Container image Copyright (c) 2016-2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
subgen  |
subgen  | This container image and its contents are governed by the NVIDIA Deep Learning Container License.
subgen  | By pulling and using the container, you accept the terms and conditions of this license:
subgen  | https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license
subgen  |
subgen  | A copy of this license is made available in this container at /NGC-DL-CONTAINER-LICENSE for your convenience.
subgen  |
subgen  |   File "/subgen/./subgen.py", line 21
subgen  |     import ffmpeg-python
subgen  |                  ^
subgen  | SyntaxError: invalid syntax
subgen exited with code 1

Explicitly adding python-ffmpeg to the Dockerfile list of pip packages to install makes everything run fine. cuda seems to be working too.

You said whisper should install it, as it is a dependency, but is there anything wrong with specifying it directly?

Thanks!

[McCloudS]

Thanks, I'll give it a shot. I hadn't pulled it and checked the change yet. Does subgen.py need import ffmpeg or ffmpeg-python?

[McCloudS]

New image appears to fix this. (Just pull mccloud/subgen) Image is larger, but easier to maintain. Your model of choice will be downloaded every container start unless you have a subgen volume mounted. I'm also trying a volume mount in the dockerfile that should fix persistence if you don't have a mount.

Let me know!

[lyfenwebos]

Confirm, I do not get that issues anymore

[McCloudS]

Thanks for the feedback! Persistent volume is working, so it won't re-download the models if you don't have one mounted.

[Texugooftomorrow2]

Working again, thanks!