search

mdhiggins / sickbeard_mp4_automator

Automatically convert video files to a standardized format with metadata tagging to create a beautiful and uniform media library
MIT License
1.52k stars 202 forks source link

Docker vaapi build: No such file or directory: '/usr/local/bin/ffprobe #1717

Closed dfeng777 closed 3 months ago

dfeng777 commented 3 months ago

Describe the bug After building an image using the vaapi build option in the docker compose file, neither ffmpeg or ffprobe can be run from within the docker shell. Running a manual.py command also errors out with the message in the title ("No such file or directory: '/usr/local/bin/ffprobe).

Command or context you are trying to run /usr/local/sma/venv/bin/python3 /usr/local/sma/manual.py -c /usr/local/sma/config/test.autoProcess.ini -i /data/Media/temp/InputFile.mkv

autoProcess.ini settings

[Converter]
ffmpeg = ffmpeg
ffprobe = ffprobe
threads = 0
hwaccels = vaapi
hwaccel-decoders = h264_vaapi, h264_cuvid, mjpeg_cuvid, mpeg1_cuvid, mpeg2_cuvid, mpeg4_cuvid, vc1_cuvid, hevc_qsv, h264_qsv, hevc_vaapi
hwdevices = vaapi:/dev/dri/renderD128
hwaccel-output-format = vaapi:vaapi
output-directory = 
output-directory-space-ratio = 0.0
output-format = mp4
output-extension = mp4
temp-extension = 
minimum-size = 0
ignored-extensions = nfo, ds_store
copy-to = 
move-to = 
delete-original = True
process-same-extensions = False
bypass-if-copying-all = False
force-convert = False
post-process = False
wait-post-process = False
detailed-progress = False
opts-separator = ,
preopts = 
postopts = 
regex-directory-replace = [^\w\-_\. ]

[Permissions]
chmod = 0664
uid = -1
gid = -1

[Metadata]
relocate-moov = True
full-path-guess = True
tag = True
tag-language = eng
download-artwork = poster
sanitize-disposition = 
strip-metadata = False
keep-titles = False

[Video]
codec = h264vaapi, h264, x264, h.264
max-bitrate = 0
bitrate-ratio = 
crf = -1
crf-profiles = 
preset = 
codec-parameters = 
dynamic-parameters = False
max-width = 0
profile = 
max-level = 0.0
pix-fmt = 
prioritize-source-pix-fmt = True
filter = 
force-filter = False

[HDR]
codec = 
pix-fmt = 
space = bt2020nc
transfer = smpte2084
primaries = bt2020
preset = 
codec-parameters = 
filter = 
force-filter = False
profile = 

[Audio]
codec = aac
languages = eng,chi,zho,jp,jpn,twi
default-language = eng
include-original-language = True
first-stream-of-language = False
channel-bitrate = 256
variable-bitrate = 0
max-bitrate = 0
max-channels = 0
filter = 
profile = 
force-filter = False
sample-rates = 
sample-format = 
copy-original = False
aac-adtstoasc = False
ignored-dispositions = 
force-default = False
unique-dispositions = False
stream-codec-combinations = 

[Audio.Sorting]
sorting = language, channels.d, map, d.comment
default-sorting = channels.d, map, d.comment
codecs = 

[Universal Audio]
codec = aac
channel-bitrate = 128
variable-bitrate = 0
first-stream-only = False
filter = 
profile = 
force-filter = False

[Audio.ChannelFilters]

[Subtitle]
codec = mov_text
codec-image-based = 
languages = eng,zho,cht
default-language = eng
force-default = False
include-original-language = False
first-stream-of-language = False
encoding = 
burn-subtitles = False
burn-dispositions = 
embed-subs = True
embed-image-subs = False
embed-only-internal-subs = False
filename-dispositions = forced
ignore-embedded-subs = False
ignored-dispositions = 
unique-dispositions = False
attachment-codec = 
remove-bitstream-subs = False

[Subtitle.Sorting]
sorting = language, d.comment, d.default.d, d.forced.d
codecs = 
burn-sorting = language, d.comment, d.default.d, d.forced.d

[Subtitle.CleanIt]
enabled = False
config-path = 
tags = 

[Subtitle.FFSubsync]
enabled = False

[Subtitle.Subliminal]
download-subs = False
download-forced-subs = False
include-hearing-impaired-subs = False
providers = 

[Subtitle.Subliminal.Auth]
opensubtitles = [redacted]
tvsubtitles = 

[Sonarr]
host = 127.0.0.1
port = [redacted]
apikey = [redacted]
ssl = False
webroot = /sonarr
force-rename = False
rescan = True
in-progress-check = True
block-reprocess = False

[Radarr]
host = 127.0.0.1
port = [redacted]
apikey = [redacted]
ssl = False
webroot = /radarr
force-rename = False
rescan = True
in-progress-check = True
block-reprocess = False

[Sickbeard]
host = localhost
port = 8081
ssl = False
apikey = 
webroot = 
username = 
password = 

[Sickrage]
host = localhost
port = 8081
ssl = False
apikey = 
webroot = 
username = 
password = 

[SABNZBD]
convert = True
sickbeard-category = sickbeard
sickrage-category = sickrage
sonarr-category = sonarr
radarr-category = radarr
bypass-category = bypass
output-directory = 
path-mapping = 

[Deluge]
sickbeard-label = sickbeard
sickrage-label = sickrage
sonarr-label = sonarr
radarr-label = radarr
bypass-label = bypass
convert = True
host = localhost
port = 58846
username = 
password = 
output-directory = 
remove = False
path-mapping = 

[qBittorrent]
sickbeard-label = sickbeard
sickrage-label = sickrage
sonarr-label = Sonarr
radarr-label = Radarr
bypass-label = bypass
convert = True
action-before = 
action-after = 
host = [redacted]
port = [redacted]
ssl = False
username = [redacted]
password = [redacted]
output-directory = 
path-mapping = 

[uTorrent]
sickbeard-label = sickbeard
sickrage-label = sickrage
sonarr-label = sonarr
radarr-label = radarr
bypass-label = bypass
convert = True
webui = False
action-before = 
action-after = 
host = localhost
ssl = False
port = 8080
username = 
password = 
output-directory = 
path-mapping = 

[Plex]
username = 
password = 
servername = 
host = [redacted]
port = 32400
refresh = True
token = [redacted]
ssl = True
ignore-certs = False
path-mapping =

Log files

2024-07-08 16:40:02 - MANUAL - INFO - Manual processor started.
2024-07-08 16:40:02 - MANUAL - INFO - Python 64-bit 3.12.3 (main, Apr 18 2024, 07:52:31) [GCC 13.2.1 20240309].
2024-07-08 16:40:02 - MANUAL - INFO - Guessit version: 3.8.0.
2024-07-08 16:40:02 - MANUAL - INFO - /usr/local/sma/venv/bin/python3
2024-07-08 16:40:02 - MANUAL - INFO - Loading config file /usr/local/sma/config/test.autoProcess.ini.
2024-07-08 16:40:02 - MANUAL - ERROR - isValidSource unexpectedly threw an exception, returning None.
Traceback (most recent call last):
  File "/usr/local/sma/resources/mediaprocessor.py", line 359, in isValidSource
    info = self.converter.probe(inputfile)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/sma/converter/__init__.py", line 350, in probe
    return self.ffmpeg.probe(fname, posters_as_video)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/sma/converter/ffmpeg.py", line 621, in probe
    stdout_data = self._get_stdout([
                  ^^^^^^^^^^^^^^^^^^
  File "/usr/local/sma/converter/ffmpeg.py", line 571, in _get_stdout
    p = self._spawn(cmds)
        ^^^^^^^^^^^^^^^^^
  File "/usr/local/sma/converter/ffmpeg.py", line 564, in _spawn
    return Popen(cmds, shell=False, stdin=PIPE, stdout=PIPE, stderr=PIPE,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/subprocess.py", line 1026, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.12/subprocess.py", line 1955, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/bin/ffprobe'
2024-07-08 16:40:02 - MANUAL - INFO - File /data/Media/temp/InputFile.mkv is not in a valid format

FFMpeg headers

root@[redacted]:/usr/local/sma/config# ffmpeg -hwaccel
/bin/ash: ffmpeg: not found

System Information

services:
  sonarr:
#    image: mdhiggins/sonarr-sma:latest
    build:
      context: https://github.com/mdhiggins/sonarr-sma.git#build
      args:
        - ffmpeg_tag=7.0-vaapi
    container_name: sonarr
    environment:
      - PUID=[redacted] #CHANGE_TO_YOUR_UID
      - PGID=[redacted] #CHANGE_TO_YOUR_GID
      - TZ=America/Los_Angeles #CHANGE_TO_YOUR_TZ
      - UMASK=022
    volumes:
      - /volume1/docker/sonarr:/config
      - /volume1/docker/sma:/usr/local/sma/config
      - /volume1/Data:/data
    devices:
      - /dev/dri:/dev/dri
    ports:
      - [redacted]:[redacted]/tcp
    network_mode: synobridge
    security_opt:
      - no-new-privileges:true
    restart: always

  radarr:
#    image: mdhiggins/radarr-sma:latest
    build:
      context: https://github.com/mdhiggins/radarr-sma.git#build
      args:
        - ffmpeg_tag=7.0-vaapi
    container_name: radarr
    environment:
      - PUID=[redacted] #CHANGE_TO_YOUR_UID
      - PGID=[redacted]#CHANGE_TO_YOUR_GID
      - TZ=America/Los_Angeles #CHANGE_TO_YOUR_TZ
      - UMASK=022
    volumes:
      - /volume1/docker/radarr:/config
      - /volume1/docker/sma:/usr/local/sma/config
      - /volume1/Data:/data
    devices:
      - /dev/dri:/dev/dri
    ports:
      - [redacted]:[redacted]/tcp
    network_mode: synobridge
    security_opt:
      - no-new-privileges:true
    restart: always

Expected behavior Expected manual.py script to find and execute ffprobe and ffmpeg to start processing file.

Additional context Was able to successfully use the mdhiggins/radarr-sma:latest image to run ffmpeg from the container shell and to run a manual.py job (just without hardware acceleration).

Issues posted without any logs or autoProcess settings will be closed

mdhiggins commented 3 months ago

Looks like you're building a container using the latest sonarr container which has been rebased to alpine but then using an ffmpeg build from a non alpine based container which will not work

dfeng777 commented 3 months ago

Thanks for the response. So how would I construct the compose file to make them consistent? I had commented out the image: mdhiggins/sonarr-sma:latest line. Do I need to different ffmpeg build tag and/or specify a sonarr build tag?

mdhiggins commented 3 months ago

Depend on what your specific end goals are

If its just vaapi encoding that you're after, the ffmpeg build included in the alpine repo supports vaapi, so just enable SMA_USE_REPO and SMA_HWACCEL by setting them to true. Doing this means you no longer need to build your own container and can just use the images. If you were building ffmpeg for another reason then it'll depend, but alpine ffmpeg build containers are somewhat limited

dfeng777 commented 3 months ago

Thanks again. As you suggested, I switched back to the mhiggins/sonarr-sma image with the SMA_USE_REPO and SMA_HWACCEL env variables enabled and the vaapi-enabled version of ffmpeg is working. My goals are simply to encode h265/hevc/mkv media to h264/avc/mp4/aac for maximum compatibility.

After fiddling around with autoProcess.ini options for a while and after reading various support threads, it seems like the only way I could get a 10-bit h265 file to encode to 8-bit h264 was to add the following line to postopts:

-vf scale_vaapi=w=1920:h=1080:format=nv12

This was as described in this post:

https://github.com/mdhiggins/sickbeard_mp4_automator/issues/1666#issuecomment-1673404167

I'm not sure what this option does and if it's still the best way to get 10-bit encoding working, but would appreciate the latest direction. Let me know if a new issue/thread is appropriate.

mdhiggins commented 3 months ago

That's probably not the best way to do it, the vaapi encoder will add the scalar automatically if you are decreasing your size

What you're going to want to do is set your pix_fmt to something 8 bit instead but which format is up to you