Docker mod

[abhishekjain14]

I've been considering using this with nzbget instead of radarr/sonarr to simplify my workflow and have the conversions happen on a local disk instead of a NAS. I don't see any up to date containers out there, but came across this little tidbit: https://blog.linuxserver.io/2019/09/14/customizing-our-containers/

Any reason SMA couldn't work as a docker mod, allowing us the capability to add it into any container that could use it? If not, any interest in doing so, @mdhiggins? If not, I'd be willing to give it a try.

[mdhiggins]

Might want to just retry that. Seems like maybe a bad download

[mdhiggins]

@Protektor-Desura

Docker updated their API and broke docker-mods. Linuxserver is working on a fix.

[Protektor-Desura]

That sucks that the API changed without any warning or at least not much. Thanks for the heads up.

[mdhiggins]

https://github.com/linuxserver/docker-mods/issues/160

Seems like a fix is in the works and should be making its way out

[VampiricAlien]

@mdhiggins Are you able to guide me on the right path?

I think I finely got it install but not working?

Saw this yesterday

[90-sma-config] Installing additional deluge/qbittorrent requirements

[90-sma-config] Set FFMPEG URL to https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz

[90-sma-config] Downloading and installing FFMpeg from https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz

Today:

converter.ffmpeg.FFMpegError: ffmpeg binary not found: /usr/local/bin/ffmpeg
raise FFMpegError("ffmpeg binary not found: " + self.ffmpeg_path)   
"/usr/local/sma/converter/ffmpeg.py", line 489, in __init__ 
self.ffmpeg = FFMpeg(ffmpeg_path=ffmpeg_path,   
 "/usr/local/sma/converter/__init__.py", line 26, in __init__   
self.converter = Converter(settings.ffmpeg, settings.ffprobe)   
"/usr/local/sma/resources/mediaprocessor.py", line 43, in __init__  
mp = MediaProcessor(settings)   
File "/usr/local/sma/postSonarr.py", line 201, in <module>  

I get that FFMPEG isn't install in that location, I have put FFMPEG in /sma/bin/ffmpeg and set the new path in autoProcess.ini but it gets overwritten Inside postSonarr.sh

#!/usr/bin/env bash

# env check
if [[ -z "${SMA_PATH}" ]]; then
    export SMA_PATH="/usr/local/sma"
fi

$SMA_PATH/venv/bin/python3 $SMA_PATH/postSonarr.py

SMA_HWACCEL true SMA_PATH /usr/local/sma SMA_STRIP_COMPONENTS 0 SMA_UPDATE false SMA_USE_REPO false

If I didn't have everything setup, I would install your sonarr app with everything install. Also Sonarr can't access the path after restart unless things are install?

One last question, is this mod suppose to be an image or container?

[mdhiggins]

0f461997c1ce1d214d76fb3a0df33a952ccaaf6d

Pushed an update that just sets ffmpeg/ffprobe generically instead of a specific path, if you want to override your ffmpeg/ffprobe binaries just put them somewhere in the path so that they are recognized by which ffmpeg and which ffprobe

echo $PATH to see

That should help

[VampiricAlien]

/usr/local/sma/config/postSonarr.sh: line 8: /usr/local/sma/venv/bin/python3: No such file or directory

Is it possible to have all custom paths? /usr/local/sma/python/ and/or /usr/local/sma/ffmpeg?

shortly after restart:

[90-sma-config] Updating SMA from github master
[90-sma-config] Creating python virtual environment and installing requirements
 DEPRECATION: qtfaststart is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559

  DEPRECATION: enzyme is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559

  DEPRECATION: pysrt is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559
[90-sma-config] Installing additional deluge/qbittorrent requirements
[90-sma-config] Set FFMPEG URL to https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz
[90-sma-config] Downloading and installing FFMpeg from https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz

I have SMA update set to true but also

[Error] postSonarr.sh: Traceback (most recent call last): 
[Error] postSonarr.sh:   File "/usr/local/sma/postSonarr.py", line 201, in <module> 
[Error] postSonarr.sh:     mp = MediaProcessor(settings) 
[Error] postSonarr.sh:   File "/usr/local/sma/resources/mediaprocessor.py", line 43, in __init__ 
[Error] postSonarr.sh:     self.converter = Converter(settings.ffmpeg, settings.ffprobe) 
[Error] postSonarr.sh:   File "/usr/local/sma/converter/__init__.py", line 26, in __init__ 
[Error] postSonarr.sh:     self.ffmpeg = FFMpeg(ffmpeg_path=ffmpeg_path, 
[Error] postSonarr.sh:   File "/usr/local/sma/converter/ffmpeg.py", line 489, in __init__ 
[Error] postSonarr.sh:     raise FFMpegError("ffmpeg binary not found: " + self.ffmpeg_path) 
[Error] postSonarr.sh: converter.ffmpeg.FFMpegError: ffmpeg binary not found: /usr/local/sma/bin/ffmpeg/ffmpeg 

The binaries are inside the folder with access to all but which ffmpeg say it is locaated in /usr/local/bin/ffmpeg after it goes to download, I have SMA_HWACCEL set to true. When everthing is working ffmpeg says The encoder you have chosen h265vaapi (hevc_vaapi) is not listed as supported in your FFMPEG build

Sorry about the read.

[mdhiggins]

You'll need to mount ffmpeg/ffprobe somewhere within your system PATH (well, the container system path) that will not only prevent the startup script from downloading it but also prevent the path issue

[mdhiggins]

Went ahead and added two variables if for some reason you cannot use the container path

SMA_FFMPEG_PATH and SMA_FFPROBE_PATH

Setting should block downloading and I updated the script to use these paths if present when settings autoProcess.ini

[VampiricAlien]

latest docker pull mdhiggins/sma-mod:latest Last pushed 23 minutes ago by mdhiggins

Pulled and paths set.... still installing after setup Docker paths and config set

[Converter]
ffmpeg = /usr/local/sma/ffmpeg
ffprobe = /usr/local/sma/ffprobe

Is there an easier way to get this working without reinstalling containers?

[mdhiggins]

Sadly no. For small changes to the main script SMA_UPDATE is enough but this needed more. Looks like that worked though the paths were left alone. Is that working for you?

[VampiricAlien]

after Running for 24 minutes in goes to install and setup which ffmpeg /usr/local/bin/ffmpeg whereis ffmpeg ffmpeg: /usr/local/bin/ffmpeg

I renamed the old folder and let it setup everything but all it did was create autoProcess.ini and init.py but I have to wait until sma sets up python and ffmpeg but I am just going to rename the other folder to sma

No it hasn't work.

[VampiricAlien]

I've set the FFMPEG link to /usr/local/sma/bin/ffmpeg.zip in hope that they are installed from the zip file

[VampiricAlien]

Now I get

E: Unable to correct problems, you have held broken packages.

[90-sma-config] Creating python virtual environment and installing requirements

  WARNING: The scripts pip, pip3 and pip3.10 are installed in '/usr/local/sma/venv/bin' which is not on PATH.

  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.

  DEPRECATION: qtfaststart is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559

  DEPRECATION: enzyme is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559

  WARNING: The script pbr is installed in '/usr/local/sma/venv/bin' which is not on PATH.

  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.

  WARNING: The scripts mid3cp, mid3iconv, mid3v2, moggsplit, mutagen-inspect and mutagen-pony are installed in '/usr/local/sma/venv/bin' which is not on PATH.

  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.

  WARNING: The script normalizer is installed in '/usr/local/sma/venv/bin' which is not on PATH.

  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.

  WARNING: The script chardetect is installed in '/usr/local/sma/venv/bin' which is not on PATH.

  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.

  DEPRECATION: pysrt is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559

  WARNING: The script jsonschema is installed in '/usr/local/sma/venv/bin' which is not on PATH.

  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.

  WARNING: The script guessit is installed in '/usr/local/sma/venv/bin' which is not on PATH.

  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.

  WARNING: The script cleanit is installed in '/usr/local/sma/venv/bin' which is not on PATH.

  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.

  WARNING: The script subliminal is installed in '/usr/local/sma/venv/bin' which is not on PATH.

  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.

[90-sma-config] Installing additional deluge/qbittorrent requirements

[90-sma-config] Downloading and installing FFMpeg from /usr/local/sma/bin/ffmpeg.tar.xz

xz: (stdin): File format not recognized

tar: Child returned status 1

tar: Error is not recoverable: exiting now

chown: cannot access '/usr/local/bin/ffmpeg': No such file or directory

chown: cannot access '/usr/local/bin/ffprobe': No such file or directory

chmod: cannot access '/usr/local/bin/ffmpeg': No such file or directory

chmod: cannot access '/usr/local/bin/ffprobe': No such file or directory

[ls.io-init] done.

Update: Changed to zip file and path is now /usr/local/sma 2nd update pulled s6v2 from docker - didn't work

[mdhiggins]

Forgot you were on s6v2, I haven't brought those changes to s6v2 so I just merged them over from master

Can you share your full docker config? The more logs you post the less sense the setup is making

[VampiricAlien]

I meant to say is I pulled s6V2 because the changes to the path on the main wasn't working, I am using mdhiggins/sma-mod:latest (went back to this one) and get errors about the path as post above

[90-sma-config] Updating SMA from github master
[90-sma-config] Creating python virtual environment and installing requirements
  WARNING: The scripts pip, pip3 and pip3.10 are installed in '/usr/local/sma/venv/bin' which is not on PATH.
 Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.

Before the new changes where made I think I just need to change FFmpeg path?

[VampiricAlien]

Sonarr tried to use the script (forgot the setup didn't put ffmpeg in the folder)

[Error] postSonarr.sh: Traceback (most recent call last): 
[Error] postSonarr.sh:   File "/usr/local/sma//postSonarr.py", line 201, in <module> 
[Error] postSonarr.sh:     mp = MediaProcessor(settings) 
[Error] postSonarr.sh:   File "/usr/local/sma/resources/mediaprocessor.py", line 43, in __init__ 
[Error] postSonarr.sh:     self.converter = Converter(settings.ffmpeg, settings.ffprobe) 
[Error] postSonarr.sh:   File "/usr/local/sma/converter/__init__.py", line 26, in __init__ 
[Error] postSonarr.sh:     self.ffmpeg = FFMpeg(ffmpeg_path=ffmpeg_path, 
[Error] postSonarr.sh:   File "/usr/local/sma/converter/ffmpeg.py", line 489, in __init__ 
[Error] postSonarr.sh:     raise FFMpegError("ffmpeg binary not found: " + self.ffmpeg_path) 
[Error] postSonarr.sh: converter.ffmpeg.FFMpegError: ffmpeg binary not found: /usr/local/sma/ffmpeg `

Binaries are back in the folder however when goes to run the script again it won't work due to the path not being set like it was before. sadly I delete the image from a few days ago and using the one from 9 hours ago.

While I wait for the new changes to be reverted back? can the install of python happen within the first new minutes of startup and not after 10 minutes if update is set to true in container config?

[mdhiggins]

Share your docker config

And ideally what content is in your volumes with regards to FFMPEG

[VampiricAlien]

container_name: sonarr
    devices:
      - CgroupPermissions: rwm
        PathInContainer: /dev/dri/renderD128
        PathOnHost: /dev/dri/renderD128
      - CgroupPermissions: rwm
        PathInContainer: /dev/dri/
        PathOnHost: /dev/dri/
      - CgroupPermissions: rwm
        PathInContainer: /dev/dri/card0
        PathOnHost: /dev/dri/card0
      - CgroupPermissions: rwm
        PathInContainer: /usr/lib/x86_64-linux-gnu/dri//195_drv_video.so
        PathOnHost: /usr/lib/x86_64-linux-gnu/dri//195_drv_video.so
      - CgroupPermissions: rwm
        PathInContainer: /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so
        PathOnHost: /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so
    entrypoint:
      - /init
    environment:
      - PUID=1000
      - PGID=1000
      - DOCKER_MODS=mdhiggins/sma-mod:latest
      - SMA_HWACCEL=true
      - SMA_PATH=/usr/local/sma/
      - SMA_STRIP_COMPONENTS=1
      - SMA_USE_REPO=false
      - SMA_UPDATE=true
      - LIBVA_DRIVER_NAME=iHD
      - LIBVA_DRIVERS_PATH=/usr/lib/x86_64-linux-gnu/dri/
      - PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
      - HOME=/root
      - LANGUAGE=en_US.UTF-8
      - LANG=en_US.UTF-8
      - TERM=xterm
      - S6_CMD_WAIT_FOR_SERVICES_MAXTIME=0
      - S6_VERBOSITY=1
      - S6_STAGE2_HOOK=/docker-mods
      - LSIO_FIRST_PARTY=true
      - XDG_CONFIG_HOME=/config/xdg
      - SONARR_BRANCH=main
      - SMA_FFMPEG_PATH=/usr/local/sma/ffmpeg
      - SMA_FFPROBE_PATH=/usr/local/sma/ffprobe
      - SMA_FFMPEG_URL=/usr/local/sma/bin/ffmpeg
    hostname: sonarr
    image: lscr.io/linuxserver/sonarr:latest

No FFMPEG mounted, do they need to be mounted?

[mdhiggins]

Alright so this is a bit confusing and I guess I'm not fully sure what your aim is here. I'm guessing just looking at this you're trying to set up intel based hardware acceleration? correct me if that's not the case

You're making a few fundamental mistakes here and also creating lots of work for yourself

• SMA_PATH only needs to be set if you're overriding it
• SMA_FFMPEG/FFPROBE_PATH should only be set if you're providing FFMPEG/FFPROBE binaries yourself via a mounted volume. Setting this blocks the normal downloading process under the assumption that you're providing them somewhere
• SMA_FFMPEG_URL needs to be a URL not a path, this is to replace the default default https://johnvansickle.com source of FFMPEG. This does nothing when the above SMA_FFMPEG/FFPROBE_PATH are set
• SMA_STRIP_COMPONENTS only needs to be modified if you're providing a different download URL that is packaged differently. This does nothing if SMA_FFMPEG/FFPROBE_PATH is set and won't help if SMA_FFMPEG_URL isn't a URL. The code this impacts is just one line shown below

  wget -q ${SMA_FFMPEG_URL} -O /tmp/ffmpeg.tar.xz
  tar --exclude='*.html' --exclude='*.txt' -xJf /tmp/ffmpeg.tar.xz -C /usr/local/bin --strip-components ${SMA_STRIP_COMPONENTS}

• You probably don't need to be passing all these drivers and environment variables for intel transcoding
• Not sure why you're modifying the S6 environment variables either but I'll leave that alone

The biggest issue here is that you're essentially telling the script you are going to provide FFMPEG and then not doing so, and you're also blocking the FFMPEG_URL with an invalid URL

I made the following test

  sonarrtest:
    image: lscr.io/linuxserver/sonarr:latest
    container_name: sonarrtest
    devices:
      - /dev/dri:/dev/dri
    environment:
      - PUID=${PUID}
      - PGID=${PGID}
      - DOCKER_MODS=mdhiggins/sma-mod:latest
      - SMA_USE_REPO=true
      - SMA_UPDATE=true
      - SMA_HWACCEL=true

And I am able to open

michael@plexrack:~$ dcp exec sonarrtest bash
root@abe0899e3fac:/# ffmpeg -hwaccels
ffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.2.0-19ubuntu1)
  configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Hardware acceleration methods:
vdpau
cuda
vaapi
qsv
drm
opencl

root@abe0899e3fac:/# /usr/local/sma/venv/bin/python3 /usr/local/sma/manual.py
Manual processor started.
Python 64-bit 3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0].
Guessit version: 3.7.1.
/usr/local/sma/venv/bin/python3
Loading config file /usr/local/sma/config/autoProcess.ini.
Enter path to file:
#:

root@abe0899e3fac:/# cd /dev/dri
root@abe0899e3fac:/dev/dri# ls -la
total 0
drwxr-xr-x 2 root root            80 Apr 11 10:58 .
drwxr-xr-x 6 root root           360 Apr 11 10:58 ..
crw-rw---- 1 root video     226,   0 Apr 11 10:58 card0
crw-rw---- 1 root videoj3e5 226, 128 Apr 11 10:58 renderD128

The repo included FFMPEG already has VAAPI and QSV included so this is a very easy way to get that via SMA_USE_REPO

I would then stop modifying autoProcess.ini and let the script auto configure the FFMPEG path

SMA_MOD also includes a script that sets the /dev/dri permissions so you don't need to do anything there except make sure you have appropriate permissions on your host machine

Obviously you'll need to include the mount points for your data/configuration files which I assume are omitted here for brevity

[VampiricAlien]

I appreciate the effort in your last post and have made the changes I hope fixes this.

I've removed everything and replaced with

- SMA_USE_REPO=true
- SMA_UPDATE=true
- SMA_HWACCEL=true

Mounted

 /usr/local/bin/ffmpeg:/usr/local/bin/ffmpeg
  /usr/local/bin/ffprobe:/usr/local/bin/ffprobe

Kept only /dev/dri/ driver path and tested with ls -la

How long does it take to setup python on each restart of container? 25 minutes and counting.

[mdhiggins]

You aren't supposed to mount FFMPEG, that was the whole point of using the repo and not needing to include all the extra stuff I outlined above

The only volumes you need are your media and config folders. Something like this. Let the container handle FFMPEG

    volumes:
      - /opt/appdata/sonarr:/config
      - /opt/appdata/sma:/usr/local/sma/config
      - /mnt/storage/tv:/tv
      - /mnt/storage/downloads:/downloads

[VampiricAlien]

You aren't supposed to mount FFMPEG, that was the whole point of using the repo and not needing to include all the extra stuff I outlined above

The only volumes you need are your media and config folders. Something like this. Let the container handle FFMPEG

    volumes:
      - /opt/appdata/sonarr:/config
      - /opt/appdata/sma:/usr/local/sma/config
      - /mnt/storage/tv:/tv
      - /mnt/storage/downloads:/downloads

Sorry for confusing mix up,. They have been removed.

root@sonarr:/# /usr/local/sma/manual.py
Traceback (most recent call last):
  File "/usr/local/sma/manual.py", line 5, in <module>
    import guessit
ModuleNotFoundError: No module named 'guessit'
root@sonarr:/# /usr/local/sma/postSonarr.sh
2023-04-12 01:24:09 - SonarrPostProcess - INFO - Sonarr extra script post processing started.
2023-04-12 01:24:09 - SonarrPostProcess - ERROR - Invalid event type None, script only works for On Download/On Import and On Upgrade.
root@sonarr:/# /usr/local/sma/postSonarr.py
Traceback (most recent call last):
  File "/usr/local/sma/postSonarr.py", line 4, in <module>
    import requests
ModuleNotFoundError: No module named 'requests'
root@sonarr:/# ^C
root@sonarr:/# 

fatal: detected dubious ownership in repository at '/usr/local/sma'
To add an exception for this directory, call:
git config --global --add safe.directory /usr/local/sma

After installing requirements.txt I was able to run the command manual.py, any idea why there isn't access to guessit and others?

[VampiricAlien]

Modules are located in /sma/tutorial-env/lib/python3.10/site-packages not sure why I had to re-download them in the container.

Did a test with manual.py

[AVHWDeviceContext @ 0x55d66c6f1e40] Failed to initialise VAAPI connection: -1 (unknown libva error).
Device creation failed: -5.
Failed to set value 'vaapi=sma:/dev/dri/renderD128' for option 'init_hw_device': Input/output error
Error parsing global options: Input/output error

root@sonarr:/# ls -al /dev/dri
total 0
drwxr-xr-x  2 root root            80 Apr 12 09:01 .
drwxr-xr-x 17 root root          4820 Apr 12 09:01 ..
crw-rw----  1 root video     226,   0 Apr 12 09:01 card0
crw-rw----  1 root videovt7q 226, 128 Apr 12 09:01 renderD128

I know this is a container error since I can and do use the script outside of docker. I've used enough of your time to try and fix this. I don't want to take up more of your time :)

[mdhiggins]

What do your permissions look like on the host container for /dev/dri?

[VampiricAlien]

total 0
drwxr-xr-x  2 root root            80 Apr 12 09:13 .
drwxr-xr-x 17 root root          4820 Apr 12 09:13 ..
crw-rw----  1 root video     226,   0 Apr 12 09:13 card0
crw-rw----  1 root videovt7q 226, 128 Apr 12 09:13 renderD128

OS level

total 0
drwxr-xr-x   3 root root        100 Feb 24 12:05 .
drwxr-xr-x  21 root root       5000 Apr 12 05:08 ..
drwxr-xr-x   2 root root         80 Mar 21 18:56 by-path
crw-rw----+  1 root video  226,   0 Mar 21 18:56 card0
crw-rw----+  1 root render 226, 128 Mar 21 18:56 renderD128

[mdhiggins]

Does your user (1000) belong to the video/render groups?

[VampiricAlien]

Does your user (1000) belong to the video/render groups?

Yes it does have access to both groups. I can use the script outside of docker.

[mdhiggins]

You're using your same docker user as your main user? Cause that would impact things and being able to use it outside the container does not mean the permissions carry over necessarily

Also if you could share the whole FFMPEG command I can take a look

[mdhiggins]

Also the full error output from FFMPEG might be helpful here too, can see if its throwing a dependency error of some kind

[VampiricAlien]

2023-04-12 08:34:06 - MANUAL - INFO - FFmpeg command:
2023-04-12 08:34:06 - MANUAL - INFO - ======================
2023-04-12 08:34:06 - MANUAL - INFO - /usr/bin/ffmpeg -fix_sub_duration -hwaccel vaapi -hwaccel_output_format vaapi -i "/storage/tv/file.mkv.original" -vcodec hevc_vaapi -map 0:0 -field_order progressive -metadata:s:v title=FHD -metadata:s:v handler_name=FHD -preset fast -profile:v main10 -level 5.2 -tag:v hvc1 -qp 0 -vaapi_device /dev/dri/renderD128 -vf "format=p010|vaapi,hwupload,scale_vaapi=format=p010" -c:a:0 copy -map 0:1 -metadata:s:a:0 title=Stereo -metadata:s:a:0 handler_name=Stereo -metadata:s:a:0 language=eng -disposition:a:0 +default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -c:s:0 copy -map 0:2 -metadata:s:s:0 title= -metadata:s:s:0 handler_name= -metadata:s:s:0 language=eng -disposition:s:0 +default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -f matroska -threads 0 -metadata:g encoding_tool=SMA -y "/storage/tv/file.mkv"
2023-04-12 08:34:06 - MANUAL - INFO - ======================
2023-04-12 08:34:06 - MANUAL - ERROR - Error converting file, FFMPEG error.
Traceback (most recent call last):
  File "/usr/local/sma/resources/mediaprocessor.py", line 2314, in convert
    for timecode, debug in conv:
  File "/usr/local/sma/converter/__init__.py", line 329, in convert
    for timecode, debug in self.ffmpeg.convert(outfile,
  File "/usr/local/sma/converter/ffmpeg.py", line 795, in convert
    raise FFMpegConvertError('Exited with code %d' % p.returncode, cmd,
converter.ffmpeg.FFMpegConvertError: <unprintable FFMpegConvertError object>
2023-04-12 08:34:06 - MANUAL - ERROR - /usr/bin/ffmpeg -fix_sub_duration -hwaccel vaapi -hwaccel_output_format vaapi -i /storage/tv/file.mkv.original -vcodec hevc_vaapi -map 0:0 -field_order progressive -metadata:s:v title=FHD -metadata:s:v handler_name=FHD -preset fast -profile:v main10 -level 5.2 -tag:v hvc1 -qp 0 -vaapi_device /dev/dri/renderD128 -vf format=p010|vaapi,hwupload,scale_vaapi=format=p010 -c:a:0 copy -map 0:1 -metadata:s:a:0 title=Stereo -metadata:s:a:0 handler_name=Stereo -metadata:s:a:0 language=eng -disposition:a:0 +default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -c:s:0 copy -map 0:2 -metadata:s:s:0 title= -metadata:s:s:0 handler_name= -metadata:s:s:0 language=eng -disposition:s:0 +default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -f matroska -threads 0 -metadata:g encoding_tool=SMA -y /storage/tv/file.mkv
2023-04-12 08:34:06 - MANUAL - ERROR - ffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.2.0-19ubuntu1)
  configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
[AVHWDeviceContext @ 0x55e38efb8f80] Failed to initialise VAAPI connection: -1 (unknown libva error).
Device creation failed: -5.
Failed to set value '/dev/dri/renderD128' for option 'vaapi_device': Input/output error
Error parsing global options: Input/output error

2023-04-12 08:34:06 - MANUAL - ERROR - There was an error processing file /storage/tv/file.mkv, no output data received

[mdhiggins]

Hm, can you check and see if intel-media-driver and libva packages are installed? Inside the container that is

[VampiricAlien]

Also the full error output from FFMPEG might be helpful here too, can see if its throwing a dependency error of some kind

I used the script at OS level after the files are put in place. I have the same user ID so the files are not locked and I can access them.

The following NEW packages will be installed:
  intel-media-va-driver-non-free libigdgmm12 intel-media-va-driver

[VampiricAlien]

libva info: VA-API version 1.14.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_14
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.14 (libva 2.12.0)
vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 22.3.1 ()

[mdhiggins]

Is that from inside the container?

[VampiricAlien]

Yes it is

Now I get

  Metadata:
    encoder         : libebml v1.4.0 + libmatroska v1.6.1
  Duration: 00:43:34.03, start: 0.000000, bitrate: 4960 kb/s
  Stream #0:0: Video: h264 (High), yuv420p(tv, bt709, progressive), 1920x1080, SAR 1:1 DAR 16:9, 23.98 fps, 23.98 tbr, 1k tbn, 47.95 tbc (default)
  Stream #0:1(eng): Audio: aac (HE-AAC), 48000 Hz, stereo, fltp (default)
  Stream #0:2(eng): Subtitle: subrip
    Metadata:
      title           : English
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> hevc (hevc_vaapi))
  Stream #0:1 -> #0:1 (copy)
  Stream #0:2 -> #0:2 (copy)
Press [q] to stop, [?] for help
[Parsed_scale_vaapi_2 @ 0x560ea59ac580] Failed to create processing pipeline config: 12 (the requested VAProfile is not supported).
[Parsed_scale_vaapi_2 @ 0x560ea59ac580] Failed to configure output pad on Parsed_scale_vaapi_2
Error reinitializing filters!
Failed to inject frame into filter network: Input/output error
Error while processing the decoded data for stream #0:0
Conversion failed!

[mdhiggins]

If you take the original command

/usr/bin/ffmpeg -fix_sub_duration -hwaccel vaapi -hwaccel_output_format vaapi -i "/storage/tv/file.mkv.original" -vcodec hevc_vaapi -map 0:0 -field_order progressive -metadata:s:v title=FHD -metadata:s:v handler_name=FHD -preset fast -profile:v main10 -level 5.2 -tag:v hvc1 -qp 0 -vaapi_device /dev/dri/renderD128 -vf "format=p010|vaapi,hwupload,scale_vaapi=format=p010" -c:a:0 copy -map 0:1 -metadata:s:a:0 title=Stereo -metadata:s:a:0 handler_name=Stereo -metadata:s:a:0 language=eng -disposition:a:0 +default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -c:s:0 copy -map 0:2 -metadata:s:s:0 title= -metadata:s:s:0 handler_name= -metadata:s:s:0 language=eng -disposition:s:0 +default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -f matroska -threads 0 -metadata:g encoding_tool=SMA -y "/storage/tv/file.mkv"

And remove the filter part and run it manually

/usr/bin/ffmpeg -fix_sub_duration -hwaccel vaapi -hwaccel_output_format vaapi -i "/storage/tv/file.mkv.original" -vcodec hevc_vaapi -map 0:0 -field_order progressive -metadata:s:v title=FHD -metadata:s:v handler_name=FHD -preset fast -profile:v main10 -level 5.2 -tag:v hvc1 -qp 0 -vaapi_device /dev/dri/renderD128 -c:a:0 copy -map 0:1 -metadata:s:a:0 title=Stereo -metadata:s:a:0 handler_name=Stereo -metadata:s:a:0 language=eng -disposition:a:0 +default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -c:s:0 copy -map 0:2 -metadata:s:s:0 title= -metadata:s:s:0 handler_name= -metadata:s:s:0 language=eng -disposition:s:0 +default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -f matroska -threads 0 -metadata:g encoding_tool=SMA -y "/storage/tv/file.mkv"

Does that change anything?

[mdhiggins]

Can also try installing just the intel-media-va-driver-non-free package without the free variant and see if that helps

[mdhiggins]

Actually I'm surprised you were able to install that before, I assume sonarr is on alpine these days and I don't think that has the nonfree package, or any of the packages you listed previously. Is sonarr not on alpine on your build?

[VampiricAlien]

Still get the same error.

I think it's ubuntu?

[mdhiggins]

Ah nevermind I'm on the develop branch which has switched over to alpine, guess the latest is still ubuntu for now

[mdhiggins]

I'm guessing this is some kind of dependency missing and all searching over the error usually points to people recommending installing the non-free variant. Not sure what else could be missing

[VampiricAlien]

Linux sonarr 5.19.0-32-generic #33~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Mon Jan 30 17:03:34 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

I have installed both divers and had to install from requirements.txt

[mdhiggins]

            libva-drm2 \
            libva2 \
            i965-va-driver \
            intel-media-va-driver \
            intel-media-va-driver-non-free \
            mesa-va-drivers

Those are the packages that are installed on ubuntu based systems using the SMA_HWACCEL option, might want to check and see if they are actually being installed and add any missing ones

[VampiricAlien]

libva-drm2 is already the newest version (2.14.0-1).
libva-drm2 set to manually installed.
libva2 is already the newest version (2.14.0-1).
libva2 set to manually installed.
intel-media-va-driver is already the newest version (22.3.1+dfsg1-1ubuntu1).
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 intel-media-va-driver-non-free : Conflicts: intel-media-va-driver but 22.3.1+dfsg1-1ubuntu1 is to be installed
E: Unable to correct problems, you have held broken packages.

[mdhiggins]

Might need to remove intel-media-va-driver and then install intel-media-va-driver-non-free

Updated 8e525011024f8b4df7a0c829768ab666587df2ef to prevent installing the free version which I guess was conflicting

[VampiricAlien]

Preopts
[
    "-fix_sub_duration",
    "-init_hw_device",
    "vaapi=sma:/dev/dri/renderD128",
    "-hwaccel_device",
    "sma",
    "-hwaccel",
    "vaapi",
    "-hwaccel_output_format",
    "vaapi"
]
Postopts
[
    "-threads",
    "0",
    "-metadata:g",
    "encoding_tool=SMA"

it's working!

Thanks for help, hope I didn't keep you from anything important.

[mdhiggins]

Glad you got things working! I'll update the packages accordingly to not use the free version before the nonfree

[VampiricAlien]

E: Unable to locate package mesa-dri-gallium But doesn't effect anything for me. Updated image and restarted. No errors!

I own you a few beers!

[mdhiggins]

Whoops fixed, that was supposed to be in the alpine package list