mdhiggins / sonarr-sma

Sonarr docker based on linuxserver/sonarr with SMA built in using python3
MIT License
37 stars 18 forks source link

linuxserver.io's docker image uses Alpine, Apline no longer has non-free repos #60

Open TravisRoy opened 1 month ago

TravisRoy commented 1 month ago

Linuxsever.io's docker container for Sonarr uses their Alpine build. Alpine dropped non-free repos, so that means no more intel-media-va-driver-non-free needed for vaapi support.

Linuxserver.io's ffmpeg doker container uses their Debian build and has support for vaapi.

TravisRoy commented 1 month ago

Okay, this is messy, I know, I've never made a Dockerfile before, and I was totally winging it. But I redid it with the base Debian official image, used the part that pulls Sonarr from the linuxserver.io Sonarr image, and then your code for the SMA part. Like I said, I know this has issues, there are multiple ENV variables set due to issues I was running into.

But I hope this can help you out, I'm sure it could work for Radarr too, and it no longer relies on the linuxserver.io image that seems to be changing often enough to be annoying.


# syntax=docker/dockerfile:1

FROM debian as rootfs-stage

# set label
LABEL maintainer="Sc00ter"

# set environment variables
ENV XDG_CONFIG_HOME="/config/xdg"
ENV SONARR_CHANNEL="v4-stable"
ENV SONARR_BRANCH="main"
ENV SMA_PATH /usr/local/sma
ENV SMA_RS Sonarr
ENV SMA_UPDATE false
ENV SMA_FFMPEG_PATH ffmpeg
ENV SMA_FFPROBE_PATH ffprobe
ENV SMA_FFMPEG_URL https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz

# environment
ENV REL=jammy
ENV ARCH=amd64

# install packages and add non-free repo
RUN \
  apt update && apt install -y \
    bash \
    curl \
    tzdata \
    xz-utils \
    software-properties-common && \
  add-apt-repository -y --component non-free

# grab base tarball
RUN \
  mkdir /root-out && \
  curl -o \
    /rootfs.tar.gz -L \
    https://partner-images.canonical.com/core/${REL}/current/ubuntu-${REL}-core-cloudimg-${ARCH}-root.tar.gz && \
  tar xf \
    /rootfs.tar.gz -C \
    /root-out && \
  rm -rf \
    /root-out/var/log/*

# set version for s6 overlay
ARG S6_OVERLAY_VERSION="3.1.6.2"
ARG S6_OVERLAY_ARCH="x86_64"

# add s6 overlay
ADD https://github.com/just-containers/s6-overlay/releases/download/v${S6_OVERLAY_VERSION}/s6-overlay-noarch.tar.xz /tmp
RUN tar -C /root-out -Jxpf /tmp/s6-overlay-noarch.tar.xz
ADD https://github.com/just-containers/s6-overlay/releases/download/v${S6_OVERLAY_VERSION}/s6-overlay-${S6_OVERLAY_ARCH}.tar.xz /tmp
RUN tar -C /root-out -Jxpf /tmp/s6-overlay-${S6_OVERLAY_ARCH}.tar.xz

# add s6 optional symlinks
ADD https://github.com/just-containers/s6-overlay/releases/download/v${S6_OVERLAY_VERSION}/s6-overlay-symlinks-noarch.tar.xz /tmp
RUN tar -C /root-out -Jxpf /tmp/s6-overlay-symlinks-noarch.tar.xz
ADD https://github.com/just-containers/s6-overlay/releases/download/v${S6_OVERLAY_VERSION}/s6-overlay-symlinks-arch.tar.xz /tmp
RUN tar -C /root-out -Jxpf /tmp/s6-overlay-symlinks-arch.tar.xz

# Runtime stage
FROM scratch
COPY --from=rootfs-stage /root-out/ /
ARG BUILD_DATE
ARG VERSION
ARG MODS_VERSION="v3"
ARG PKG_INST_VERSION="v1"
LABEL build_version="Linuxserver.io version:- ${VERSION} Build-date:- ${BUILD_DATE}"
LABEL maintainer="TheLamer"

# set environment variables
ARG DEBIAN_FRONTEND="noninteractive"
ENV HOME="/root" \
  LANGUAGE="en_US.UTF-8" \
  LANG="en_US.UTF-8" \
  TERM="xterm" \
  S6_CMD_WAIT_FOR_SERVICES_MAXTIME="0" \
  S6_VERBOSITY=1 \
  S6_STAGE2_HOOK=/docker-mods \
  VIRTUAL_ENV=/lsiopy \
  PATH="/lsiopy/bin:$PATH"

RUN \
  echo "**** Ripped from Ubuntu Docker Logic ****" && \
  set -xe && \
  echo '#!/bin/sh' \
    > /usr/sbin/policy-rc.d && \
  echo 'exit 101' \
    >> /usr/sbin/policy-rc.d && \
  chmod +x \
    /usr/sbin/policy-rc.d && \
  dpkg-divert --local --rename --add /sbin/initctl && \
  cp -a \
    /usr/sbin/policy-rc.d \
    /sbin/initctl && \
  sed -i \
    's/^exit.*/exit 0/' \
    /sbin/initctl && \
  echo 'force-unsafe-io' \
    > /etc/dpkg/dpkg.cfg.d/docker-apt-speedup && \
  echo 'DPkg::Post-Invoke { "rm -f /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin || true"; };' \
    > /etc/apt/apt.conf.d/docker-clean && \
  echo 'APT::Update::Post-Invoke { "rm -f /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin || true"; };' \
    >> /etc/apt/apt.conf.d/docker-clean && \
  echo 'Dir::Cache::pkgcache ""; Dir::Cache::srcpkgcache "";' \
    >> /etc/apt/apt.conf.d/docker-clean && \
  echo 'Acquire::Languages "none";' \
    > /etc/apt/apt.conf.d/docker-no-languages && \
  echo 'Acquire::GzipIndexes "true"; Acquire::CompressionTypes::Order:: "gz";' \
    > /etc/apt/apt.conf.d/docker-gzip-indexes && \
  echo 'Apt::AutoRemove::SuggestsImportant "false";' \
    > /etc/apt/apt.conf.d/docker-autoremove-suggests && \
  mkdir -p /run/systemd && \
  echo 'docker' \
    > /run/systemd/container && \
  echo "**** install apt-utils and locales ****" && \
  apt-get update && \
  apt-get upgrade -y && \
  apt-get install -y \
    apt-utils \
    locales && \
  echo "**** install packages ****" && \
  apt-get install -y \
    catatonit \
    cron \
    curl \
    gnupg \
    jq \
    netcat \
    tzdata && \
  echo "**** generate locale ****" && \
  locale-gen en_US.UTF-8 && \
  echo "**** create abc user and make our folders ****" && \
  useradd -u 911 -U -d /config -s /bin/false abc && \
  usermod -G users abc && \
  mkdir -p \
    /app \
    /config \
    /defaults \
    /lsiopy && \
  echo "**** cleanup ****" && \
  apt-get autoremove && \
  apt-get clean && \
  rm -rf \
    /tmp/* \
    /var/lib/apt/lists/* \
    /var/tmp/* \
    /var/log/*

ENV SONARR_CHANNEL="v4-stable"
ENV SONARR_BRANCH="main"
ENV SMA_PATH /usr/local/sma
ENV SMA_RS Sonarr
ENV SMA_UPDATE false
ENV SMA_FFMPEG_PATH ffmpeg
ENV SMA_FFPROBE_PATH ffprobe
ENV SMA_FFMPEG_URL https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz

# Start Sonarr
RUN \
  echo "**** install packages ****" && \
  apt update && apt install -y \
    libicu-dev \
    python3-guessit \
    python3-tmdbsimple \
    python3-mutagen \
    libsqlite3-dev \
    dialog \
    whiptail \
    apt-utils \
    xmlstarlet \
    dirmngr \
    curl \
    ffmpeg \
    gnupg \
    apt-transport-https \
    ca-certificates \
    wget \
    git \
    python3-pip \
    python3-venv \
    intel-media-va-driver-non-free \
    software-properties-common && \
  apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF && \
  apt-add-repository 'deb https://download.mono-project.com/repo/ubuntu stable-focal main' && \
  apt install -y mono-complete && \
  pip install qtfaststart && \
  pip install plexapi && \

  echo "**** install sonarr ****" && \
  mkdir -p /app/sonarr/bin && \
  if [ -z ${SONARR_VERSION+x} ]; then \
    SONARR_VERSION=$(curl -sX GET http://services.sonarr.tv/v1/releases \
    | jq -r "first(.[] | select(.releaseChannel==\"${SONARR_CHANNEL}\") | .version)"); \
  fi && \
  curl -o \
    /tmp/sonarr.tar.gz -L \
    "https://services.sonarr.tv/v1/update/${SONARR_BRANCH}/download?version=${SONARR_VERSION}&os=linuxmusl&runtime=netcore&arch=x64" && \
  tar xzf \
    /tmp/sonarr.tar.gz -C \
    /app/sonarr/bin --strip-components=1 && \
  echo -e "UpdateMethod=docker\nBranch=${SONARR_BRANCH}\nPackageVersion=${VERSION:-LocalBuild}\nPackageAuthor=[linuxserver.io](https://linuxserver.io)" > /app/sonarr/package_info && \
  echo "**** cleanup ****" && \
  rm -rf \
    /app/sonarr/bin/Sonarr.Update \
    /tmp/*

# Install SMA
RUN \
# make directory
  mkdir ${SMA_PATH} && \
# download repo
  git config --global --add safe.directory ${SMA_PATH} && \
  git clone https://github.com/mdhiggins/sickbeard_mp4_automator.git ${SMA_PATH} && \
# install pip, venv, and set up a virtual self contained python environment
  python3 -m venv ${SMA_PATH}/venv && \
  ${SMA_PATH}/venv/bin/pip install -r ${SMA_PATH}/setup/requirements.txt

# add local files
COPY root/ /

# ports and volumes
EXPOSE 8989

VOLUME /config
VOLUME /usr/local/sma/config

ENTRYPOINT ["/app/sonarr/bin/Sonarr"]
mdhiggins commented 1 month ago

Sonarr has been on Alpine for quite a while now, almost a year? The VAAPI configurations and packages have been updated for Alpine and should be working fine and the containers run a dynamic start script that adjust which packages get installed depending on if its alpine or ubuntu based

Not looking to fork away from linuxserver as parent at this time but definitely no reason you couldn't implement a custom solution

Take a look here

https://github.com/mdhiggins/sonarr-sma/blob/master/root/etc/s6-overlay/s6-rc.d/init-sma-config/run

That's where the packages and hardware acceleration environment variables are handled

TravisRoy commented 1 month ago

The change to remove non-free was recent with Alpine. That's what broke vaapi support.

intel-media-va-driver-non-free doesn't exist in Alpine anymore.

mdhiggins commented 1 month ago

The script doesn't try to install that on Alpine though, only Ubuntu based containers

Alpine uses

libva libgomp vidstab ca-certificates libva-intel-driver intel-media-driver mesa-va-gallium mesa-dri-gallium

TravisRoy commented 1 month ago

There is no ubuntu container from linuxserver.io for sonarr anymore, it pulls alpine even if you need the non-free drivers.

mdhiggins commented 1 month ago

Yeah I understand that, its been like that for well over a year, the packages in alpine are named differently, alpine never had a package named intel-media-va-driver-non-free and the script has not attempted to install it since the alpine migration

the alpine hardware packages are listed above and seem to be working fine in my testing

mdhiggins commented 1 month ago

And just to officially confirm, I just pulled the latest sonarr container, running alpine, and did a quick vaapi test conversion and it works fine

root@0e2c51781c3e:/downloads/import# cat /etc/os-release
NAME="Alpine Linux"
ID=alpine
VERSION_ID=3.19.1
PRETTY_NAME="Alpine Linux v3.19"
HOME_URL="https://alpinelinux.org/"
BUG_REPORT_URL="https://gitlab.alpinelinux.org/alpine/aports/-/issues"
root@0e2c51781c3e:ffmpeg -y -vaapi_device /dev/dri/renderD128 -i ./input.mkv -vf 'format=nv12,hwupload' -c:v h264_vaapi ./output.mp4
ffmpeg version 6.1.1 Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 13.2.1 (Alpine 13.2.1_git20231014) 20231014
  configuration: --prefix=/usr --disable-librtmp --disable-lzma --disable-static --disable-stripping --enable-avfilter --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-libmp3lame --enable-libopenmpt --enable-libopus --enable-libplacebo --enable-libpulse --enable-librav1e --enable-librist --enable-libsoxr --enable-libsrt --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-lto=auto --enable-lv2 --enable-openssl --enable-pic --enable-postproc --enable-pthreads --enable-shared --enable-vaapi --enable-vdpau --enable-version3 --enable-vulkan --optflags=-O3 --enable-libjxl --enable-libsvtav1 --enable-libvpl
  libavutil      58. 29.100 / 58. 29.100
  libavcodec     60. 31.102 / 60. 31.102
  libavformat    60. 16.100 / 60. 16.100
  libavdevice    60.  3.100 / 60.  3.100
  libavfilter     9. 12.100 /  9. 12.100
  libswscale      7.  5.100 /  7.  5.100
  libswresample   5.  0.100 /  5.  0.100
  libpostproc    57.  3.100 / 57.  3.100
Input #0, matroska,webm, from './input.mkv':
  Metadata:
    encoder         : libebml v1.0.0 + libmatroska v1.0.0
    creation_time   : 2010-08-21T18:06:43.000000Z
    TITLE           : test
    DATE_RELEASED   : 2010
    COMMENT         : Matroska Validation File 8, secondary audio commentary track, misc subtitle tracks
  Duration: 00:00:46.67, start: 0.000000, bitrate: 5445 kb/s
  Stream #0:0: Video: h264 (Main), yuv420p(tv, smpte170m/smpte170m/bt709, progressive), 1024x576, SAR 1:1 DAR 16:9, 24 fps, 24 tbr, 1k tbn (default)
  Stream #0:1: Audio: aac (LC), 48000 Hz, stereo, fltp (default)
  Stream #0:2(eng): Subtitle: subrip (default)
  Stream #0:3(hun): Subtitle: subrip
  Stream #0:4(ger): Subtitle: subrip
  Stream #0:5(fre): Subtitle: subrip
  Stream #0:6(spa): Subtitle: subrip
  Stream #0:7(ita): Subtitle: subrip
  Stream #0:8(eng): Audio: aac (LC), 22050 Hz, mono, fltp
    Metadata:
      title           : Commentary
  Stream #0:9(jpn): Subtitle: subrip
  Stream #0:10: Subtitle: subrip
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_vaapi))
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
[h264_vaapi @ 0x7f2324696800] No quality level set; using default (20).
Output #0, mp4, to './output.mp4':
  Metadata:
    DATE_RELEASED   : 2010
    COMMENT         : Matroska Validation File 8, secondary audio commentary track, misc subtitle tracks
    TITLE           : test
    encoder         : Lavf60.16.100
  Stream #0:0: Video: h264 (High) (avc1 / 0x31637661), vaapi(tv, smpte170m/smpte170m/bt709, progressive), 1024x576 [SAR 1:1 DAR 16:9], q=2-31, 24 fps, 12288 tbn (default)
    Metadata:
      encoder         : Lavc60.31.102 h264_vaapi
  Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      encoder         : Lavc60.31.102 aac
[out#0/mp4 @ 0x7f232469a840] video:17101kB audio:728kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.230990%
frame= 1104 fps=334 q=-0.0 Lsize=   17870kB time=00:00:45.98 bitrate=3183.5kbits/s speed=13.9x
[aac @ 0x7f2324362000] Qavg: 518.684
TravisRoy commented 1 month ago

Then what package is installed to allow vaapi to encode? because the free one doesn't.

TravisRoy commented 1 month ago

Oh, I see you did a h264, I was doing hevc encoding.

mdhiggins commented 1 month ago

HEVC works fine too

root@0e2c51781c3e:/downloads/import# ffmpeg -y -vaapi_device /dev/dri/renderD128 -i ./input.mkv -vf 'format=nv12,hwupload' -c:v hevc_vaapi ./output.mp4
ffmpeg version 6.1.1 Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 13.2.1 (Alpine 13.2.1_git20231014) 20231014
  configuration: --prefix=/usr --disable-librtmp --disable-lzma --disable-static --disable-stripping --enable-avfilter --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-libmp3lame --enable-libopenmpt --enable-libopus --enable-libplacebo --enable-libpulse --enable-librav1e --enable-librist --enable-libsoxr --enable-libsrt --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-lto=auto --enable-lv2 --enable-openssl --enable-pic --enable-postproc --enable-pthreads --enable-shared --enable-vaapi --enable-vdpau --enable-version3 --enable-vulkan --optflags=-O3 --enable-libjxl --enable-libsvtav1 --enable-libvpl
  libavutil      58. 29.100 / 58. 29.100
  libavcodec     60. 31.102 / 60. 31.102
  libavformat    60. 16.100 / 60. 16.100
  libavdevice    60.  3.100 / 60.  3.100
  libavfilter     9. 12.100 /  9. 12.100
  libswscale      7.  5.100 /  7.  5.100
  libswresample   5.  0.100 /  5.  0.100
  libpostproc    57.  3.100 / 57.  3.100
Input #0, matroska,webm, from './input.mkv':
  Metadata:
    encoder         : libebml v1.0.0 + libmatroska v1.0.0
    creation_time   : 2010-08-21T18:06:43.000000Z
    TITLE           : test
    DATE_RELEASED   : 2010
    COMMENT         : Matroska Validation File 8, secondary audio commentary track, misc subtitle tracks
  Duration: 00:00:46.67, start: 0.000000, bitrate: 5445 kb/s
  Stream #0:0: Video: h264 (Main), yuv420p(tv, smpte170m/smpte170m/bt709, progressive), 1024x576, SAR 1:1 DAR 16:9, 24 fps, 24 tbr, 1k tbn (default)
  Stream #0:1: Audio: aac (LC), 48000 Hz, stereo, fltp (default)
  Stream #0:2(eng): Subtitle: subrip (default)
  Stream #0:3(hun): Subtitle: subrip
  Stream #0:4(ger): Subtitle: subrip
  Stream #0:5(fre): Subtitle: subrip
  Stream #0:6(spa): Subtitle: subrip
  Stream #0:7(ita): Subtitle: subrip
  Stream #0:8(eng): Audio: aac (LC), 22050 Hz, mono, fltp
    Metadata:
      title           : Commentary
  Stream #0:9(jpn): Subtitle: subrip
  Stream #0:10: Subtitle: subrip
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> hevc (hevc_vaapi))
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
[hevc_vaapi @ 0x7f6d80a4c800] No quality level set; using default (25).
Output #0, mp4, to './output.mp4':
  Metadata:
    DATE_RELEASED   : 2010
    COMMENT         : Matroska Validation File 8, secondary audio commentary track, misc subtitle tracks
    TITLE           : test
    encoder         : Lavf60.16.100
  Stream #0:0: Video: hevc (Main) (hev1 / 0x31766568), vaapi(tv, smpte170m/smpte170m/bt709, progressive), 1024x576 [SAR 1:1 DAR 16:9], q=2-31, 24 fps, 12288 tbn (default)
    Metadata:
      encoder         : Lavc60.31.102 hevc_vaapi
  Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      encoder         : Lavc60.31.102 aac
[out#0/mp4 @ 0x7f6d80a50840] video:6915kB audio:728kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.539977%
frame= 1104 fps=179 q=-0.0 Lsize=    7684kB time=00:00:45.98 bitrate=1368.8kbits/s speed=7.44x
[aac @ 0x7f6d8082c000] Qavg: 518.684
mdhiggins commented 1 month ago

I'm using SMA_USE_REPO=true and SMA_HWACCEL=true for the environment variables with no additional packages installed on a clean container pull

TravisRoy commented 1 month ago

[AVHWDeviceContext @ 0x55f122258d40] No VA display found for device /dev/dri/renderD128.

I get that after a fresh docker install.

mdhiggins commented 1 month ago

You do have the device mounted in the container correct? And the hardware works outside the container for encoding?

mdhiggins commented 1 month ago

You also might need to check your host permissions to ensure your docker user can access the device

TravisRoy commented 1 month ago

When I use the container I made, it works fine.

mdhiggins commented 1 month ago

When I use the container I made, it works fine too but that doesn't help

Still doesn't mean it's not a permission issue

Might be useful to post your docker configuration to compare to mine that's working

Muckoma commented 1 month ago

SMA_HWACCEL

How do you mount the device in Docker? I'm assuming only through docker compose script but can't do it via Synology docker UI?

mdhiggins commented 1 month ago

I've never used synology so you're on your own there but for docker compose its very similar to how volumes work

The devices section provides access

  sonarr:
    image: ghcr.io/mdhiggins/sonarr-sma
    container_name: sonarr
    devices:
      - /dev/dri:/dev/dri
    volumes:
      - /opt/appdata/sonarr:/config
      - /opt/appdata/sma:/usr/local/sma/config
      - /mnt/storage/tv:/tv
      - /mnt/storage/downloads:/downloads
    tmpfs:
      - /tmp
    environment:
      - PUID=${PUID}
      - PGID=${PGID}
      - TZ=${TZ}
      - SMA_UPDATE=true
      - SMA_USE_REPO=true
      - SMA_HWACCEL=true
    restart: unless-stopped

The container already contains a startup script that should handle ensuring that your PUID/PGID permissions are appropriately set inside the container for the devices

https://github.com/mdhiggins/sonarr-sma/blob/master/root/etc/s6-overlay/s6-rc.d/init-gid-video/run

You do need to make sure the devices have the appropriate permissions on the host though

For example, my docker images are run by dockeruser which is set by the PUID and PGID environment variables

The devices have the following permissions on the host machine

crw-rw---- 1 root video  226,   0 May 24 03:30 card0
crw-rw---- 1 root render 226, 128 May 24 03:30 renderD128

and dockeruser belongs to both the video and render groups

dockeruser : dockeruser tty uucp dialout video render docker