Docker mod

[abhishekjain14]

I've been considering using this with nzbget instead of radarr/sonarr to simplify my workflow and have the conversions happen on a local disk instead of a NAS. I don't see any up to date containers out there, but came across this little tidbit: https://blog.linuxserver.io/2019/09/14/customizing-our-containers/

Any reason SMA couldn't work as a docker mod, allowing us the capability to add it into any container that could use it? If not, any interest in doing so, @mdhiggins? If not, I'd be willing to give it a try.

[mdhiggins]

The mod is interesting, I can look into how that works can probably adapt most of the main functionality from the sonarr/radarr containers

[mdhiggins]

https://hub.docker.com/repository/docker/mdhiggins/sma-mod

Got a prelim version working in my tests after much trial and error

Mount the same points as the radarr/sonarr containers for the volumes and see if that works

[abhishekjain14]

It doesn't seem to be copying in postSonarr.sh to the right spot in my testing, but I am still poking at it.

[mdhiggins]

Looks like postRadarr.sh wasn't included but postSonarr.sh should have been, added in the latest update

[mdhiggins]

Tested and both postSonarr.sh and postRadarr.sh are included in latest update

[abhishekjain14]

21-4-10 14:58:17.0|Error|postSonarr.sh|/usr/local/sma/postSonarr.sh: line 3: /venv/bin/python3: No such file or directory
21-4-10 14:58:17.0|Debug|CustomScript|Executed external script: /usr/local/sma/postSonarr.sh - Status: 127
21-4-10 14:58:17.0|Debug|CustomScript|Script Output:
4/10/2021 9:58:17 PM - Error - /usr/local/sma/postSonarr.sh: line 3: /venv/bin/python3: No such file or directory

is the issue I am getting in the trace logs for sonarr. any thoughts?

[mdhiggins]

Looks like your SMA_PATH environment variable isn't being set, just pushed an update that attempts to fix this, the old method of setting I was using was apparently not working, let me know if this new update fixes it

[abhishekjain14]

So interestingly enough...with the newest fix I'm able to view SMA_PATH inside the container, but the script isn't able to access it:

root@07c26184cf9e:/usr/local/sma# echo $SMA_PATH
/usr/local/sma
root@07c26184cf9e:/usr/local/sma# ./postSonarr.sh
./postSonarr.sh: line 3: /venv/bin/python3: No such file or directory
root@07c26184cf9e:/usr/local/sma# cat postSonarr.sh
#!/usr/bin/env bash

$SMA_PATH/venv/bin/python3 $SMA_PATH/postSonarr.py

Given that the path is relative to the container, any issues with possibly hardcoding this?

[mdhiggins]

Try the latest update, think I got it working

[abhishekjain14]

We're close, I think.

Here's what I see in Sonarr:

21-4-12 16:39:40.4|Error|postSonarr.sh|/usr/local/sma/postSonarr.sh: line 3: /usr/local/sma: Is a directory
21-4-12 16:39:40.5|Warn|SonarrErrorPipeline|Invalid request Validation failed:
 -- : Script exited with code: 126

And in the container:

root@835d044acdcb:/usr/local/sma#` ./postSonarr.sh
./postSonarr.sh: line 3: /usr/local/sma: Is a directory

[mdhiggins]

Whoops looks like the new method was adding a newline at the end of the variable, just fixed it, container rebuilding now

[abhishekjain14]

When the script is called by Sonarr with a tv show:

2021-04-12 20:25:27 - SonarrPostProcess - INFO - Sonarr extra script post processing started.
2021-04-12 20:25:27 - resources.readsettings - INFO - /usr/local/sma/venv/bin/python3
2021-04-12 20:25:27 - resources.readsettings - INFO - Loading config file /usr/local/sma/config/autoProcess.ini.
2021-04-12 20:25:27 - SonarrPostProcess - ERROR - Error reading environment variables
Traceback (most recent call last):
  File "/usr/local/sma/postSonarr.py", line 178, in <module>
    tvdb_id = int(os.environ.get('sonarr_series_tvdbid'))
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

Within Sonarr when testing the script:

21-4-12 20:23:23.6|Warn|SonarrErrorPipeline|Invalid request Validation failed:
 -- : Script exited with code: 1

The closest thing I could find was this issue:

https://forums.sonarr.tv/t/custom-script-exited-with-code-1-error-postsonarr-py-script-from-sickbeard-mp4-automator-on-macos-mojave/22749/4

But it looks like a python 2 issue.

[mdhiggins]

Alright so using the with-contenv bash seems to include the variable but subsequently wipe the environment variables set by sonarr. This is getting frustrating

Just uploaded an alternative solution see if that works

[abhishekjain14]

This leads back to the previous issue of:

sonarr        | [Error] postSonarr.sh: /usr/local/sma/postSonarr.sh: line 3: /venv/bin/python3: No such file or directory
sonarr        | [Warn] SonarrErrorPipeline: Invalid request Validation failed:
sonarr        |  -- : Script exited with code: 127

in the sonarr logs.

[mdhiggins]

Alright I give up, latest update just handles it at the script level which should behave

[abhishekjain14]

Testing it right now. Thanks for working on this!

[abhishekjain14]

Testing in Sonarr works! I will report back with trying an actual conversion through Sonarr, but thanks for this! To make a script for NZBGet, the format would be similar to the other scripts I assume? Calling python, then NZBGetpostprocess.py?

[abhishekjain14]

And it's kicking it off! Perfect, thank you!

[mdhiggins]

NZBGet is a little weird cause it doesn't execute the script but actually reads and executes the python code natively so a shell script won't work, you'd probably have to copy or simlink NZBGetPostProcess to the final script destination

[mdhiggins]

Actually for NZBGet just overriding the SMA_PATH environment variable might be best to have it go to the NZBGet ScriptDir

[mdhiggins]

Looks like by default it goes to your /downloads/scripts

[abhishekjain14]

The rabbit hole goes deeper...because linuxserver is using alpine as its base image, the sma-config script fails. I'm taking a crack at it to see what I can get working, and if I'm successful, I'll send a PR over to you

[mdhiggins]

I'll probably have to include different tags (debian vs alpine etc)

[abhishekjain14]

Yeah, looks like it. Do you have any bandwidth for this? I've been meaning to poke at it, but haven't had much of a chance to do so.

[mdhiggins]

I might have some time later this week I'll update here if I get something together

[mdhiggins]

Added an alpine tag how's that work for you?

[abhishekjain14]

nzbget        | [cont-init.d] 90-sma-config: executing...
nzbget        | /var/run/s6/etc/cont-init.d/90-sma-config: line 38: dpkg: command not found
nzbget        | tar: short read
nzbget        | chown: cannot access '/usr/local/bin/ffmpeg': No such file or directory
nzbget        | chown: cannot access '/usr/local/bin/ffprobe': No such file or directory
nzbget        | chmod: cannot access '/usr/local/bin/ffmpeg': No such file or directory
nzbget        | chmod: cannot access '/usr/local/bin/ffprobe': No such file or directory
nzbget        | ERROR:root:autoProcess.ini does not exist
nzbget        | [cont-init.d] 90-sma-config: exited 1.

[abhishekjain14]

You, kind sir, have a pull request: https://github.com/mdhiggins/sma-mod/pull/1

[abhishekjain14]

Running 90-sma-config after that shows no errors. Testing out the post process script to see what happens

[abhishekjain14]

nzbget        | [INFO] Executing post-process-script NZBGetPostProcess.py for XXX.YYY
nzbget        | [INFO] NZBGetPostProcess: Traceback (most recent call last):
nzbget        | [INFO] NZBGetPostProcess:   File "/config/scripts/NZBGetPostProcess.py", line 84, in <module>
nzbget        | [INFO] NZBGetPostProcess:     from resources.mediaprocessor import MediaProcessor
nzbget        | [INFO] NZBGetPostProcess:   File "/usr/local/sma/resources/mediaprocessor.py", line 12, in <module>
nzbget        | [INFO] NZBGetPostProcess:     from resources.metadata import Metadata
nzbget        | [INFO] NZBGetPostProcess:   File "/usr/local/sma/resources/metadata.py", line 12, in <module>
nzbget        | [INFO] NZBGetPostProcess:     import tmdbsimple as tmdb
nzbget        | [INFO] NZBGetPostProcess: ModuleNotFoundError: No module named 'tmdbsimple'
nzbget        | [ERROR] NZBGetPostProcess: Wrong path to sickbeard_mp4_automator: /usr/local/sma/

But I see you've merged the PR...let me run it from the top and try again

[mdhiggins]

Hm this will be tricky, its failing there because NZBGet is using its own python environment while everything is installed under the virtual environment

I'm not 100% sure how to solve that, I'll have to investigate

[abhishekjain14]

Thanks man. I'll try to look into this as well. Do you happen to accept donations in any form? I'd love to at least be able to buy you a beer for all this work

[mdhiggins]

Does modifying the shebang at the top of the NZBGetPostProcess.py file help?

#!/usr/bin/env python3

to

#!/usr/local/sma/venv/bin/python3

[abhishekjain14]

Trying it right now

[abhishekjain14]

Sidenote...my method of installing dpkg is failing...perhaps it needs to be moved into the Dockerfile. Manually running "apk add dpkg" in the container allows me to run 90-sma-config with no errors

[abhishekjain14]

Success, it works!!! A little too well....NZBGetPostProcess is sending this to SMA in the logs:

2021-04-19 06:13:52 - NZBGetPostProcess - INFO - ======================
2021-04-19 06:13:52 - NZBGetPostProcess - INFO - 0
2021-04-19 06:13:53 - NZBGetPostProcess - INFO - 0
2021-04-19 06:13:53 - NZBGetPostProcess - INFO - 1
2021-04-19 06:13:54 - NZBGetPostProcess - INFO - 1
2021-04-19 06:13:54 - NZBGetPostProcess - INFO - 1
2021-04-19 06:13:55 - NZBGetPostProcess - INFO - 2
2021-04-19 06:13:56 - NZBGetPostProcess - INFO - 3
2021-04-19 06:13:56 - NZBGetPostProcess - INFO - 3
2021-04-19 06:13:57 - NZBGetPostProcess - INFO - 4
2021-04-19 06:13:57 - NZBGetPostProcess - INFO - 5
2021-04-19 06:13:58 - NZBGetPostProcess - INFO - 5
2021-04-19 06:13:58 - NZBGetPostProcess - INFO - 6
2021-04-19 06:13:59 - NZBGetPostProcess - INFO - 6
2021-04-19 06:13:59 - NZBGetPostProcess - INFO - 8
2021-04-19 06:14:00 - NZBGetPostProcess - INFO - 9
2021-04-19 06:14:00 - NZBGetPostProcess - INFO - 10
2021-04-19 06:14:01 - NZBGetPostProcess - INFO - 10
2021-04-19 06:14:01 - NZBGetPostProcess - INFO - 11
2021-04-19 06:14:02 - NZBGetPostProcess - INFO - 13

[abhishekjain14]

But that's a mere nitpick! I think we almost have it

[abhishekjain14]

Yep, as far as I can tell it's working smoothly!

[Protektor-Desura]

Looks interesting. I may have to give this a spin and test it out for my setup.

[abhishekjain14]

@mdhiggins When you get a moment, could you take a look at https://github.com/mdhiggins/sma-mod/pull/3? I don't know how I'd test a docker mod like this locally

[Protektor-Desura]

I will say you will probably need to add Mesa upgrades if your docker isn't running Mesa 20.1 or higher or you can't do HEVC encoding/transcoding with VAAPI. Linuxserver/Jellyfin has an AMD/VAAPI docker mod for just that. I recommend using Ubuntu as the base for your docker mod since that is what so many of the Linuxserver dockers use.

[abhishekjain14]

My latest PR against alpine didn't fix the dpkg install issue. Any thoughts on what I might be doing wrong?

[Protektor-Desura]

The Linuxserver Radarr docker is built again Ubuntu 20.04 Focal. You might want to give that a try.

FROM ghcr.io/linuxserver/baseimage-ubuntu:focal https://github.com/linuxserver/docker-radarr/blob/master/Dockerfile

[abhishekjain14]

The issue is NZBGet is built against alpine, so the same scripts don't work between the two, as far as I can tell

[mdhiggins]

0755ce3eb2ba3b527c6c3f347d98ebbb47a9c237

Just pushed this update which uses the arch command instead of dpkg to determine the architecture, I think I account for all possible outputs, let me know

[mdhiggins]

Out of curiosity, does this file work with NZBGet?

#!/usr/local/sma/venv/bin/python3
#
##############################################################################
### NZBGET POST-PROCESSING SCRIPT                                          ###

# Converts files and passes them to Sonarr for further processing.
#
# NOTE: This script requires Python to be installed on your system.

##############################################################################
### OPTIONS                                                                ###

# Change to full path to MP4 Automator folder. No quotes and a trailing /
#MP4_FOLDER=~/sickbeard_mp4_automator/

# Convert file before passing to destination (True, False)
#SHOULDCONVERT=False

# Category for Couchpotato
#CP_CAT=Couchpotato

# Category for Sonarr
#SONARR_CAT=Sonarr

# Category for Radarr
#RADARR_CAT=Radarr

# Category for Sickbeard
#SICKBEARD_CAT=Sickbeard

# Category for Sickrage
#SICKRAGE_CAT=Sickrage

# Category for bypassing any further processing but still converting
#BYPASS_CAT=Bypass

# Custom output_directory setting
#OUTPUT_DIR=

# Custom path mapping setting
#PATH_MAPPING=

### NZBGET POST-PROCESSING SCRIPT                                          ###
##############################################################################

import sys
import os

path = os.path.join(os.path.split(sys.argv[0])[0], "NZBGetPostProcess.py")
exec(open(path).read())

Idea being that I could have that as a wrapper file and include it with sma-mod without needing to duplicate NZBGetPostProcess.py and modify the shebang It should just include and exec NZBGetPostProcess.py using its own shebang but I'm not sure how NZBGet will handle reading it, if you could test and let me know that would be great

[mdhiggins]

Updated sma-mod to include this file

[abhishekjain14]

So far, so good! SMA is running successfully, and I will update as soon as it completes, but it looks great, as do your changes with the arch command. I'm able to get the container up and running without having to go in and install dpkg manually.

I am seeing this error in the logs:

2021-04-29 19:58:39 - NZBGetPostProcess - ERROR - Pymediainfo exception.
Traceback (most recent call last):
  File "/usr/local/sma/resources/mediaprocessor.py", line 306, in isValidSource
    media_info = MediaInfo.parse(inputfile)
  File "/usr/local/sma/venv/lib/python3.8/site-packages/pymediainfo/__init__.py", line 422, in parse
    lib, handle, lib_version_str, lib_version = cls._get_library(library_file)
  File "/usr/local/sma/venv/lib/python3.8/site-packages/pymediainfo/__init__.py", line 324, in _get_library
    raise OSError("Failed to load library")
OSError: Failed to load library

From looking at the log file, this was happening previously as well, and I didn't notice. Any idea what may be causing it, or if it's safe to ignore?

[mdhiggins]

This error is from pymediainfo missing its requirement package. I updated the 90-sma-config file to include this package on ubuntu based installations (python-pymediainfo)

Alpine doesn't have a package yet though it is on edge so I guess its being added at some point (py3-pymediainfo)

Either way it can ignored, its used to pull some supplemental data about the source file but not required

[Protektor-Desura]

I tried to add this to linuxserver/radarr and it is giving me an error.

[mod-init] Attempting to run Docker Modification Logic [mod-init] Applying mdhiggins/sma-mod files to container gzip: stdin: not in gzip format tar: Child returned status 1 tar: Error is not recoverable: exiting now

---

version: "2.1" services: radarr: image: linuxserver/radarr:latest container_name: radarr hostname: radarr network_mode: bridge environment:

• PUID=1000
• PGID=1000
• TZ=America/Chicago
• DOCKER_MODS=mdhiggins/sma-mod #optional volumes:
• /home/docker/radarr:/config
• /srv/media/libraries:/libraries
• /srv/media/downloads:/downloads ports:
• 7878:7878 restart: unless-stopped