MacOS 12.3 removed python 2.7 - How to get nzbtomedia to recognize python 3.9 or python3?

[rr0ss0rr]

Apple removed python 2.7 in /usr/bin and I think replaced it with a python3 wrapper. I do have python 3.9 installed via homebrew with a soft link pointing to it in /usr/local/bin/python. Looks like sabnzbd is throwing an error from nzbtomedia: ScriptExit(127) env: python: No such file or directory.
/usr/local/bin has been added to /etc/paths along with /usr/local/opt/python/libexec/bin. What can I do to have nzbtomedia recognize that python is either in /usr/local/bin or /usr/local/opt/python/libexec/bin without going through hoops on opening up /usr/bin?

Thanks

[clinton-hall]

I am always perplexed by these issues. I don't have MacOS, so can only go with general posix configurations. What you have done with the linking sounds like it should work... I assume you have restarted sabnzbd (or indeed restart your system) to ensure the PATH in use has been updated?

[rr0ss0rr]

Me to. I've had python 3 installed for quite some time even though I believe that nzbtomedia was still using 2.7. Apple locking down /usr/bin doesn't help one bit. I'll probably change the 2 scripts that I use to point to /usr/local/bin/python, but then I can get out of sync with the master. Oh well.

[rEes9P]

Ran into a bunch of env: python: No such file or directory messages after upgrading to macOS Monterey. Is the fantastic nzbToMedia likely to work with Python3 anytime in the near future? What is the best workaround for now? Thanks!

[clinton-hall]

Just to clarify here. nzbToMedia will work with Python2.7 (but i am not specifically testing new features) and all currently supported versions of Python3.

The issues with MacOS are related to the inability to install or link ANY python into /usr/bin

As a note. If NZBGet is available, that has a great feature where you can define extension alias so that any .py file will get executed with /usr/local/bin/python etc. I'm not sure of anything similar for SABnzbd... you could try to create a .sh file, and let SabNZBD run that as the script. The contents would be:

#!/bin/bash
/usr/local/bin/python <path to scripts>/nzbToMedia.py "$@"

[rEes9P]

Just to clarify here. nzbToMedia will work with Python2.7 (but i am not specifically testing new features) and all currently supported versions of Python3.

The issues with MacOS are related to the inability to install or link ANY python into /usr/bin

As a note. If NZBGet is available, that has a great feature where you can define extension alias so that any .py file will get executed with /usr/local/bin/python etc.

Thanks for this @clinton-hall. I am using NZBGet. My current ShellOverride setting is .py2=/usr/bin/python;.py3=/usr/local/bin/python3

The command which python on Terminal shows /usr/local/opt/python/libexec/bin/python

Could you please tell me what modification I need to make to ShellOverride? Is that the NZBGet setting you are referring to or is there another one I should change as well?

[clinton-hall]

@rEes9P You are very close.... the problem is the script is a .py, not .py2 or .py3 so... set your ShellOverride .py=/usr/bin/python3 you can always just add this to the list .py2=/usr/bin/python;.py3=/usr/local/bin/python3;.py=/usr/local/bin/python3

[rEes9P]

@clinton-hall, many thanks! I added .py=/usr/local/bin/python3 to my original ShellOverride list and nzbToMedia is back up and running. I still can't get FakeDetector, PasswordDetector, ExtendedUnrar and EasySort to work though 🤷‍♂️ Your expertise would be much appreciated in finding any leads!

[clinton-hall]

ok... the issue here is I don't believe those scripts have been updated to support Python3...

I would try: Change ShellOverride

.py2=/usr/bin/python2;.py3=/usr/local/bin/python3;.py=/usr/local/bin/python3

this will force .py2 files to call python2, .py3 files to call python3, and other ,py files to call python.

then edit the extension of these scripts that don't support python3. i.e. rename FakeDetector.py to FakeDetector.py2 and then set that up appropriately in NZBGet... if that works, try the same with the other scripts.

[rEes9P]

Thanks for this @clinton-hall. I don't have Python2 installed though as it's no longer bundled with the version of macOS I am on and is not easily available on Homebrew either :-/ I'm guessing that would come in the way of the workaround you recommend?

[clinton-hall]

if you don't have python2, then you won't be able to use these legacy scripts until someone upgrades them to be Python3 compatible.

I might be able to have a look at fakedetector and see what needs to be updated...

[rEes9P]

Thank you! Only if and when convenient 🙏🏼

I did get your Python3 version of DeleteSamples

[clinton-hall]

try this for FakeDetector https://github.com/clinton-hall/FakeDetector

[clinton-hall]

password detector should work fine with python3 check you have the latest https://github.com/Prinz23/NZBGetPasswordDetector

[clinton-hall]

latest ExtendedUnrar should work with python3 https://github.com/chazlarson/NZBGetScripts

[clinton-hall]

EasySort

#!/usr/bin/env python
#
# EasySort post-processing script for NZBGet.
#
# Copyright (C) 2015 Andrey Prygunkov <hugbug@users.sourceforge.net>
#
# This program is free software; you can redistribute it and/or modify it
# under the terms of the GNU Lesser General Public License as published by
# the Free Software Foundation; either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.    See the
# GNU Lesser General Public License for more details.
#
# You should have received a copy of the GNU Lesser General Public License
# along with the program.  If not, see <http://www.gnu.org/licenses/>.
#

##############################################################################
### NZBGET POST-PROCESSING SCRIPT                                           ###

# Move files into other location.
#
# This script moves files with specified extensions into another
# directory. It can also move into parent directory (flatten mode). In addition
# there is an option to delete source directory with all remaining files.
#
# Info about pp-script:
# Author: Andrey Prygunkov (nzbget@gmail.com).
# License: GPLv3 (http://www.gnu.org/licenses/gpl.html).
# PP-Script Version: 1.0.
#
# NOTE: This script requires Python 2.x to be installed on your system.

##############################################################################
### OPTIONS                                                                   ###

# Destination directory to move files into.
#
# Set to ".." to move into parent directory (flatten mode).
#DestDir=..

# File extensions to process.
#
# Only files with these extensions are processed. Extensions must
# be separated with commas.
#
# Example=.mkv,.avi,.divx
#
# If the list of extensions is empty all files are processed.
#Extensions=

# Minimum file size (Kilobytes).
#
# Smaller files are ignored.
#MinSize=1000

# Overwrite files at destination (yes, no).
#
# If not active the files are still moved into destination but
# unique suffixes are added at the end of file names, e.g. My.File.(2).mkv.
#Overwrite=no

# Delete download directory after renaming (yes, no).
#
# Delete download directory after processing. If no files could be processed,
# the directory remains untouched.
#Cleanup=yes

# Preview mode (yes, no).
#
# When active no changes to file system are made but the destination
# file names are logged. Useful for debugging.
#Preview=no

# Print more logging messages (yes, no).
#
# For debugging or if you need to report a bug.
#Verbose=no

### NZBGET POST-PROCESSING SCRIPT                                           ###
##############################################################################

import sys
import os
import shutil

# Exit codes used by NZBGet
POSTPROCESS_SUCCESS=93
POSTPROCESS_NONE=95
POSTPROCESS_ERROR=94

# Check if the script is called from nzbget 15.0 or later
if not 'NZBOP_NZBLOG' in os.environ:
    print('*** NZBGet post-processing script ***')
    print('This script is supposed to be called from nzbget (15.0 or later).')
    sys.exit(POSTPROCESS_ERROR)

# Check if directory still exist (for post-process again)
if not os.path.exists(os.environ['NZBPP_DIRECTORY']):
    print(('[INFO] Destination directory %s doesn\'t exist, exiting' % os.environ['NZBPP_DIRECTORY']))
    sys.exit(POSTPROCESS_NONE)

# Check status for errors
if os.environ['NZBPP_TOTALSTATUS'] != 'SUCCESS':
    print(('[WARNING] Download of "%s" has failed, exiting' % (os.environ['NZBPP_NZBNAME'])))
    sys.exit(POSTPROCESS_NONE)

# Check if all required script config options are present in config file
required_options = ('NZBPO_DestDir', 'NZBPO_Extensions', 'NZBPO_MinSize',
    'NZBPO_Overwrite', 'NZBPO_Cleanup', 'NZBPO_Preview', 'NZBPO_Verbose')
for optname in required_options:
    if (not optname.upper() in os.environ):
        print(('[ERROR] Option %s is missing in configuration file. Please check script settings' % optname[6:]))
        sys.exit(POSTPROCESS_ERROR)

# Init script config options
nzb_name=os.environ['NZBPP_NZBNAME']
download_dir=os.environ['NZBPP_DIRECTORY']
dest_dir=os.environ['NZBPO_DESTDIR']
extensions=os.environ['NZBPO_EXTENSIONS'].lower().split(',')
min_size=int(os.environ['NZBPO_MINSIZE'])
min_size <<= 10
overwrite=os.environ['NZBPO_OVERWRITE'] == 'yes'
cleanup=os.environ['NZBPO_CLEANUP'] == 'yes'
preview=os.environ['NZBPO_PREVIEW'] == 'yes'
verbose=os.environ['NZBPO_VERBOSE'] == 'yes'

if dest_dir == '':
    print('[WARNING] Option DestDir cannot be empty, exiting')
    sys.exit(POSTPROCESS_ERROR)

# Relative paths
if not (dest_dir[1] == '/' or dest_dir[1] == '\\' or (len(dest_dir) > 2) and dest_dir[2] == ':'):
    dest_dir = os.path.join(download_dir, dest_dir) 
dest_dir = os.path.abspath(dest_dir)
if verbose:
    print(('Normalized dest directory: %s' % dest_dir))

if preview:
    print('[WARNING] *** PREVIEW MODE ON - NO CHANGES TO FILE SYSTEM ***')

# List of moved files (source path)
moved_src_files = []

# List of moved files (destination path)
moved_dst_files = []

# Separator character used between file name and opening brace
# for duplicate files such as "My Movie (2).mkv"
dupe_separator = ' '

def guess_dupe_separator(filename):
    """ Find out a char most suitable as dupe_separator
    """ 
    global dupe_separator

    dupe_separator = ' '
    fname = os.path.splitext(filename)[0]

    if (fname.find('.') > -1):
        dupe_separator = '.'
        return

    if (fname.find('_') > -1):
        dupe_separator = '_'
        return

def unique_name(new):
    """ Adds unique numeric suffix to destination file name to avoid overwriting
        such as "filename.(2).ext", "filename.(3).ext", etc.
        If existing file was created by the script it is renamed to "filename.(1).ext".
    """
    fname, fext = os.path.splitext(new)
    suffix_num = 2
    while True:
        new_name = fname + dupe_separator + '(' + str(suffix_num) + ')' + fext
        if not os.path.exists(new_name) and new_name not in moved_dst_files:
            break
        suffix_num += 1
    return new_name

def optimized_move(old, new):
    try:
        os.rename(old, new)
    except OSError as ex:
        print(('[DETAIL] Rename failed ({}), performing copy: {}'.format(ex, new)))
        shutil.copyfile(old, new)
        os.remove(old)

def rename(old, new):
    """ Moves the file to its sorted location.
        It creates any necessary directories to place the new file and moves it.
    """
    if os.path.exists(new) or new in moved_dst_files:
        if overwrite and new not in moved_dst_files:
            os.remove(new)
            optimized_move(old, new)
            print(('[INFO] Overwrote: %s' % new))
        else:
            # rename to filename.(2).ext, filename.(3).ext, etc.
            new = unique_name(new)
            rename(old, new)
    else:
        if not preview:
            if not os.path.exists(os.path.dirname(new)):
                os.makedirs(os.path.dirname(new))
            optimized_move(old, new)
        print(('[INFO] Moved: %s' % new))
    moved_src_files.append(old)
    moved_dst_files.append(new)
    return new

def cleanup_download_dir():
    if verbose:
        print('Cleanup')

    # Now delete all files with nice logging                
    for root, dirs, files in os.walk(download_dir):
        for filename in files:
            path = os.path.join(root, filename)
            if not preview or path not in moved_src_files:
                if not preview:
                    os.remove(path)
                print(('[INFO] Deleted: %s' % path))
    if not preview:
        shutil.rmtree(download_dir)
    print(('[INFO] Deleted: %s' % download_dir))

def construct_path(filename):
    """ Parses the filename and generates new name for renaming """

    if verbose:
        print(("filename: %s" % filename))

    filename = os.path.basename(filename)

    if verbose:
        print(("basename: %s" % filename))

    # Find out a char most suitable as dupe_separator
    guess_dupe_separator(filename)

    new_path = os.path.join(dest_dir, filename)

    if verbose:
        print(('destination path: %s' % new_path))

    return new_path

# Flag indicating that anything was moved. Cleanup possible.
files_moved = False

# Flag indicating any error. Cleanup is disabled.
errors = False

# Process all the files in download_dir and its subdirectories
move_files = []

for root, dirs, files in os.walk(download_dir):
    for old_filename in files:
        try:
            if verbose:
                print(('[INFO] Processing: %s' % old_filename))

            old_path = os.path.join(root, old_filename)

            # Check extension
            if extensions != ['']:
                ext = os.path.splitext(old_filename)[1].lower()
                if ext not in extensions: continue

            # Check minimum file size
            if os.path.getsize(old_path) < min_size:
                print(('[INFO] Skipping small: %s' % old_filename))
                continue

            # This is our file, we should process it
            move_files.append(old_path)

        except Exception as e:
            errors = True
            print(('[ERROR] Failed: %s' % old_filename))
            print(('[ERROR] %s' % e))
            traceback.print_exc()

if verbose:
    print(('File list: %s' % move_files))

for old_path in move_files:
    try:
        new_path = construct_path(old_path)

        # Move file
        if new_path:
            rename(old_path, new_path)
            files_moved = True

    except Exception as e:
        errors = True
        print(('[ERROR] Failed: %s' % old_filename))
        print(('[ERROR] %s' % e))
        traceback.print_exc()

# Inform NZBGet about new destination path
finaldir = ''
uniquedirs = []
for filename in moved_dst_files:
    dir = os.path.dirname(filename)
    if dir not in uniquedirs:
        uniquedirs.append(dir)
        finaldir += '|' if finaldir != '' else ''
        finaldir += dir

if finaldir != '':
    print(('[NZB] FINALDIR=%s' % finaldir))

# Cleanup if:
# 1) files were moved AND
# 2) no errors happen
if cleanup and files_moved and not errors:
    cleanup_download_dir()

# Returing status to NZBGet
if errors:
    sys.exit(POSTPROCESS_ERROR)
elif files_moved:
    sys.exit(POSTPROCESS_SUCCESS)
else:
    sys.exit(POSTPROCESS_NONE)

[rEes9P]

Thanks a ton for all your help @clinton-hall! Very generous of you to find these resources for me. Everything now works fine except for FakeDetector and PasswordDetector. I am posting this here as the Issues page for both seem to be empty.

FakeDetector The post-processing of this script goes through and is marked as success but it throws up a bunch of error messages along the lines of FakeDetector: Failed Filename.part084.rar: can only concatenate str (not "bytes") to str

PasswordDetector I can't seem to get this to work (gets a Failure status) and am pasting relevant bits from the logs

ERROR   Thu May 19 2022 18:22:48    Post-process-script PasswordDetector/PasswordDetector.py for Filename.1080p failed (terminated with unknown status)
INFO    Thu May 19 2022 18:22:48    PasswordDetector: ModuleNotFoundError: No module named 'requests'
INFO    Thu May 19 2022 18:22:48    PasswordDetector: import requests
INFO    Thu May 19 2022 18:22:48    PasswordDetector: File "/Users/username/Library/Application Support/NZBGet/scripts/PasswordDetector/PasswordDetector.py", line 68, in <module>
INFO    Thu May 19 2022 18:22:48    PasswordDetector: Traceback (most recent call last):
INFO    Thu May 19 2022 18:22:48    Executing post-process-script PasswordDetector/PasswordDetector.py for Filename.1080p
INFO    Thu May 19 2022 18:22:44    PasswordDetector: ModuleNotFoundError: No module named 'requests'
INFO    Thu May 19 2022 18:22:44    PasswordDetector: import requests
INFO    Thu May 19 2022 18:22:44    PasswordDetector: File "/Users/username/Library/Application Support/NZBGet/scripts/PasswordDetector/PasswordDetector.py", line 68, in <module>
INFO    Thu May 19 2022 18:22:44    PasswordDetector: Traceback (most recent call last):
INFO    Thu May 19 2022 18:22:44    Executing queue-script PasswordDetector/PasswordDetector.py for Filename.1080p
INFO    Thu May 19 2022 18:22:44    PasswordDetector: ModuleNotFoundError: No module named 'requests'
INFO    Thu May 19 2022 18:22:44    PasswordDetector: import requests
INFO    Thu May 19 2022 18:22:44    PasswordDetector: File "/Users/username/Library/Application Support/NZBGet/scripts/PasswordDetector/PasswordDetector.py", line 68, in <module>
INFO    Thu May 19 2022 18:22:44    PasswordDetector: Traceback (most recent call last):
DETAIL  Thu May 19 2022 18:22:44    Executing queue-script PasswordDetector/PasswordDetector.py for Filename.1080p

[rr0ss0rr]

With PasswordDetector, Try installing "requests" pip3 install requests

[clinton-hall]

For Fake Detector, I have just made a small change to https://github.com/clinton-hall/FakeDetector.git that may resolve this... I think this may have something to do with encoding, and there may be more changes (sorry I'm really not in a position to test this just now) so I have forced the dir to be forced to string not bytes. Let me know if this makes any difference.

[rEes9P]

With PasswordDetector, Try installing "requests" pip3 install requests

Did as you suggested and PasswordDetector seems to work now! Thanks!

On an aside, interestingly I just noticed that PasswordDetector seems to run twice, once as a queue script and once as a post-process skip. I went back and looked at old logs (before upgrading to macOS Monterey and the resultant dropped support of Python2.7) and that's what the earlier logs show as well. Very interesting! I just noticed this. Relevant logs are below.

INFO    Fri May 20 2022 10:27:37    Post-process-script PasswordDetector/PasswordDetector.py for filename successful
DETAIL  Fri May 20 2022 10:27:37    PasswordDetector: Removing temp file /Users/username/Library/Application Support/NZBGet/tmp/PasswordDetector/22272
DETAIL  Fri May 20 2022 10:27:37    PasswordDetector: Detecting completed for filename
DETAIL  Fri May 20 2022 10:27:37    PasswordDetector: Detecting password for filename
INFO    Fri May 20 2022 10:27:37    Executing post-process-script PasswordDetector/PasswordDetector.py for filename
DETAIL  Fri May 20 2022 10:27:35    PasswordDetector: Detecting completed for filename
DETAIL  Fri May 20 2022 10:27:35    PasswordDetector: Detecting password for filename
INFO    Fri May 20 2022 10:27:35    Executing queue-script PasswordDetector/PasswordDetector.py for filename
DETAIL  Fri May 20 2022 10:27:34    PasswordDetector: Detecting completed for filename
DETAIL  Fri May 20 2022 10:27:34    PasswordDetector: Detecting password for filename
DETAIL  Fri May 20 2022 10:27:34    Executing queue-script PasswordDetector/PasswordDetector.py for filename
DETAIL  Fri May 20 2022 10:27:33    PasswordDetector: Detecting completed for filename
DETAIL  Fri May 20 2022 10:27:33    PasswordDetector: Detecting password for filename
DETAIL  Fri May 20 2022 10:27:33    Executing queue-script PasswordDetector/PasswordDetector.py for filename
DETAIL  Fri May 20 2022 10:27:32    PasswordDetector: Detecting completed for filename
DETAIL  Fri May 20 2022 10:27:32    PasswordDetector: Detecting password for filename
DETAIL  Fri May 20 2022 10:27:32    Executing queue-script PasswordDetector/PasswordDetector.py for filename
DETAIL  Fri May 20 2022 10:27:31    PasswordDetector: Detecting completed for filename
DETAIL  Fri May 20 2022 10:27:31    PasswordDetector: Detecting password for filename
DETAIL  Fri May 20 2022 10:27:31    Executing queue-script PasswordDetector/PasswordDetector.py for filename
DETAIL  Fri May 20 2022 10:27:30    PasswordDetector: Detecting completed for filename
DETAIL  Fri May 20 2022 10:27:30    PasswordDetector: Detecting password for filename
DETAIL  Fri May 20 2022 10:27:30    Executing queue-script PasswordDetector/PasswordDetector.py for filename
DETAIL  Fri May 20 2022 10:27:29    PasswordDetector: Detecting completed for filename
DETAIL  Fri May 20 2022 10:27:29    PasswordDetector: Detecting password for filename
DETAIL  Fri May 20 2022 10:27:29    Executing queue-script PasswordDetector/PasswordDetector.py for filename
DETAIL  Fri May 20 2022 10:27:28    PasswordDetector: Detecting completed for filename
DETAIL  Fri May 20 2022 10:27:28    PasswordDetector: Detecting password for filename
DETAIL  Fri May 20 2022 10:27:28    Executing queue-script PasswordDetector/PasswordDetector.py for filename
DETAIL  Fri May 20 2022 10:27:27    PasswordDetector: Detecting completed for filename
DETAIL  Fri May 20 2022 10:27:27    PasswordDetector: Detecting password for filename
DETAIL  Fri May 20 2022 10:27:27    Executing queue-script PasswordDetector/PasswordDetector.py for filename
DETAIL  Fri May 20 2022 10:27:26    PasswordDetector: Detecting completed for filename
DETAIL  Fri May 20 2022 10:27:26    PasswordDetector: Detecting password for filename
DETAIL  Fri May 20 2022 10:27:26    Executing queue-script PasswordDetector/PasswordDetector.py for filename
DETAIL  Fri May 20 2022 10:27:25    PasswordDetector: Detecting completed for filename
DETAIL  Fri May 20 2022 10:27:25    PasswordDetector: Detecting password for filename
DETAIL  Fri May 20 2022 10:27:25    Executing queue-script PasswordDetector/PasswordDetector.py for filename
DETAIL  Fri May 20 2022 10:27:24    PasswordDetector: Detecting completed for filename
DETAIL  Fri May 20 2022 10:27:24    PasswordDetector: Detecting password for filename
DETAIL  Fri May 20 2022 10:27:24    Executing queue-script PasswordDetector/PasswordDetector.py for filename
DETAIL  Fri May 20 2022 10:27:23    PasswordDetector: Detecting completed for filename
DETAIL  Fri May 20 2022 10:27:23    PasswordDetector: Detecting password for filename
DETAIL  Fri May 20 2022 10:27:23    Executing queue-script PasswordDetector/PasswordDetector.py for filename
DETAIL  Fri May 20 2022 10:27:22    PasswordDetector: Detecting completed for filename
DETAIL  Fri May 20 2022 10:27:22    PasswordDetector: Detecting password for filename
DETAIL  Fri May 20 2022 10:27:22    Executing queue-script PasswordDetector/PasswordDetector.py for filename
DETAIL  Fri May 20 2022 10:27:21    PasswordDetector: Detecting completed for filename
DETAIL  Fri May 20 2022 10:27:21    PasswordDetector: Created temp file /Users/username/Library/Application Support/NZBGet/tmp/PasswordDetector/22272
DETAIL  Fri May 20 2022 10:27:21    PasswordDetector: Detecting password for filename
DETAIL  Fri May 20 2022 10:27:21    Executing queue-script PasswordDetector/PasswordDetector.py for filename
INFO    Fri May 20 2022 10:27:17    PasswordDetector: Moving last rar-file to the top: filename.part57.rar
INFO    Fri May 20 2022 10:27:17    PasswordDetector: Sorting inner files for earlier fake detection for filename
INFO    Fri May 20 2022 10:27:17    Executing queue-script PasswordDetector/PasswordDetector.py for filename

[rEes9P]

For Fake Detector, I have just made a small change to https://github.com/clinton-hall/FakeDetector.git that may resolve this... I think this may have something to do with encoding, and there may be more changes (sorry I'm really not in a position to test this just now) so I have forced the dir to be forced to string not bytes. Let me know if this makes any difference.

FakeDetector still doesn't seem to work, @clinton-hall. The FakeDetector: Failed Filename.part084.rar: can only concatenate str (not "bytes") to str messages are no longer there but it still fails. Have made a gist of the pertinent logs so as to not make this post too long.

Many thanks again for being so generous with your time!

[clinton-hall]

Wow... sorry about this. Doing a bit of cleanup and I see I missed this! Is this still an issue? I have made a quick couple of changes to fake detector for Py3, but again, I don't really have anything to test this.