clinton-hall / nzbToMedia

Provides NZB and Torrent postprocessing To CouchPotatoServer, SickBeard/SickRage, HeadPhones, Mylar and Gamez
GNU General Public License v3.0
671 stars 176 forks source link

Returned log from SickBeard was not as expected.! #1158

Closed Nemesis001 closed 7 years ago

Nemesis001 commented 7 years ago

Trying to figure out where this is coming from. Most posts on the issue mention to make sure SickRage and nzbToMedia are up to date. Tried both with fork set to auto and sickrage. Nothing works.

Not much I can gather from the log:

2017-01-06 00:33:51 INFO    ::MAIN: Loading config from [C:\NZBtoMedia\nzbToMedia\autoProcessMedia.cfg]
2017-01-06 00:33:51 INFO    ::MAIN: Checking database structure...
2017-01-06 00:33:51 DEBUG   ::MAIN: Checking Initial Schema database upgrade
2017-01-06 00:33:51 DEBUG   ::MAIN: InitialSchema upgrade not required
2017-01-06 00:33:51 DEBUG   ::MAIN: Checking if we can use git commands: git version
2017-01-06 00:33:51 DEBUG   ::MAIN: Executing git version with your shell in C:\NZBtoMedia\nzbToMedia
2017-01-06 00:33:51 DEBUG   ::MAIN: Not using: git
2017-01-06 00:33:51 DEBUG   ::MAIN: Unable to find your git executable - Set git_path in your autoProcessMedia.cfg OR delete your .git folder and run from source to enable updates.
2017-01-06 00:33:51 DEBUG   ::MAIN: No git specified, can't use git commands
2017-01-06 00:33:51 INFO    ::MAIN: Checking if git needs an update
2017-01-06 00:33:51 DEBUG   ::MAIN: No git specified, can't use git commands
2017-01-06 00:33:51 ERROR   ::MAIN: Unable to determine installed version via git, please check your logs!
2017-01-06 00:33:51 INFO    ::MAIN: No update needed
2017-01-06 00:33:51 INFO    ::MAIN: nzbToMedia Version:11.02 Branch:master (Windows 10)
2017-01-06 00:33:51 WARNING ::MAIN: Failed to locate ffmpeg.exe. Transcoding disabled!
2017-01-06 00:33:51 WARNING ::MAIN: Install ffmpeg with x264 support to enable this feature  ...
2017-01-06 00:33:51 WARNING ::MAIN: Failed to locate ffprobe.exe. Video corruption detection disabled!
2017-01-06 00:33:51 WARNING ::MAIN: Install ffmpeg with x264 support to enable this feature  ...
2017-01-06 00:33:51 INFO    ::MAIN: #########################################################
2017-01-06 00:33:51 INFO    ::MAIN: ## ..::[nzbToMedia.pyc]::.. ##
2017-01-06 00:33:51 INFO    ::MAIN: #########################################################
2017-01-06 00:33:51 DEBUG   ::MAIN: Options passed into nzbToMedia: ['C:\\NZBtoMedia\\nzbToMedia\\nzbToSickBeard.py', '\\\\san\\media\\Multimedia\\New.Girl.S06E11.Raisins.Back.720p.WEB-DL.DD5.1.H.264-CLZ', 'New.Girl.S06E11.Raisins.Back.720p.WEB-DL.DD5.1.H.264-CLZ.nzb', 'New.Girl.S06E11.Raisins.Back.720p.WEB-DL.DD5.1.H.264-CLZ', '', 'tv', 'alt.binaries.tv', '0']
2017-01-06 00:33:51 INFO    ::MAIN: Script triggered from SABnzbd
2017-01-06 00:33:51 DEBUG   ::MAIN: Searching for nzoid from SAbnzbd ...
2017-01-06 00:33:53 DEBUG   ::MAIN: Found nzoid: SABnzbd_nzo_i7fgxz
2017-01-06 00:33:53 DEBUG   ::MAIN: Adding NZB download info for directory \\san\media\Multimedia\New.Girl.S06E11.Raisins.Back.720p.WEB-DL.DD5.1.H.264-CLZ to database
2017-01-06 00:33:53 INFO    ::MAIN: Auto-detected SECTION:SickBeard
2017-01-06 00:33:53 DEBUG   ::MAIN: Checking for archives to extract in directory: \\san\media\Multimedia\New.Girl.S06E11.Raisins.Back.720p.WEB-DL.DD5.1.H.264-CLZ
2017-01-06 00:33:53 INFO    ::MAIN: Calling SickBeard:tv to post-process:New.Girl.S06E11.Raisins.Back.720p.WEB-DL.DD5.1.H.264-CLZ.nzb
2017-01-06 00:33:53 DEBUG   ::SERVER: Attempting to connect to server at http://localhost:8081
2017-01-06 00:33:53 DEBUG   ::SERVER: Server responded at http://localhost:8081
2017-01-06 00:33:53 INFO    ::MAIN: SickBeard:tv fork set to sickrage
2017-01-06 00:33:53 INFO    ::MAIN: FLATTEN: Flattening directory: \\san\media\Multimedia\New.Girl.S06E11.Raisins.Back.720p.WEB-DL.DD5.1.H.264-CLZ
2017-01-06 00:33:53 DEBUG   ::MAIN: Checking for empty folders in:\\san\media\Multimedia\New.Girl.S06E11.Raisins.Back.720p.WEB-DL.DD5.1.H.264-CLZ
2017-01-06 00:33:53 POSTPROCESS::SICKBEARD: SUCCESS: The download succeeded, sending a post-process request
2017-01-06 00:33:53 DEBUG   ::SICKBEARD: Opening URL: http://localhost:8081/home/postprocess/processEpisode with params: {'nzbName': 'New.Girl.S06E11.Raisins.Back.720p.WEB-DL.DD5.1.H.264-CLZ.nzb', 'quiet': 1, 'failed': '0', 'proc_dir': '\\\\san\\media\\Multimedia\\New.Girl.S06E11.Raisins.Back.720p.WEB-DL.DD5.1.H.264-CLZ'}
2017-01-06 00:33:53 POSTPROCESS::SICKBEARD: Auto post processing task for \\san\media\Multimedia\New.Girl.S06E11.Raisins.Back.720p.WEB-DL.DD5.1.H.264-CLZ was added to the queue
2017-01-06 00:33:53 ERROR   ::MAIN: A problem was reported in the C:\NZBtoMedia\nzbToMedia\nzbToSickBeard.py script.

This is the configuration file:

# nzbToMedia Configuration
# For more information, visit https://github.com/clinton-hall/nzbToMedia/wiki

[General]
    # Enable/Disable update notifications
    version_notify = 1
    # Enable/Disable automatic updates
    auto_update = 0
    # Set to the full path to the git executable
    git_path = ""
    # GitHUB user for repo
    git_user = ""
    # GitHUB branch for repo
    git_branch = ""
    # Enable/Disable forceful cleaning of leftover files following postprocess 
    force_clean = 0
    # Enable/Disable logging debug messages to nzbtomedia.log
    log_debug = 1
    # Enable/Disable logging database messages to nzbtomedia.log
    log_db = 1
    # Enable/Disable logging environment variables to debug nzbtomedia.log (helpful to track down errors calling external tools.)
    log_env = 0
    # Enable/Disable logging git output to debug nzbtomedia.log (helpful to track down update failures.)
    log_git = 0
    # Set to the directory where your ffmpeg/ffprobe executables are located
    ffmpeg_path = ""
    # Enable/Disable media file checking using ffprobe.
    check_media = 1
    # Enable/Disable a safety check to ensure we don't process all downloads in the default_downloadDirectories by mistake.
    safe_mode = 1
    # Turn this on to disable additional extraction attempts for failed downloads. Default = 0 will attempt to extract and verify if media is present.
    no_extract_failed = 0

[Posix]
    ### Process priority setting for External commands (Extractor and Transcoder) on Posix (Unix/Linux/OSX) systems.
    # Set the Niceness value for the nice command. These range from -20 (most favorable to the process) to 19 (least favorable to the process).
    niceness = 0
    # Set the ionice scheduling class. 0 for none, 1 for real time, 2 for best-effort, 3 for idle.
    ionice_class = 0
    # Set the ionice scheduling class data. This defines the class data, if the class accepts an argument. For real time and best-effort, 0-7 is valid data.
    ionice_classdata = 0

[CouchPotato]
    #### autoProcessing for Movies
    #### movie - category that gets called for post-processing with CPS
    [[movie]]
        enabled = 0
        apikey = ""
        host = localhost
        port = 5050
        ###### ADVANCED USE - ONLY EDIT IF YOU KNOW WHAT YOU'RE DOING ######
        ssl = 0
        web_root = ""
        # Enable/Disable linking for Torrents
        Torrent_NoLink = 0
        keep_archive = 1
        method = renamer
        delete_failed = 0
        wait_for = 2
        extract = 1
        # Set this to minimum required size to consider a media file valid (in MB)
        minSize = 0
        # Enable/Disable deleting ignored files (samples and invalid media files)
        delete_ignored = 0
        ##### Enable if Couchpotato is on a remote server for this category
        remote_path = 0
        ##### Set to path where download client places completed downloads locally for this category
        watch_dir = ""
        ##### Set the recursive directory permissions to the following (0 to disable)
        chmodDirectory = 0

[SickBeard]
    #### autoProcessing for TV Series
    #### tv - category that gets called for post-processing with SB
    [[tv]]
        enabled = 1
        host = localhost
        port = 8081
        username = ""
        password = ""
        ###### ADVANCED USE - ONLY EDIT IF YOU KNOW WHAT YOU'RE DOING ######
        web_root = ""
        ssl = 0
        fork = sickrage
        delete_failed = 0
        # Enable/Disable linking for Torrents
        Torrent_NoLink = 0
        keep_archive = 1
        process_method = ""
        # force processing of already processed content when running a manual scan.
        force = 0
        # tell SickRage/Medusa to delete all source files after processing.
        delete_on = 0
        # tell Medusa to ignore check for associated subtitle check when postponing release
        ignore_subs = 0
        extract = 1
        nzbExtractionBy = Downloader
        # Set this to minimum required size to consider a media file valid (in MB)
        minSize = 0
        # Enable/Disable deleting ignored files (samples and invalid media files)
        delete_ignored = 0
        ##### Enable if SickBeard is on a remote server for this category
        remote_path = 0
        ##### Set to path where download client places completed downloads locally for this category
        watch_dir = ""
        ##### Set the recursive directory permissions to the following (0 to disable)
        chmodDirectory = 0

[NzbDrone]
    #### autoProcessing for TV Series
    #### ndCategory - category that gets called for post-processing with NzbDrone
    [[tv]]
        enabled = 0
        apikey = ""
        host = localhost
        port = 8989
        username = ""
        password = ""
        ###### ADVANCED USE - ONLY EDIT IF YOU KNOW WHAT YOU'RE DOING ######
        web_root = ""
        ssl = 0
        delete_failed = 0
        # Enable/Disable linking for Torrents
        Torrent_NoLink = 0
        keep_archive = 1
        extract = 1
        nzbExtractionBy = Downloader
        wait_for = 2
        # Set this to minimum required size to consider a media file valid (in MB)
        minSize = 0
        # Enable/Disable deleting ignored files (samples and invalid media files)
        delete_ignored = 0
        ##### Enable if NzbDrone is on a remote server for this category
        remote_path = 0
        ##### Set to path where download client places completed downloads locally for this category
        watch_dir = ""

[HeadPhones]
    #### autoProcessing for Music
    #### music - category that gets called for post-processing with HP
    [[music]]
        enabled = 0
        apikey = ""
        host = localhost
        port = 8181
        ###### ADVANCED USE - ONLY EDIT IF YOU KNOW WHAT YOU'RE DOING ######
        ssl = 0
        web_root = ""
        wait_for = 2
        # Enable/Disable linking for Torrents
        Torrent_NoLink = 0
        keep_archive = 1
        extract = 1
        # Set this to minimum required size to consider a media file valid (in MB)
        minSize = 0
        # Enable/Disable deleting ignored files (samples and invalid media files)
        delete_ignored = 0
        ##### Enable if HeadPhones is on a remote server for this category
        remote_path = 0
        ##### Set to path where download client places completed downloads locally for this category
        watch_dir = ""

[Mylar]
    #### autoProcessing for Comics
    #### comics - category that gets called for post-processing with Mylar
    [[comics]]
        enabled = 0
        host = localhost
        port = 8090
        username = ""
        password = ""
        ###### ADVANCED USE - ONLY EDIT IF YOU KNOW WHAT YOU'RE DOING ######
        web_root = ""
        ssl = 0
        # Enable/Disable linking for Torrents
        Torrent_NoLink = 0
        keep_archive = 1
        extract = 1
        # Set this to minimum required size to consider a media file valid (in MB)
        minSize = 0
        # Enable/Disable deleting ignored files (samples and invalid media files)
        delete_ignored = 0
        ##### Enable if Mylar is on a remote server for this category
        remote_path = 0
        ##### Set to path where download client places completed downloads locally for this category
        watch_dir = ""

[Gamez]
    #### autoProcessing for Games
    #### games - category that gets called for post-processing with Gamez
    [[games]]
        enabled = 0
        apikey = ""
        host = localhost
        port = 8085
        ###### 
        library = Set to path where you want the processed games to be moved to.
        ###### ADVANCED USE - ONLY EDIT IF YOU KNOW WHAT YOU'RE DOING ######
        ssl = 0
        web_root = ""
        # Enable/Disable linking for Torrents
        Torrent_NoLink = 0
        keep_archive = 1
        extract = 1
        # Set this to minimum required size to consider a media file valid (in MB)
        minSize = 0
        # Enable/Disable deleting ignored files (samples and invalid media files)
        delete_ignored = 0
        ##### Enable if Gamez is on a remote server for this category
        remote_path = 0
        ##### Set to path where download client places completed downloads locally for this category
        watch_dir = ""

[Network]
    # Enter Mount points as LocalPath,RemotePath and separate each pair with '|'
    # e.g. MountPoints = /volume1/Public/,E:\|/volume2/share/,\\NAS\
    mount_points = ""

[Nzb]
    ###### clientAgent - Supported clients: sabnzbd, nzbget
    clientAgent = sabnzbd
    ###### SabNZBD (You must edit this if your using nzbToMedia.py with SabNZBD)
    sabnzbd_host = http://localhost
    sabnzbd_port = 8082
    sabnzbd_apikey = ************************
    ###### Enter the default path to your default download directory (non-category downloads). this directory is protected by safe_mode.
    default_downloadDirectory = \\san\media\Multimedia

[Torrent]
    ###### clientAgent - Supported clients: utorrent, transmission, deluge, rtorrent, vuze, other
    clientAgent = other
    ###### useLink - Set to hard for physical links, sym for symbolic links, move to move, move-sym to move and link back, and no to not use links (copy)
    useLink = hard
    ###### outputDirectory - Default output directory (categories will be appended as sub directory to outputDirectory)
    outputDirectory = /abs/path/to/complete/
    ###### Enter the default path to your default download directory (non-category downloads). this directory is protected by safe_mode. 
    default_downloadDirectory = ""
    ###### Other categories/labels defined for your downloader. Does not include CouchPotato, SickBeard, HeadPhones, Mylar categories.
    categories = music_videos, pictures, software, manual
    ###### A list of categories that you don't want to be flattened (i.e preserve the directory structure when copying/linking.
    noFlatten = pictures, manual
    ###### uTorrent Hardlink solution (You must edit this if your using TorrentToMedia.py with uTorrent)
    uTorrentWEBui = http://localhost:8090/gui/
    uTorrentUSR = your username
    uTorrentPWD = your password
    ###### Transmission (You must edit this if your using TorrentToMedia.py with Transmission)
    TransmissionHost = localhost
    TransmissionPort = 9091
    TransmissionUSR = your username
    TransmissionPWD = your password
    #### Deluge (You must edit this if your using TorrentToMedia.py with deluge. Note that the host/port is for the deluge daemon, not the webui)
    DelugeHost = localhost
    DelugePort = 58846
    DelugeUSR = your username
    DelugePWD = your password
    ###### ADVANCED USE - ONLY EDIT IF YOU KNOW WHAT YOU'RE DOING ######
    deleteOriginal = 0
    chmodDirectory = 0
    resume = 1
    resumeOnFailure = 1

[Extensions]
    compressedExtensions = .zip, .rar, .7z, .gz, .bz, .tar, .arj, .1, .01, .001
    mediaExtensions = .mkv, .avi, .divx, .xvid, .mov, .wmv, .mp4, .mpg, .mpeg, .vob, .iso, .m4v, .ts
    audioExtensions = .mp3, .aac, .ogg, .ape, .m4a, .asf, .wma, .flac
    metaExtensions = .nfo, .sub, .srt, .jpg, .gif

[Plex]
    # Only enter these details if you want to update plex library after processing.
    # Do not enter these details if you send the plex notifications from Sickbeard/CouchPotato.
    plex_host = localhost
    plex_port = 32400
    plex_token = ""
    plex_ssl = 0
    # Enter Plex category to section mapping as Category,section and separate each pair with '|'
    # e.g. plex_sections = movie,3|tv,4
    plex_sections = ""

[Transcoder]
    # getsubs. enable to download subtitles.
    getSubs = 0
    # subLanguages. create a list of languages in the order you want them in your subtitles. 
    subLanguages = eng, spa, fra
    # transcode. enable to use transcoder
    transcode = 0
    ###### duplicate =1 will create a new file. =0 will replace the original
    duplicate = 1
    # concat. joins cd1 cd2 etc into a single video.
    concat = 1
    # IgnoreExtensions is a comma-separated list of extensions that will not be transcoded.
    ignoreExtensions = .avi, .mkv, .mp4
    # outputFastStart. 1 will use -movflags + faststart. 0 will disable this from being used.
    outputFastStart = 0
    # outputQualityPercent. used as -q:a value. 0 will disable this from being used.
    outputQualityPercent = 0
    # outputVideoPath. Set path you want transcoded videos moved to. Leave blank to disable.
    outputVideoPath = ""
    # processOutput. 1 will send the outputVideoPath to SickBeard/CouchPotato. 0 will send original files.
    processOutput = 0
    # audioLanguage. set the 3 letter language code you want as your primary audio track.
    audioLanguage = eng
    # allAudioLanguages. 1 will keep all audio tracks (uses AudioCodec3) where available.
    allAudioLanguages = 0
    # allSubLanguages. 1 will keep all existing sub languages. 0 will discard those not in your list above.
    allSubLanguages = 0
    # embedSubs. 1 will embed external sub/srt subs into your video if this is supported.
    embedSubs = 1
    # burnInSubtitle. burns the default sub language into your video (needed for players that don't support subs)
    burnInSubtitle = 0
    # extractSubs. 1 will extract subs from the video file and save these as external srt files.
    extractSubs = 0
    # externalSubDir. set the directory where subs should be saved (if not the same directory as the video)
    externalSubDir = ""
    # hwAccel. 1 will set ffmpeg to enable hardware acceleration (this requires a recent ffmpeg)
    hwAccel = 0
    # generalOptions. Enter your additional ffmpeg options here with commas to separate each option/value (i.e replace spaces with commas).
    generalOptions = ""
    # outputDefault. Loads default configs for the selected device. The remaining options below are ignored.
    # If you want to use your own profile, leave this blank and set the remaining options below.
    # outputDefault profiles allowed: iPad, iPad-1080p, iPad-720p, Apple-TV2, iPod, iPhone, PS3, xbox, Roku-1080p, Roku-720p, Roku-480p, mkv, mp4-scene-release
    outputDefault = ""
    #### Define custom settings below.
    outputVideoExtension = .mp4
    outputVideoCodec = libx264
    VideoCodecAllow = ""
    outputVideoPreset = medium
    outputVideoResolution = 1920:1080
    outputVideoFramerate = 24
    outputVideoBitrate = 800000
    outputVideoCRF = 19
    outputVideoLevel = 3.1
    outputAudioCodec = ac3
    AudioCodecAllow = ""
    outputAudioChannels = 6
    outputAudioBitrate = 640k
    outputAudioTrack2Codec = libfaac
    AudioCodec2Allow = ""
    outputAudioTrack2Channels = 2
    outputAudioTrack2Bitrate = 128000
    outputAudioOtherCodec = libmp3lame
    AudioOtherCodecAllow = ""
    outputAudioOtherChannels = ""
    outputAudioOtherBitrate = 128000
    outputSubtitleCodec = ""

[WakeOnLan]
    ###### set wake = 1 to send WOL broadcast to the mac and test the server (e.g. xbmc) the host and port specified.
    wake = 0
    host = 192.168.1.37
    port = 80
    mac = 00:01:2e:2D:64:e1

[UserScript]
    #Use user_script for uncategorized downloads
    #Set the categories to use external script.
    #Use "UNCAT" to process non-category downloads, and "ALL" for all defined categories.
    [[UNCAT]]
        #Enable/Disable this subsection category
        enabled = 0
        Torrent_NoLink = 0
        keep_archive = 1
        extract = 1
        #Enable if you are sending commands to a remote server for this category
        remote_path = 0
        #What extension do you want to process? Specify all the extension, or use "ALL" to process all files.
        user_script_mediaExtensions = .mkv, .avi, .divx, .xvid, .mov, .wmv, .mp4, .mpg, .mpeg
        #Specify the path to your custom script. Use "None" if you wish to link this category, but NOT run any external script.
        user_script_path = /nzbToMedia/userscripts/script.sh
        #Specify the argument(s) passed to script, comma separated in order.
        #for example FP,FN,DN, TN, TL for file path (absolute file name with path), file name, absolute directory name (with path), Torrent Name, Torrent Label/Category.
        #So the result is /media/test/script/script.sh FP FN DN TN TL. Add other arguments as needed eg -f, -r
        user_script_param = FN
        #Set user_script_runOnce = 0 to run for each file, or 1 to only run once (presumably on the entire directory).
        user_script_runOnce = 0
        #Specify the successcodes returned by the user script as a comma separated list. Linux default is 0
        user_script_successCodes = 0
        #Clean after? Note that delay function is used to prevent possible mistake :) Delay is intended as seconds
        user_script_clean = 1
        delay = 120
        #Unique path (directory) created for every download. set 0 to disable.
        unique_path = 1
        ##### Set to path where download client places completed downloads locally for this category
        watch_dir = ""

[ASCII]
    #Set convert =1 if you want to convert any "foreign" characters to ASCII (UTF8) before passing to SB/CP etc. Default is disabled (0).
    convert = 0

[passwords]
    # enter the full path to a text file containing passwords to be used for extraction attempts.
    # In the passwords file, every password should be on a new line
    PassWordFile = ""

[Custom]
    # enter a list (comma separated) of Group Tags you want removed from filenames to help with subtitle matching.
    # e.g remove_group = [rarbag],-NZBgeek
    # be careful if your "group" is a common "real" word. Please report if you have any group replacements that would fall in this category.
    remove_group = ""

Any suggestion where should I look further?

Thanks,

gerbil42 commented 7 years ago

SickRage added a post-processing queuing function with the latest git update, so I'm guessing the new return code for "added to the post-processing queue" isn't what nzbtomedia expects, and nzbtomedia needs an update?

Edit:
I looked at the log format before and after updates and I believe the updated SickRage logging format isn't currently matching nzbtomedia AutoProcessTV.py:

Old:

2017-01-02 17:05:41 INFO     Thread-27 :: Processing succeeded for <path>
2017-01-02 17:05:41 INFO     Thread-27 :: Deleting folder <path>
2017-01-02 17:05:41 INFO     Thread-27 :: Successfully processed

Matches this section from AutoProcessTV.py

        if section == "SickBeard":
            for line in r.iter_lines():
                if line: 
                    logger.postprocess("%s" % (line), section)
                    if "Moving file from" in line:
                        inputName = os.path.split(line)[1]
                    if "Processing succeeded" in line or "Successfully processed" in line:
                        Success = True

New (the lines no longer have a Thread-# prefix

2017-01-05 13:23:41 INFO     POSTPROCESSOR-AUTO :: Processing succeeded for <path>
2017-01-05 13:23:41 INFO     POSTPROCESSOR-AUTO :: Deleting folder (if it's empty): <path>
2017-01-05 13:23:41 INFO     POSTPROCESSOR-AUTO :: Successfully processed
2017-01-05 13:23:41 INFO     POSTPROCESSOR-AUTO :: Auto post processing task for <path> completed

I'm guessing the change from Thread-# to PostProcessor-Auto is the culprit here, because the order and phrasing of the rest of the log is the same.

clinton-hall commented 7 years ago

The Thread-# to PostProcessor-Auto isn't an issue. in both logs shown above both Processing succeeded and Successfully processed are returned.

The issue with the first log (in the original post) is that SickRage only returned Auto post processing task for \\san\media\Multimedia\New.Girl.S06E11.Raisins.Back.720p.WEB-DL.DD5.1.H.264-CLZ was added to the queue

So I am guessing, when the queue is empty, the returned logging is ok, but when there is more going on, and SickRage queues the process, this doesn't return anything useful.

The big problem is I can't just say it is good if SickRage says the process is queued, because there are other processes that follow, like resuming torrents or deleting, and I need to know SickRage has finished renaming etc first...

I am wondering if there needs to be an option to say "don't queue" when sending to SickRage, and just keep the connection open until it has finished...??

Which Branch/fork of SickRage is this on?

gerbil42 commented 7 years ago

https://github.com/SickRage/SickRage/commit/5cc4f54739ccd020c584367f9f07799cb827fc4a is the current commit my instance is on.

I don't see any General or Post-Processing setting to disable the new queue function.

It appears that even the first item to go into the empty queue uses the new logging line format:

2017-01-05 13:14:04 INFO     TORNADO :: User logged into the SickRage web interface
2017-01-05 13:14:07 INFO     TORNADO :: User logged into the SickRage web interface
2017-01-05 13:14:08 INFO     Thread-29 :: Auto post processing task for D:\sabnzbd\Downloads\complete\tv\The Expanse S01E01 720p HDTV x264-BATV was added to the queue
2017-01-05 13:14:08 INFO     POSTPROCESSOR-AUTO :: Beginning auto post processing task: D:\sabnzbd\Downloads\complete\tv\The Expanse S01E01 720p HDTV x264-BATV
2017-01-05 13:14:08 INFO     POSTPROCESSOR-AUTO :: PostProcessing Path: D:\sabnzbd\Downloads\complete\tv
2017-01-05 13:14:08 INFO     POSTPROCESSOR-AUTO :: Processing D:\sabnzbd\Downloads\complete\tv\The Expanse S01E01 720p HDTV x264-BATV\The.Expanse.S01E01.720p.HDTV.x264-BATV.mkv (The.Expanse.S01E01.720p.HDTV.x264-BATV.nzb)
2017-01-05 13:14:08 INFO     POSTPROCESSOR-AUTO :: This download is marked a priority download so I'm going to replace an existing file if I find one
2017-01-05 13:14:08 INFO     POSTPROCESSOR-AUTO :: Process method is 'move' and src and destination are on the same device, skipping free space check
2017-01-05 13:14:08 INFO     POSTPROCESSOR-AUTO :: Using nzb_name for release name.
2017-01-05 13:14:09 INFO     POSTPROCESSOR-AUTO :: 280619: Changed modify date of 'The Expanse - S01E01 - 720p HDTV.mkv' to show air date Dec 14,2015 (22:00)
2017-01-05 13:14:09 INFO     POSTPROCESSOR-AUTO :: Processing succeeded for D:\sabnzbd\Downloads\complete\tv\The Expanse S01E01 720p HDTV x264-BATV\The.Expanse.S01E01.720p.HDTV.x264-BATV.mkv
2017-01-05 13:14:09 INFO     POSTPROCESSOR-AUTO :: Deleting folder (if it's empty): D:\sabnzbd\Downloads\complete\tv\The Expanse S01E01 720p HDTV x264-BATV
2017-01-05 13:14:09 INFO     POSTPROCESSOR-AUTO :: Successfully processed
2017-01-05 13:14:09 INFO     POSTPROCESSOR-AUTO :: Auto post processing task for D:\sabnzbd\Downloads\complete\tv\The Expanse S01E01 720p HDTV x264-BATV completed
mannibis commented 7 years ago

Yea, I just ran into this issue as well and was about to post an issue about it. The PP is fine, it's just the return code is PP-Failure in NZBGet. I hope it gets sorted out with the SickRage guys. Thanks for all your work Clinton.

aeonsablaze commented 7 years ago

Same issue over here. Possible (maybe bad) idea for a workaround if Sickrage doesn't fix this. Could the script submit the task to the queue, and then watch the SR logs until the appropriate "Processing Succeeded" line shows up?

drspangle commented 7 years ago

The issue has been temporarily fixed in the latest commit of SickRage: https://github.com/SickRage/SickRage/commit/203d7b4ce1f0020bca2da75a796c8510c6a79b07

However, I've observed that this has caused a race condition when force_clean=1 where the files that are queued for postprocessing are deleted by nzbToMedia before SickRage gets a chance to process them. Setting force_clean=0 works, but leaves behind a pile of cruft that needs to be manually deleted, since SABnzbd and other downloaders won't delete leftover files if the post-download script executes successfully (errorcode 0).

Something to keep in mind for the hopefully forthcoming patch.

Update: Seems that failed download handling has been totally broken as well. This is probably more of an issue for the SickRage developers, but in either case, failed nzbs in SABnzbd are not being added to the failed downloads table, nor are they marked as such in the history.

Nemesis001 commented 7 years ago

I can confirm as well.

miigotu commented 7 years ago

This is due to the new post processing queue in SickRage, which solves a whole slew of other issues in SickRage that are caused when auto/manual/api post processing is called on top of each other or more than once each. Now, instead of returning the log of the task you started it can only return whether the task was added to the queue or not, and the task will run as soon as the previous task completes.

The string nzbtomedia checks for on success I have added as a hidden string in the html so the call from nzbtomedia succeeds, but nzbtomedia should not be deleting any files. I have fixed that problem in the post processor on windows where users had to use copy instead of move to get the correct permissions, which left files behind after the copy ofcourse. They can now check the icacls option in the post processor and use the move method now and the files will be deleted as they should be automatically.

Make sure force_clean=0 in nzbtomedia, and wait for the task to complete.

gerbil42 commented 7 years ago

What's the proper guidance now, for interaction between sab, nzbtomedia, and sickrage?
Should the box at the top of SickRage Post-Processing be checked or not? When it wasn't checked, nzbtomedia script reports success, and sickrage reports that the folder/file is queued, but nothing happens. Now with force_clean=0, the file isn't deleted by nzbtomedia, but it isn't moved either. I tried kicking things lose with Manual Post-Processing and all that did was report "yes, this has been queued".

It wasn't until I checked the box at the top of the Post Processing settings (which the page tells you not to do if using an external script (nzbtomedia) that things sorta work. But I'm also getting a lot of failures that report that the file is locked for reading/writing:

2017-01-08 21:32:14 INFO     TORNADO :: User logged into the SickRage web interface
2017-01-08 21:32:15 INFO     Thread-25 :: Auto post processing task for D:\sabnzbd\Downloads\complete\tv\NCIS Los Angeles S08E12 720p HDTV X264-DIMENSION was added to the queue
2017-01-08 21:32:16 INFO     POSTPROCESSOR-AUTO :: Beginning auto post processing task: D:\sabnzbd\Downloads\complete\tv\NCIS Los Angeles S08E12 720p HDTV X264-DIMENSION
2017-01-08 21:32:16 INFO     POSTPROCESSOR-AUTO :: PostProcessing Path: D:\sabnzbd\Downloads\complete\tv
2017-01-08 21:32:16 INFO     POSTPROCESSOR-AUTO :: Processing D:\sabnzbd\Downloads\complete\tv\NCIS Los Angeles S08E12 720p HDTV X264-DIMENSION\NCIS.Los.Angeles.S08E12.720p.HDTV.X264-DIMENSION.mkv (NCIS.Los.Angeles.S08E12.720p.HDTV.X264-DIMENSION.nzb)
2017-01-08 21:32:16 INFO     POSTPROCESSOR-AUTO :: This download is marked a priority download so I'm going to replace an existing file if I find one
2017-01-08 21:32:16 INFO     POSTPROCESSOR-AUTO :: Process method is 'move' and src and destination are on the same device, skipping free space check
2017-01-08 21:32:16 INFO     POSTPROCESSOR-AUTO :: Using nzb_name for release name.
2017-01-08 21:32:17 WARNING  POSTPROCESSOR-AUTO :: Processing failed for D:\sabnzbd\Downloads\complete\tv\NCIS Los Angeles S08E12 720p HDTV X264-DIMENSION\NCIS.Los.Angeles.S08E12.720p.HDTV.X264-DIMENSION.mkv: File is locked for reading/writing
2017-01-08 21:32:17 WARNING  POSTPROCESSOR-AUTO :: Problem(s) during processing, failed the following files/folders:  
2017-01-08 21:32:17 WARNING  POSTPROCESSOR-AUTO :: [D:\sabnzbd\Downloads\complete\tv\NCIS Los Angeles S08E12 720p HDTV X264-DIMENSION\NCIS.Los.Angeles.S08E12.720p.HDTV.X264-DIMENSION.mkv : Processing failed: File is locked for reading/writing]
2017-01-08 21:32:17 INFO     POSTPROCESSOR-AUTO :: Auto post processing task for D:\sabnzbd\Downloads\complete\tv\NCIS Los Angeles S08E12 720p HDTV X264-DIMENSION completed

Hmm, it looks like nzbtomedia is still firing CLEANDIRS, even with force_clean=0 set.

2017-01-08 21:32:08 INFO    ::MAIN: Script triggered from SABnzbd 0.7.17+
2017-01-08 21:32:10 INFO    ::MAIN: Auto-detected SECTION:SickBeard
2017-01-08 21:32:10 INFO    ::MAIN: Calling SickBeard:tv to post-process:NCIS.Los.Angeles.S08E12.720p.HDTV.X264-DIMENSION.nzb
2017-01-08 21:32:11 INFO    ::MAIN: Attempting to auto-detect tv fork
2017-01-08 21:32:13 INFO    ::MAIN: SickBeard:tv fork auto-detection successful ...
2017-01-08 21:32:13 INFO    ::MAIN: SickBeard:tv fork set to sickragetv
2017-01-08 21:32:13 INFO    ::MAIN: FLATTEN: Flattening directory: D:\sabnzbd\Downloads\complete\tv\NCIS Los Angeles S08E12 720p HDTV X264-DIMENSION
2017-01-08 21:32:13 POSTPROCESS::SICKBEARD: SUCCESS: The download succeeded, sending a post-process request
2017-01-08 21:32:15 POSTPROCESS::SICKBEARD: Auto post processing task for D:\sabnzbd\Downloads\complete\tv\NCIS Los Angeles S08E12 720p HDTV X264-DIMENSION was added to the queue
2017-01-08 21:32:15 POSTPROCESS::SICKBEARD: <br\><span class='hidden'>Processing succeeded</span>
2017-01-08 21:32:16 INFO    ::CLEANDIRS: Directory D:\sabnzbd\Downloads\complete\tv\NCIS Los Angeles S08E12 720p HDTV X264-DIMENSION has been processed, removing ...
2017-01-08 21:32:17 INFO    ::MAIN: The D:\SickRage\contrib\nzbToMedia\nzbToSickBeard.py script completed successfully.
clinton-hall commented 7 years ago

Cleandirs is always triggered, but no media files are deleted unless force_clean = 1

The issues I need to work through here are how to handle the new queue feature with torrents etc. The problem here is that I stop the torrent client from seeding to avoid any file locks, but once successful rename by SickBeard/SickRage/CouchPotato etc it then resumes seeding, or deletes the torrent.

In this case, when queued, we can't resume seeding or remove the torrent until we have verified that the rename event has been completed. Also, when using move-sym the original file is moved and a symlink created to point to that location. now, once the rename has completed, the script will transverse the links and make the original torrent location link direct to the end library file... again, this all needs to be done when the rename event has completed.

I'll try and have a solution to this shortly.

gerbil42 commented 7 years ago

So do you believe that it's not NZBtoMedia causing the SickRage processing failure where it's reporting that the "file is locked for reading/writing"?

Or is it your belief that it is, and you're having to fix the NZBtoMedia script to pause until the true end of the PP auto-process queue completion and the move/rename has truly finished from the SickRage side?

clinton-hall commented 7 years ago

I believe in this case it does appear that nzbToMedia has deleted the file before it was processed. I am looking to see ho cleandirs is functioning, as I didn't think this deleted media files... but it appears it did.

I am also discussing this with miigotu to determine how to best handle this

gerbil42 commented 7 years ago

Great. Standing by!

heymoe commented 7 years ago

Confirmed that the latest SR + nightly nzbToMedia has things back to normal.

drspangle commented 7 years ago

heymoe, have you tested whether we can safely set force_clean = 1 without any problems?

iernie commented 7 years ago

@drspangle I have force_clean = 1 with latest SR and nightly nzbToMedia and it works for me. Haven't experienced anything else at least.

heymoe commented 7 years ago

Yep, set force_clean back to 1 and all is working like it did before.