clinton-hall / nzbToMedia

Provides NZB and Torrent postprocessing To CouchPotatoServer, SickBeard/SickRage, HeadPhones, Mylar and Gamez
GNU General Public License v3.0
673 stars 176 forks source link

Latest Dev | Sickbeard Torrent Hard Link #314

Closed ghost closed 10 years ago

ghost commented 10 years ago

Previously, prior to all the latest code changes; TorrentToMedia would hardlink from the seeding directory to a process directory; from which SickBeard would then process, unrar, move, etc.

With latest updates, TorrentToMedia ignores any hard linking and sends the seed directory, original, to SickBeard.

Is this by design? I have SickBeard performing unrar (I've found it more capable), so I need it to perform in a processing directory and leave seeding alone. The original working order, TorrentToMedia doing a hardlink, then SickBeard set to move, was working perfectly.

Any advice?

        fork = auto
        delete_failed = 1
        nzbExtractionBy = Destination
        Torrent_NoLink = 0
        process_method = 
clinton-hall commented 10 years ago

Ah... https://github.com/clinton-hall/nzbToMedia/commit/7d4ccf53cc93fbae84f5dc0222c2bb3318d1cee4

This was not by design... but I believe a minor bug.

So lines 51-57 of TorrentToMedia need to be replaced with

if config.issubsection(inputCategory,["SickBeard"):
    fork, fork_params = autoFork("SickBeard", inputCategory)
        Torrent_NoLink = int(config()["SickBeard"][inputCategory]["Torrent_NoLink"])  # 0
         if fork in config.SICKBEARD_TORRENT and Torrent_NoLink == 1:
             Logger.info("MAIN: Calling autoProcessTV to post-process: %s",inputName)
             result = autoProcessTV().processEpisode(inputDirectory, inputName, 0, clientAgent=clientAgent, inputCategory=inputCategory)
             if result != 0:
                 Logger.info("MAIN: A problem was reported in the autoProcessTV script.")
                 Logger.info("MAIN: All done.")
                 sys.exit()

Then, to link all archive files and not extract them, we need to change line 192-208

        elif fileExtension in compressedContainer:
            if config().issubsection(inputCategory,['SickBeard']) and config()["SickBeard"][inputCategory]["nzbExtractionBy"] == "Destination":
                Logger.info("MAIN: Found archive file %s in %s", fileExtension, filePath)
                try:
                    copy_link(filePath, targetDirectory, useLink, outputDestination)
                    copy_list.append([filePath, os.path.join(outputDestination, file)])
                except:
                    Logger.exception("MAIN: Failed to link file: %s", file)
                continue
            # find part numbers in second "extension" from right, if we have more than 1 compressed file in the same directory.
            if re.search(r'\d+', os.path.splitext(fileName)[1]) and os.path.dirname(filePath) in extracted_folder and not any(item in os.path.splitext(fileName)[1] for item in ['.720p','.1080p','.x264']):
                part = int(re.search(r'\d+', os.path.splitext(fileName)[1]).group())
                if part == 1: # we only want to extract the primary part.
                    Logger.debug("MAIN: Found primary part of a multi-part archive %s. Extracting", file)
                else:
                    Logger.debug("MAIN: Found part %s of a multi-part archive %s. Ignoring", part, file)
                    continue
            Logger.info("MAIN: Found compressed archive %s for file %s", fileExtension, filePath)
            try:
                extractor.extract(filePath, outputDestination)
                extractionSuccess = True # we use this variable to determine if we need to pause a torrent or not in uTorrent (don't need to pause archived content)
                extracted_folder.append(os.path.dirname(filePath))
            except:
                Logger.exception("MAIN: Extraction failed for: %s", file)
            continue

Now the question is, should this just be

[SickBeard]
    [[inputCategory]]
        ExtractionBy = Destination

(note, ExtractionBy, not nzbExtractionBy). Or if we should have a separate nzbExtractionBy and TorrentExtractionBy...

This idea of allow extraction by Destination (SickBeard, CouchPotato etc), Downlaoder (SABnzbd, NZBGet, transmission etc) or nzbToMedia was a work in progress before all the recent changes, so i had forgotten about the specific scenario you use (use hardlink to send ALL files to SickBeard and then have SickBeard do the extraction)... So the question really is, do you download NZBs and if so where do you do the extraction?

@echel0n Really appreciate your feedback on this one. Not sure if you had any ideas towards this, or if you had even considered this "flow"... I certainly hadn't considered this prior to it being requested ;)

ghost commented 10 years ago

I do download Nzb, and I let SB do extraction instead of SAB. SAB downloads to the same /process directory, where hardlink'd torrents should go. I also define that /process as sickbeard postprocess directory.

Torrents get hard-linked to /process/sickbeard. SAB downloads "sickbeard" category to same, /process/sickbeard; and Sickbeard expects post-processing in that same directory for everything.

So, for torrents, hard-linking to destination works perfectly. SAB, nothing needs to be done, except abiding by ExtractionBy.

Thanks for your analysis on this one.

ghost commented 10 years ago

Thanks for update. Switch to the dev-CH branch.

However, still not hard linking. Processing, but not hard linking and blowing away my seeds. SickBeard was set to move. Just set to copy while we figure this out. :-)

    [[sickbeard]]
        enabled = 1
        host = 127.0.0.1
        port = 8081
        username = ""
        password = ""
        web_root = /sickbeard
        ssl = 0
        delay = 0
        TimePerGiB = 60
        watch_dir = ""
        fork = auto
        delete_failed = 1
        nzbExtractionBy = Destination
        Torrent_NoLink = 0
        process_method = ""
19:35:23|INFO    TorrentToMedia V9.3
19:35:23|INFO    MAIN: Loading config from /home/media/.scripts/nzbToMedia/autoProcessMedia.cfg
19:35:23|DEBUG   arg 0 is: /home/media/.scripts/nzbToMedia/TorrentToMedia.py
19:35:23|DEBUG   arg 1 is: /cache/torrent/download/sickbeard/TV.Show
19:35:23|DEBUG   arg 2 is: TV.Show
19:35:23|DEBUG   arg 3 is: sickbeard
19:35:23|DEBUG   arg 4 is: 123456789
19:35:23|DEBUG   MAIN: Received Directory: /cache/torrent/download/sickbeard/TV.Show | Name: TV.Show | Category: sickbeard
19:35:23|INFO    SEARCH: Files appear to be in their own directory
19:35:23|DEBUG   MAIN: Determined Directory: /cache/torrent/download/sickbeard/TV.Show | Name: TV.Show | Category: sickbeard
19:35:23|INFO    MAIN: Calling autoProcessTV to post-process: TV.Show
19:35:23|INFO    Attempting to auto-detect SickBeard fork
19:35:23|INFO    SickBeard:sickbeard fork auto-detection successful ...
19:35:23|INFO    SickBeard:sickbeard fork set to failed-torrent
19:35:23|INFO    Loading config from /home/media/.scripts/nzbToMedia/autoProcessMedia.cfg
19:35:23|INFO    The download succeeded. Sending process request to SickBeard
19:35:23|INFO    Waiting for 0.0 seconds to allow SickBeard to process newly extracted files
19:35:23|DEBUG   Opening URL: http://127.0.0.1:8081/sickbeard/home/postprocess/processEpisode?nzbName=TV.Show&failed=0&quiet=1&dir=%2Fcache%2Ftorrent%2Fdownload%2Fsickbeard%2FTV.Show
19:35:59|INFO    Processing folder /cache/torrent/download/sickbeard/TV.Show
19:35:59|INFO    TV_DOWNLOAD_DIR: /cache/process/sickbeard
19:35:59|INFO    PostProcessing Path: /cache/torrent/download/sickbeard
19:35:59|INFO    PostProcessing Dirs: [u'TV.Show']
19:35:59|INFO    PostProcessing Files: []
19:35:59|INFO    PostProcessing VideoFiles: []
19:35:59|INFO    PostProcessing RarContent: []
19:35:59|INFO    PostProcessing VideoInRar: []
19:35:59|INFO    Processing folder TV.Show
19:35:59|INFO    Processing /cache/torrent/download/sickbeard/TV.Show/TV.Show.mkv (TV.Show)
19:35:59|INFO    Parsed TV.Show into TV Show - S4E1 - 1080p.WEBRip.DD5.1.x264 (ITSat) [ABD: False]
19:35:59|INFO    Checking scene exceptions for a match on TV Show
19:35:59|INFO    Looking up TV Show in the DB
19:35:59|INFO    Lookup successful, using tvdb id 121999
19:35:59|INFO    Loading show object for tvdb_id 121999
19:35:59|INFO    Retrieving episode object for 4x1
19:35:59|INFO    The old status had a quality in it, using that: 1080p WEB-DL
19:35:59|INFO    SB snatched this episode so I'm marking it as priority
19:35:59|INFO    Is ep a priority download: True
19:35:59|INFO    File /storage/Media/TV Series/TV Show/Season 04/TV Show - S04E01 - Name - 1080p HD TV.mkv is larger than /cache/torrent/download/sickbeard/TV.Show/TV.Show.mkv
19:35:59|INFO    This download is marked a priority download so I'm going to replace an existing file if I find one
19:35:59|INFO    Deleting file /storage/Media/TV Series/TV Show/Season 04/TV Show - S04E01 - Name - 1080p HD TV.mkv
19:35:59|INFO    Deleting file /storage/Media/TV Series/TV Show/Season 04/TV Show - S04E01 - Name - 1080p HD TV.en.srt
19:35:59|INFO    Deleting file /storage/Media/TV Series/TV Show/Season 04/TV Show - S04E01 - Name - 1080p HD TV.pt.srt
19:35:59|INFO    Found release name TV.Show
19:35:59|INFO    Destination folder for this episode: /storage/Media/TV Series/TV Show/Season 04
19:35:59|INFO    Moving file from /cache/torrent/download/sickbeard/TV.Show/TV.Show.mkv to /storage/Media/TV Series/TV Show/Season 04/TV Show - S04E01 - Name - 1080p WEB-DL.mkv
19:35:59|INFO    Processing succeeded for /cache/torrent/download/sickbeard/TV.Show/TV.Show.mkv
19:35:59|INFO    Deleting folder /cache/torrent/download/sickbeard/TV.Show
19:35:59|INFO    MAIN: All done.
clinton-hall commented 10 years ago

the only way this particular sequence of logging can happen is

if fork in config.SICKBEARD_TORRENT and Torrent_NoLink == 1:

and we know SickBeard:sickbeard fork set to failed-torrent... so the first condition is clearly met.

So, can you double check the .cfg file to ensure

[SickBeard]
    [[sickbeard]]
        Torrent_NoLink = 0

Can you also pull down the latest changes and verify that Lines 58-59 of TorrentToMedia.py read as follows:

    Torrent_NoLink = int(config()["SickBeard"][inputCategory]["Torrent_NoLink"])  # 0
    if fork in config.SICKBEARD_TORRENT and Torrent_NoLink == 1:
clinton-hall commented 10 years ago

Maybe even purge the .pyc files just to be safe?

echel0n commented 10 years ago

actually I moved those lines into autoprocesstv.py

clinton-hall commented 10 years ago

wow... with double negatives?

So the logic here is that if we are on a non "torrent" fork of SickBeard (e.g default) and we have Torrent_NoLink = 0 (default) and we are using one of the torrent Clients, then we just return without doing anything?

What I was originally trying to achieve is that we bypass all the other linking/extracting and jump right into autoProcessTV when fork in config.SICKBEARD_TORRENT and Torrent_NoLink == 1 if neither/none of these are met, then we do the linking/extraction adn then hand over to autoProcessTV...

clinton-hall commented 10 years ago

what really surprised me here is that the logs show the autodetection after calling autoProcessTV... yet the autofork call is made before this line is logged.

echel0n commented 10 years ago

Same logic applies its just moved into autoprocesstv but at the top and checks for the same as it did before, I did this cause we do our auto-section detection in the autoprocess* files and I didn't want to add a redundant one to torrenttomedia specially since we now have nzbdrone added.

echel0n commented 10 years ago

oh and auto-fork detection is called now only in autoprocesstv and returns default if it fails detection so thats why your seeing it.

clinton-hall commented 10 years ago

@SevenNine

Can you confirm that you are on the dev-CH branch? the logs you have shown still appear to follow the process of dev branch.

clinton-hall commented 10 years ago

@echel0n

I agree with removing redundant section, but the problem is this check needs to be done before calling autoProcessTV.processepisode since it changes the behavior of the TorrentToMedia script.

The way this is specified (as I read it), if you are on a default fork and have Torrent_NoLink =0, then when called from TorrentToMedia, then autoProcessTV will just return 1 (failure) and then TorrentToMedia will exit.

echel0n commented 10 years ago

Thats correct. THe code can be anywhere it doesn't matter so long as it returns and there is a check in torrenttomedia to acknowledge said return and do what it needs to depending on that return, all this has done is moved the code from one spot to another so the outcome should be the same and if it isn't its just because the comparison checks in both files are incorrect somewhere but placement would not factor into causing this issue.

clinton-hall commented 10 years ago

but the problem was, when we did the checks in autoProcessMedia, the script is terminated after this check... and ultimately we do want to process this media (if these conditions aren't met) once the files have been linked etc. So we would need to pass something in differently.

At this time, we only care if we are in SickBeard category and we have Torrent_NoLink = 1. So I was trying to make sure we checked SickBeard could handle linking and extracting of files etc. Since it makes sense to put this check in autoProcessTV and only do it once, I will remove it from TorrentToMedia and assume the use only specifies Torrent_NoLink for a SickBeard fork that cahn handle this.

echel0n commented 10 years ago

I'll look over the code again and see if I missed something cause it should operate the way you intended to as I didn't want to mess with something you designed and change its flow.

clinton-hall commented 10 years ago

So if Torren_NoLink = 0 it now goes through the whole process, links files, extracts etc, then calls autoProcessTV.

If the category is for SickBeard, and that SickBeard sub-category has Torrent_NoLink = 1, we skip the linking/extracting and just jump straight to autoProcessTV

echel0n commented 10 years ago

Ok paste me the code as it used to be cause I think the premise of how its operating is wrong perhaps cause all I did was re-write it into autoprocesstv the way it was intended to flow .... then again I did have a all nighter once so anything is possible lol

clinton-hall commented 10 years ago

What it used to do is pretty much what happens in master. dev is pretty much what you have implemented (which is pretty awesome... but just causes this issue with SickBeard and NZBDrone skipping the linking/extracting, then returning and exiting if not a SickBeard-Torrent fork and not Torrent_NoLink = 1). dev-CH is my proposed fix.

But looking at dev, this is what happens

if config().issubsection(inputCategory,["SickBeard","NzbDrone"]):
    Logger.info("MAIN: Calling autoProcessTV to post-process: %s",inputName)
    result = autoProcessTV().processEpisode(inputDirectory, inputName, 0, clientAgent=clientAgent, inputCategory=inputCategory)
    if result != 0:
        Logger.info("MAIN: A problem was reported in the autoProcessTV script.")
    Logger.info("MAIN: All done.")
    sys.exit()

and at the start of autoProcessTV

   if not fork in config.SICKBEARD_TORRENT and not Torrent_NoLink == 1:
        if clientAgent in ['utorrent', 'transmission', 'deluge']:
            return 1
clinton-hall commented 10 years ago

What I am now doing instead:

if config.issubsection(inputCategory,["SickBeard"]):
    Torrent_NoLink = int(config()["SickBeard"][inputCategory]["Torrent_NoLink"])  # 0
    if Torrent_NoLink == 1:
        Logger.info("MAIN: Calling autoProcessTV to post-process: %s",inputName)
        result = autoProcessTV().processEpisode(inputDirectory, inputName, 0, clientAgent=clientAgent, inputCategory=inputCategory)
        if result != 0:
            Logger.info("MAIN: A problem was reported in the autoProcessTV script.")
        resume_torrent(clientAgent, TorrentClass, inputHash, inputID, result, inputName)
        cleanup_output(inputCategory, processCategories, result, outputDestination)
        Logger.info("MAIN: All done.")
        sys.exit()

And removing the section (from the previous post) from autoProcessTV

ghost commented 10 years ago

confirmed on dev-CH

user@ubuntu:~/.scripts/nzbToMedia$ git checkout dev-CH
Already on 'dev-CH'
clinton-hall commented 10 years ago

Ok, can you pull the latest change, purge .pyc files and try again? The previous log sequence just didn't line up with the changes i have pushed, but in anycase, I have made more since then.

echel0n commented 10 years ago

Oh damn I see what I did wrong ! I have it exiting everytime lol omg I completely missed that!

echel0n commented 10 years ago

I should have seperated some of the code and done multi conditional checks, ok maybe a bit more coffee lol

clinton-hall commented 10 years ago

Ok... I'm not completely loosing the plot over here ;)

to do it that way, i am thinking we would need to pass something in differently the first time, so that if it does exit we would continue to link, extract, then we would pass something different in to autoProcessTV so that we do run the whole way through...

clinton-hall commented 10 years ago

The latest changes I have made should be ok... in essence, the only people who set Torrent_NoLink = 1 should only do this for a "failed-torrent" sickbeard fork....

clinton-hall commented 10 years ago

and to be fair.... my "flow" doesn't exactly flow, so easy to miss that :dancer:

ghost commented 10 years ago

cleaned and cloned again.

00:08:14|INFO    ====================
00:08:14|INFO    TorrentToMedia V9.3
00:08:14|INFO    MAIN: Loading config from /home/media/.scripts/nzbToMedia/autoProcessMedia.cfg
00:08:14|INFO    SEARCH: Determined Category to be: sickbeard
00:08:14|INFO    SEARCH: Found torrent name: TV.Show
00:08:14|INFO    SEARCH: Found torrent directory /cache/torrent/download/sickbeard/TV.Show in category directory /cache/torrent/download/sickbeard
00:08:19|INFO    MAIN: Output directory set to: /cache/process/sickbeard/TV.Show
Traceback (most recent call last):
  File "./TorrentToMedia.py", line 443, in <module>
    main(inputDirectory, inputName, inputCategory, inputHash, inputID)
  File "./TorrentToMedia.py", line 56, in main
    if config.issubsection(inputCategory,["SickBeard"]):
TypeError: unbound method issubsection() must be called with config instance as first argument (got str instance instead)
[SickBeard]
    #### autoProcessing for TV Series
    #### tv - category that gets called for post-processing with SB
    [[tv]]
        enabled = 0
        host = localhost
        port = 8081
        username = ""
        password = ""
        ###### ADVANCED USE - ONLY EDIT IF YOU KNOW WHAT YOU'RE DOING ######
        web_root = ""
        ssl = 0
        delay = 0
        TimePerGiB = 60
        watch_dir = ""
        fork = auto
        delete_failed = 0
        nzbExtractionBy = Downloader
        Torrent_NoLink = 0
        process_method = ""
    [[sickbeard]]
        enabled = 1
        host = 127.0.0.1
        port = 8081
        username = ""
        password = ""
        web_root = /sickbeard
        ssl = 0
        delay = 0
        TimePerGiB = 60
        watch_dir = ""
        fork = auto
        delete_failed = 1
        nzbExtractionBy = Destination
        Torrent_NoLink = 0
        process_method = ""
echel0n commented 10 years ago

change it to config().issubsection ... missing the ()

echel0n commented 10 years ago

its cause my new changes to the config class now allow passing the config to the function from using self as to our old method required re-loading config twice

echel0n commented 10 years ago

I'm actually working on a even better version of our config class that'll allow doing config()['section].issubsection(inputCategory, checkenabled=True) ..... I'm trying to make it more practical many different possibles that could use it in.

The idea is to class the sub-section under the parent section and not to worry about all other subsections from different parents even if labeled with the same name.

ghost commented 10 years ago
  File "./TorrentToMedia.py", line 91
    Logger.debug("MAIN: Found %s files in %s", str(len(filenames)), dirpath)if
                                                                             ^
SyntaxError: invalid syntax
clinton-hall commented 10 years ago

https://github.com/clinton-hall/nzbToMedia/commit/a273f9687fa66e0d13c0da4874767e2fb4e87d4d

sorry... that really was poor of me!

ghost commented 10 years ago

thanks!

ghost commented 10 years ago

all working well on dev-CH.

ghost commented 10 years ago

well, thought so.. new one..

17:38:55|INFO    ====================
17:38:55|INFO    TorrentToMedia V9.3
17:38:55|INFO    MAIN: Loading config from /home/media/.scripts/nzbToMedia/autoProcessMedia.cfg
17:38:55|DEBUG   arg 0 is: /home/media/.scripts/nzbToMedia/TorrentToMedia.py
17:38:55|DEBUG   arg 1 is: /cache/torrent/download/sickbeard/TV.Show
17:38:55|DEBUG   arg 2 is: TV.Show
17:38:55|DEBUG   arg 3 is: sickbeard
17:38:55|DEBUG   arg 4 is: 123456
17:38:55|DEBUG   MAIN: Received Directory: /cache/torrent/download/sickbeard/TV.Show | Name: TV.Show | Category: sickbeard
17:38:55|DEBUG   SEARCH: Found Category: sickbeard in directory structure
17:38:55|INFO    SEARCH: Found a unique directory TV.Show in the category directory
17:38:55|DEBUG   MAIN: Determined Directory: /cache/torrent/download/sickbeard/TV.Show | Name: TV.Show | Category: sickbeard
17:38:55|DEBUG   MAIN: Stoping torrent TV.Show in rtorrent while processing
17:39:00|INFO    MAIN: Output directory set to: /cache/process/sickbeard/TV.Show
17:39:00|DEBUG   MAIN: Scanning files in directory: /cache/torrent/download/sickbeard/TV.Show
17:39:00|DEBUG   MAIN: Found 13 files in /cache/torrent/download/sickbeard/TV.Show
17:39:00|INFO    CREATE DESTINATION: Creating destination folder: /cache/process/sickbeard/TV.Show
17:39:00|INFO    COPYLINK: Hard linking /cache/torrent/download/sickbeard/TV.Show/tv.show.r07 to /cache/process/sickbeard/TV.Show/tv.show.r07
17:39:00|INFO    COPYLINK: Hard linking /cache/torrent/download/sickbeard/TV.Show/tv.show.r05 to /cache/process/sickbeard/TV.Show/tv.show.r05
17:39:00|INFO    COPYLINK: Hard linking /cache/torrent/download/sickbeard/TV.Show/tv.show.r06 to /cache/process/sickbeard/TV.Show/tv.show.r06
17:39:00|INFO    COPYLINK: Hard linking /cache/torrent/download/sickbeard/TV.Show/tv.show.r00 to /cache/process/sickbeard/TV.Show/tv.show.r00
17:39:00|INFO    COPYLINK: Hard linking /cache/torrent/download/sickbeard/TV.Show/tv.show.nfo to /cache/process/sickbeard/TV.Show/tv.show.nfo
17:39:00|INFO    COPYLINK: Hard linking /cache/torrent/download/sickbeard/TV.Show/tv.show.r04 to /cache/process/sickbeard/TV.Show/tv.show.r04
17:39:00|INFO    COPYLINK: Hard linking /cache/torrent/download/sickbeard/TV.Show/tv.show.sfv to /cache/process/sickbeard/TV.Show/tv.show.sfv
17:39:00|INFO    COPYLINK: Hard linking /cache/torrent/download/sickbeard/TV.Show/tv.show.r01 to /cache/process/sickbeard/TV.Show/tv.show.r01
17:39:00|INFO    COPYLINK: Hard linking /cache/torrent/download/sickbeard/TV.Show/tv.show.r08 to /cache/process/sickbeard/TV.Show/tv.show.r08
17:39:00|INFO    COPYLINK: Hard linking /cache/torrent/download/sickbeard/TV.Show/tv.show.rar to /cache/process/sickbeard/TV.Show/tv.show.rar
17:39:00|INFO    COPYLINK: Hard linking /cache/torrent/download/sickbeard/TV.Show/tv.show.r02 to /cache/process/sickbeard/TV.Show/tv.show.r02
17:39:00|INFO    COPYLINK: Hard linking /cache/torrent/download/sickbeard/TV.Show/tv.show.r09 to /cache/process/sickbeard/TV.Show/tv.show.r09
17:39:00|INFO    COPYLINK: Hard linking /cache/torrent/download/sickbeard/TV.Show/tv.show.r03 to /cache/process/sickbeard/TV.Show/tv.show.r03
17:39:00|DEBUG   MAIN: Found 1 files in /cache/torrent/download/sickbeard/TV.Show/Sample
17:39:00|INFO    MAIN: Ignoring sample file: /cache/torrent/download/sickbeard/TV.Show/Sample/tv.show.sample.mkv  
17:39:00|INFO    FLATTEN: Flattening directory: /cache/process/sickbeard/TV.Show
17:39:00|INFO    REMOVER: Removing empty folders in: /cache/process/sickbeard/TV.Show
17:39:00|ERROR   MAIN: Something failed! Please check logs. Exiting
media@download:~/.scripts/nzbToMedia$ ./TorrentToMedia.py /cache/torrent/download/sickbeard/TV.Show
18:12:59|INFO    ====================
18:12:59|INFO    TorrentToMedia V9.3
18:12:59|INFO    MAIN: Loading config from /home/media/.scripts/nzbToMedia/autoProcessMedia.cfg
18:12:59|INFO    SEARCH: Found a unique directory TV.Show in the category directory
18:13:04|INFO    MAIN: Output directory set to: /cache/process/sickbeard/TV.Show
18:13:04|INFO    COPYLINK: target file already exists. Nothing to be done
18:13:04|INFO    COPYLINK: target file already exists. Nothing to be done
18:13:04|INFO    COPYLINK: target file already exists. Nothing to be done
18:13:04|INFO    COPYLINK: target file already exists. Nothing to be done
18:13:04|INFO    COPYLINK: target file already exists. Nothing to be done
18:13:04|INFO    COPYLINK: target file already exists. Nothing to be done
18:13:04|INFO    COPYLINK: target file already exists. Nothing to be done
18:13:04|INFO    COPYLINK: target file already exists. Nothing to be done
18:13:04|INFO    COPYLINK: target file already exists. Nothing to be done
18:13:04|INFO    COPYLINK: target file already exists. Nothing to be done
18:13:04|INFO    COPYLINK: target file already exists. Nothing to be done
18:13:04|INFO    COPYLINK: target file already exists. Nothing to be done
18:13:04|INFO    COPYLINK: target file already exists. Nothing to be done
18:13:04|INFO    MAIN: Ignoring sample file: /cache/torrent/download/sickbeard/TV.Show/Sample/tv.show.sample.mkv  
18:13:04|INFO    FLATTEN: Flattening directory: /cache/process/sickbeard/TV.Show
18:13:04|INFO    REMOVER: Removing empty folders in: /cache/process/sickbeard/TV.Show
Traceback (most recent call last):
  File "/usr/lib/python2.7/logging/__init__.py", line 851, in emit
    msg = self.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 724, in format
    return fmt.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 464, in format
    record.message = record.getMessage()
  File "/usr/lib/python2.7/logging/__init__.py", line 328, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting
Logged from file TorrentToMedia.py, line 173
Traceback (most recent call last):
  File "/usr/lib/python2.7/logging/handlers.py", line 76, in emit
    if self.shouldRollover(record):
  File "/usr/lib/python2.7/logging/handlers.py", line 156, in shouldRollover
    msg = "%s\n" % self.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 724, in format
    return fmt.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 464, in format
    record.message = record.getMessage()
  File "/usr/lib/python2.7/logging/__init__.py", line 328, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting
Logged from file TorrentToMedia.py, line 173
18:13:04|ERROR   MAIN: Something failed! Please check logs. Exiting
clinton-hall commented 10 years ago

Sorry about that... Hopefully fixed now.

And the version printing will now return the git version (where available) so I will know if you are running latest or not ;)

ghost commented 10 years ago

wish i could help more with code, besides just posting all these errors.. thanks for your help!

20:43:32|WARNING MAIN: Found no media files in output.
20:43:32|ERROR   MAIN: Something failed! Please check logs. Exiting
20:44:00|INFO    ====================
20:44:00|INFO    TorrentToMedia repo:https://github.com/clinton-hall/nzbToMedia.git branch:dev-CH hash: 8e754aa
20:44:00|INFO    MAIN: Loading config from /home/media/.scripts/nzbToMedia/autoProcessMedia.cfg
20:44:00|DEBUG   arg 0 is: ./TorrentToMedia.py
20:44:00|DEBUG   arg 1 is: /cache/torrent/download/sickbeard/TV.Show
20:44:00|DEBUG   MAIN: Received Directory: /cache/torrent/download/sickbeard/TV.Show | Name:  | Category: 
20:44:00|DEBUG   SEARCH: Found Category: sickbeard in directory structure
20:44:00|INFO    SEARCH: Found a unique directory TV.Show in the category directory
20:44:00|DEBUG   MAIN: Determined Directory: /cache/torrent/download/sickbeard/TV.Show | Name: TV.Show | Category: sickbeard
20:44:00|DEBUG   MAIN: Stoping torrent TV.Show in rtorrent while processing
20:44:05|INFO    MAIN: Output directory set to: /cache/process/sickbeard/TV.Show
20:44:05|DEBUG   MAIN: Scanning files in directory: /cache/torrent/download/sickbeard/TV.Show
20:44:05|DEBUG   MAIN: Found 13 files in /cache/torrent/download/sickbeard/TV.Show
20:44:05|INFO    COPYLINK: target file already exists. Nothing to be done
20:44:05|INFO    COPYLINK: target file already exists. Nothing to be done
20:44:05|INFO    COPYLINK: target file already exists. Nothing to be done
20:44:05|INFO    COPYLINK: target file already exists. Nothing to be done
20:44:05|INFO    COPYLINK: target file already exists. Nothing to be done
20:44:05|INFO    COPYLINK: target file already exists. Nothing to be done
20:44:05|INFO    COPYLINK: target file already exists. Nothing to be done
20:44:05|INFO    COPYLINK: target file already exists. Nothing to be done
20:44:05|INFO    COPYLINK: target file already exists. Nothing to be done
20:44:05|INFO    COPYLINK: target file already exists. Nothing to be done
20:44:05|INFO    COPYLINK: target file already exists. Nothing to be done
20:44:05|INFO    COPYLINK: target file already exists. Nothing to be done
20:44:05|INFO    COPYLINK: target file already exists. Nothing to be done
20:44:05|DEBUG   MAIN: Found 1 files in /cache/torrent/download/sickbeard/TV.Show/Sample
20:44:05|INFO    MAIN: Ignoring sample file: /cache/torrent/download/sickbeard/TV.Show/Sample/tv.show.mkv  
20:44:05|INFO    FLATTEN: Flattening directory: /cache/process/sickbeard/TV.Show
20:44:05|INFO    REMOVER: Removing empty folders in: /cache/process/sickbeard/TV.Show
20:44:05|WARNING MAIN: Found no media files in output.
20:44:05|ERROR   MAIN: Something failed! Please check logs. Exiting
media@download:/cache/process/sickbeard/TV.Show$ ls -la
total 535824
drwxrwxr-x 2 media media     4096 Apr 10 17:39 .
drwxrwxr-x 5 media media     4096 Apr 10 20:10 ..
-rw-rw-r-- 2 media media 50000000 Apr 10 17:38 tv.show.r00
-rw-rw-r-- 2 media media 50000000 Apr 10 17:38 tv.show.r01
-rw-rw-r-- 2 media media 50000000 Apr 10 17:38 tv.show.r02
-rw-rw-r-- 2 media media 50000000 Apr 10 17:38 tv.show.r03
-rw-rw-r-- 2 media media 50000000 Apr 10 17:38 tv.show.r04
-rw-rw-r-- 2 media media 50000000 Apr 10 17:38 tv.show.r05
-rw-rw-r-- 2 media media 50000000 Apr 10 17:38 tv.show.r06
-rw-rw-r-- 2 media media 50000000 Apr 10 17:38 tv.show.r07
-rw-rw-r-- 2 media media 50000000 Apr 10 17:38 tv.show.r08
-rw-rw-r-- 2 media media 48624265 Apr 10 17:38 tv.show.r09
-rw-rw-r-- 2 media media 50000000 Apr 10 17:38 tv.show.rar
-rw-rw-r-- 2 media media      473 Apr 10 17:38 tv.show.sfv
-rw-rw-r-- 2 media media      332 Apr 10 17:38 tv.show.nfo
clinton-hall commented 10 years ago

Yeah... hopefully fixed now... Sorry I can't really test this combination... but I think we are getting close.

clinton-hall commented 10 years ago

Actually... can you test dev-ech branch?... All my fixes plus a lot more...

We will try and merge in to dev soon and get back to only the one testing branch.

ghost commented 10 years ago

dev-ech doesn't work at all. doesn't respect nzbExtractionBy. then failed but no mention of what. did not call SB.

ghost commented 10 years ago

dev-CH is working perfectly.

clinton-hall commented 10 years ago

Do you have logging from dev-ech? I would like to merge the co gift changes etc from this branch, so it would be great to get that branch working.

echel0n commented 10 years ago

OK dev-ech is ready to be merged into dev.

Completely re-wrote the logging interface, log settings no longer can be set from settings but I honestly don't see a reason for them to be so it makes things a lot easier.

Logs are not stored in a file named /logs/nzbtomedia.log and rotates it

Re-worked the config class, the way we checkenabled and check for subsection has changed.

Added in a check for updates class and functions, I haven’t coded it to auto-update yet but I have a idea for that. The code is there just disabled right at the minute as I took this from my sickbeard edition.

Version now returns commit hash and branch.

Config is loaded only once into memory now and referenced from there.

Added a initialization function to init.py in nzbtomedia to load all our constants and config variables into memory so that we don’t by mistake load a config that may be corrupt down the road.

Thanks, Justin.

From: Clinton Hall Sent: ‎Saturday‎, ‎April‎ ‎12‎, ‎2014 ‎1‎:‎23‎ ‎AM To: Clinton Hall Cc: echel0n

Do you have logging from dev-ech? I would like to merge the co gift changes etc from this branch, so it would be great to get that branch working.

— Reply to this email directly or view it on GitHub.

ghost commented 10 years ago

Testing dev-ech. It is not functioning.

11:31:16 INFO:: Loading config from /home/media/.scripts/nzbToMedia/autoProcessMedia.cfg
11:31:16 INFO:: nzbToMedia Version:874805193f74bce00ed26029e8eb814f9a548dec Branch:dev-ech (Linux; 3.11.0-19-generic)
11:31:16 INFO:: Checking if git needs an update
11:31:18 INFO:: No update needed
11:31:23 POSTPROCESS:: Output directory set to: /cache/process/sickbeard/TV.Show.mkv
Traceback (most recent call last):
  File "./TorrentToMedia.py", line 413, in <module>
    main(inputDirectory, inputName, inputCategory, inputHash, inputID)
  File "./TorrentToMedia.py", line 87, in main
    filePath = os.path.join(dirpath, file)
  File "/usr/lib/python2.7/posixpath.py", line 75, in join
    if b.startswith('/'):
AttributeError: 'list' object has no attribute 'startswith'

Also doesn't log properly to new location, /logs/nzbtomedia.log. Well, it doesn't grab any of the python errors.. I guess that's to be expected.

2014-04-12 11:26:30 INFO    :: Loading config from /home/media/.scripts/nzbToMed
ia/autoProcessMedia.cfg
2014-04-12 11:26:30 INFO    :: nzbToMedia Version:874805193f74bce00ed26029e8eb81
4f9a548dec Branch:dev-ech (Linux; 3.11.0-19-generic)
2014-04-12 11:26:30 INFO    :: Checking if git needs an update
2014-04-12 11:26:32 INFO    :: No update needed
2014-04-12 11:26:37 POSTPROCESS:: Output directory set to: /cache/process/sickbe
ard/TV.Show.mkv
ghost commented 10 years ago

In addition, dev-CH is not cleaning after UserScript.

    user_script_runOnce = 1
    #Specify the successcodes returned by the user script as a comma separated l
ist. Linux default is 0
    user_script_successCodes = 0
    #Clean after? Note that delay function is used to prevent possible mistake :
) Delay is intended as seconds
    user_script_clean = 1
clinton-hall commented 10 years ago

dev-ech was merged into dev. I have fixed this error for single file handling (also existed in dev-ch) Please switch to dev now...

Unless the error happens in a try except section, these won't be captured in the log....

I'm looking into the cleanup after userscript.

clinton-hall commented 10 years ago

user script clean should work....

can you post debug log to show what happened for a run where userscript was called (but no cleanup happened?)

ghost commented 10 years ago

ok.. with merge to dev; back to hard linking properly; but then not finding anything to send to SB..

[13:30:34] [WARNING]::MAIN:: Found no media files in output.
[13:30:34] [ERROR]::MAIN:: Something failed! Please check logs. Exiting
[13:30:34] [ERROR]::MAIN:: A problem was reported when trying to manually run SickBeard:sickbeard.
clinton-hall commented 10 years ago

Ok, can you show the full logs to see what's going on? You said files were linked? For whatever reason there are no media files found....

ghost commented 10 years ago

cleaning out some stuff, i've been away from home and it's been failing for a week. going to get a clean single run.

it looks to be the issue earlier. it links, but then doesn't understand nzbExtractionBy = Destination; expects to find a video file; but extraction hasn't been performed yet; so doesn't attempt to send to sickbeard for processing.