search

clinton-hall / nzbToMedia

Provides NZB and Torrent postprocessing To CouchPotatoServer, SickBeard/SickRage, HeadPhones, Mylar and Gamez
GNU General Public License v3.0
672 stars 176 forks source link

Failing from SABnzbd to Couchpotato #595

Closed FlexibleToast closed 10 years ago

FlexibleToast commented 10 years ago

I'm not sure what is going on. Reinstalled nzbToMedia and redid my autoprocess.cfg. Not sure where to go for help, so I'm here. Here is what SABnzbd log says:

[14:24:54] [INFO]::MAIN: Checking database structure...
[14:24:54] [INFO]::MAIN: Checking if git needs an update
[14:24:54] [ERROR]::MAIN: Unable to determine installed version via git, please check your logs!
[14:24:54] [INFO]::MAIN: No update needed
[14:24:54] [INFO]::MAIN: nzbToMedia Version:10.7 Branch:master (FreeBSD 9.1-RELEASE)
[14:24:54] [WARNING]::MAIN: Failed to locate 7zip, transcosing of disk images ane extraction of .7z files will not be possible!
[14:24:54] [WARNING]::MAIN: Failed to locate ffmpeg, transcoding disabled!
[14:24:54] [WARNING]::MAIN: Install ffmpeg with x264 support to enable this feature  ...
[14:24:54] [WARNING]::MAIN: Failed to locate ffprobe, video corruption detection disabled!
[14:24:54] [WARNING]::MAIN: Install ffmpeg with x264 support to enable this feature  ...
[14:24:54] [INFO]::MAIN: #########################################################
[14:24:54] [INFO]::MAIN: ## ..::[nzbToMedia.pyc]::.. ##
[14:24:54] [INFO]::MAIN: #########################################################
[14:24:54] [INFO]::MAIN: Script triggered from SABnzbd
[14:24:58] [INFO]::MAIN: Auto-detected SECTION:CouchPotato
[14:24:58] [INFO]::MAIN: Calling CouchPotato:movies to post-process:The.Judge.2014.720p.HDRip.HC.x264.AC3-CPG-NZBgeek.cp(tt1872194).nzb
[14:24:58] [INFO]::MAIN: Attemping imdbID lookup for The.Judge.2014.720p.HDRip.HC.x264.AC3-CPG-NZBgeek.cp(tt1872194).nzb
[14:24:58] [INFO]::MAIN: Searching folder and file names for imdbID ...
[14:24:58] [INFO]::MAIN: Found imdbID [tt1872194]
[14:24:58] [ERROR]::MAIN: CouchPotato returned the following non-json data
[14:24:58] [ERROR]::MAIN: 
[14:24:58] [ERROR]::MAIN: <!doctype html>
[14:24:58] [ERROR]::MAIN: <html>
[14:24:58] [ERROR]::MAIN: <head>
[14:24:58] [ERROR]::MAIN: <meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no"/>
[14:24:58] [ERROR]::MAIN: <meta name="apple-mobile-web-app-capable" content="yes">
[14:24:58] [ERROR]::MAIN: <meta name="mobile-web-app-capable" content="yes">
[14:24:58] [ERROR]::MAIN: <meta http-equiv="X-UA-Compatible" content="IE=edge" />
[14:24:58] [ERROR]::MAIN: 
[14:24:58] [ERROR]::MAIN: <link rel="stylesheet" href="/movies/minified/front.css?1414158389" type="text/css">
[14:24:58] [ERROR]::MAIN: 
[14:24:58] [ERROR]::MAIN: <script type="text/javascript" src="/movies/minified/front.js?1414158392"></script>
[14:24:58] [ERROR]::MAIN: <link href="/movies/static/images/favicon.ico" rel="icon" type="image/x-icon" />
[14:24:58] [ERROR]::MAIN: <link rel="apple-touch-icon" href="/movies/static/images/homescreen.png" />
[14:24:58] [ERROR]::MAIN: <script type="text/javascript">
[14:24:58] [ERROR]::MAIN: window.addEvent('domready', function(){
[14:24:58] [ERROR]::MAIN: new Form.Check($('remember_me'));
[14:24:58] [ERROR]::MAIN: });
[14:24:58] [ERROR]::MAIN: </script>
[14:24:58] [ERROR]::MAIN: <title>CouchPotato</title>
[14:24:58] [ERROR]::MAIN: </head>
[14:24:58] [ERROR]::MAIN: <body class="page login">
[14:24:58] [ERROR]::MAIN: <form action="" method="post">
[14:24:58] [ERROR]::MAIN: <h1>CouchPotato</h1>
[14:24:58] [ERROR]::MAIN: <div class="ctrlHolder"><input class="inlay" name="username" type="text" placeholder="Username" autocomplete="off" /></div>
[14:24:58] [ERROR]::MAIN: <div class="ctrlHolder"><input class="inlay" name="password" type="password" placeholder="Password" autocomplete="off" /></div>
[14:24:58] [ERROR]::MAIN: <div class="ctrlHolder">
[14:24:58] [ERROR]::MAIN: <label class="remember_me" title="for 30 days"><input class="inlay" id="remember_me" name="remember_me" type="checkbox" value="1" checked="checked" /> Remember me</label>
[14:24:58] [ERROR]::MAIN: <input class="button" name="submit" type="submit" value="Login" />
[14:24:58] [ERROR]::MAIN: </div>
[14:24:58] [ERROR]::MAIN: </form>
[14:24:58] [ERROR]::MAIN: </body>
[14:24:58] [ERROR]::MAIN: </html>
[14:24:58] [POSTPROCESS]::COUCHPOTATO: Starting renamer scan for The.Judge.2014.720p.HDRip.HC.x264.AC3-CPG-NZBgeek.cp(tt1872194).nzb
Traceback (most recent call last):
  File "/media/Scripts/nzbToCouchPotato.py", line 247, in <module>
    result = nzbToMedia.main(sys.argv, section)
  File "/media/Scripts/nzbToMedia.py", line 691, in main
    download_id='')
  File "/media/Scripts/nzbToMedia.py", line 580, in process
    inputCategory, failureLink)
  File "/media/Scripts/nzbtomedia/autoProcess/autoProcessMovie.py", line 240, in process
    result = r.json()
  File "/media/Scripts/libs/requests/models.py", line 759, in json
    return json.loads(self.text, **kwargs)
  File "/usr/local/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/local/lib/python2.7/json/decoder.py", line 365, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/lib/python2.7/json/decoder.py", line 383, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
Exception AttributeError: "'NoneType' object has no attribute 'path'" in <bound method PosixProcess.__del__ of <nzbtomedia.nzbToMediaUtil.PosixProcess instance at 0x802024fc8>> ignored
clinton-hall commented 10 years ago

Check your CouchPotato api key in CouchPotato, settings, general. Make sure you set the same Api key in autoProcessMedia.cfg [CouchPotato] [[movies]] api_key

FlexibleToast commented 10 years ago

Yes, that was definitely a problem. I don't know why that had the wrong api, very strange. I'm still getting an error though. It appears the movie was moved, but I now have an empty directory where it was.

Exit(1) Exception AttributeError: "’NoneType’ object has no attribute ’path’" in (More)
[19:17:20] [INFO]::MAIN: Loading config from [/media/Scripts/autoProcessMedia.cfg]
[19:17:20] [INFO]::MAIN: Checking database structure...
[19:17:20] [INFO]::MAIN: Checking if git needs an update
[19:17:20] [ERROR]::MAIN: Unable to determine installed version via git, please check your logs!
[19:17:20] [INFO]::MAIN: No update needed
[19:17:20] [INFO]::MAIN: nzbToMedia Version:10.7 Branch:master (FreeBSD 9.1-RELEASE)
[19:17:20] [WARNING]::MAIN: Failed to locate 7zip, transcosing of disk images ane extraction of .7z files will not be possible!
[19:17:20] [WARNING]::MAIN: Failed to locate ffmpeg, transcoding disabled!
[19:17:20] [WARNING]::MAIN: Install ffmpeg with x264 support to enable this feature  ...
[19:17:20] [WARNING]::MAIN: Failed to locate ffprobe, video corruption detection disabled!
[19:17:20] [WARNING]::MAIN: Install ffmpeg with x264 support to enable this feature  ...
[19:17:20] [INFO]::MAIN: #########################################################
[19:17:20] [INFO]::MAIN: ## ..::[nzbToMedia.pyc]::.. ##
[19:17:20] [INFO]::MAIN: #########################################################
[19:17:20] [INFO]::MAIN: Script triggered from SABnzbd
[19:17:22] [INFO]::MAIN: Auto-detected SECTION:CouchPotato
[19:17:22] [INFO]::MAIN: Calling CouchPotato:movies to post-process:2008 Big Buck Bunny XVID ohne PW.cp(tt1254207).nzb
[19:17:22] [INFO]::MAIN: Attemping imdbID lookup for 2008 Big Buck Bunny XVID ohne PW.cp(tt1254207).nzb
[19:17:22] [INFO]::MAIN: Searching folder and file names for imdbID ...
[19:17:22] [INFO]::MAIN: Found imdbID [tt1254207]
[19:17:22] [POSTPROCESS]::COUCHPOTATO: Starting renamer scan for 2008 Big Buck Bunny XVID ohne PW.cp(tt1254207).nzb
[19:17:26] [POSTPROCESS]::COUCHPOTATO: SUCCESS: Finished renamer scan for folder /media/NZBs/Movies/2008 Big Buck Bunny XVID ohne PW.cp(tt1254207)
[19:17:26] [POSTPROCESS]::COUCHPOTATO: Checking for status change, please stand by ...
[19:17:26] [POSTPROCESS]::COUCHPOTATO: SUCCESS: Release 2008 Big Buck Bunny XVID ohne PW.cp(tt1254207).nzb has now been marked with a status of [DONE]
Traceback (most recent call last):
  File "/media/Scripts/nzbToCouchPotato.py", line 247, in <module>
    result = nzbToMedia.main(sys.argv, section)
  File "/media/Scripts/nzbToMedia.py", line 691, in main
    download_id='')
  File "/media/Scripts/nzbToMedia.py", line 602, in process
    cleanDir(inputDirectory, sectionName, inputCategory)
  File "/media/Scripts/nzbtomedia/nzbToMediaUtil.py", line 688, in cleanDir
    num_files = len(listMediaFiles(path, minSize=minSize, delete_ignored=delete_ignored))
  File "/media/Scripts/nzbtomedia/nzbToMediaUtil.py", line 878, in listMediaFiles
    if is_sample(fullCurFile) or not is_minSize(fullCurFile, minSize):
  File "/media/Scripts/nzbtomedia/nzbToMediaUtil.py", line 185, in is_minSize
    inputSize = os.path.getsize(inputName)
  File "/usr/local/lib/python2.7/genericpath.py", line 49, in getsize
    return os.stat(filename).st_size
OSError: [Errno 2] No such file or directory: '/media/NZBs/Movies/2008 Big Buck Bunny XVID ohne PW.cp(tt1254207)/2008 Big_Buck_Bunny XVID.avi'
Exception AttributeError: "'NoneType' object has no attribute 'path'" in <bound method PosixProcess.__del__ of <nzbtomedia.nzbToMediaUtil.PosixProcess instance at 0x802024fc8>> ignored
clinton-hall commented 10 years ago

wow... looks like a timing issue. Somehow that file is removed just after a list of files is created?

FlexibleToast commented 10 years ago

Happened again for another movie. The download itself failed several times, which the script successfully had CouchPotato mark as failed and get a new copy. When it finally downloaded it did the same thing.

[07:26:43] [INFO]::MAIN: Loading config from [/media/Scripts/autoProcessMedia.cfg]
[07:26:43] [INFO]::MAIN: Checking database structure...
[07:26:43] [INFO]::MAIN: Checking if git needs an update
[07:26:43] [ERROR]::MAIN: Unable to determine installed version via git, please check your logs!
[07:26:43] [INFO]::MAIN: No update needed
[07:26:43] [INFO]::MAIN: nzbToMedia Version:10.7 Branch:master (FreeBSD 9.1-RELEASE)
[07:26:43] [WARNING]::MAIN: Failed to locate 7zip, transcosing of disk images ane extraction of .7z files will not be possible!
[07:26:43] [WARNING]::MAIN: Failed to locate ffmpeg, transcoding disabled!
[07:26:43] [WARNING]::MAIN: Install ffmpeg with x264 support to enable this feature  ...
[07:26:43] [WARNING]::MAIN: Failed to locate ffprobe, video corruption detection disabled!
[07:26:43] [WARNING]::MAIN: Install ffmpeg with x264 support to enable this feature  ...
[07:26:43] [INFO]::MAIN: #########################################################
[07:26:43] [INFO]::MAIN: ## ..::[nzbToMedia.pyc]::.. ##
[07:26:43] [INFO]::MAIN: #########################################################
[07:26:43] [INFO]::MAIN: Script triggered from SABnzbd
[07:26:45] [INFO]::MAIN: Auto-detected SECTION:CouchPotato
[07:26:45] [INFO]::MAIN: Calling CouchPotato:movies to post-process:Transformers.Age.of.Extinction.2014.720p.BluRay.x264-BLOW.REPOST.cp(tt2109248).nzb
[07:26:45] [INFO]::MAIN: Attemping imdbID lookup for Transformers.Age.of.Extinction.2014.720p.BluRay.x264-BLOW.REPOST.cp(tt2109248).nzb
[07:26:45] [INFO]::MAIN: Searching folder and file names for imdbID ...
[07:26:45] [INFO]::MAIN: Found imdbID [tt2109248]
[07:26:46] [POSTPROCESS]::COUCHPOTATO: Starting renamer scan for Transformers.Age.of.Extinction.2014.720p.BluRay.x264-BLOW.REPOST.cp(tt2109248).nzb
[07:27:36] [POSTPROCESS]::COUCHPOTATO: SUCCESS: Finished renamer scan for folder /media/NZBs/Movies/Transformers Age of Extinction 2014 720p BluRay x264-BLOW REPOST cp(tt2109248)
[07:27:36] [POSTPROCESS]::COUCHPOTATO: Checking for status change, please stand by ...
[07:27:36] [POSTPROCESS]::COUCHPOTATO: SUCCESS: Release Transformers.Age.of.Extinction.2014.720p.BluRay.x264-BLOW.REPOST.cp(tt2109248).nzb has now been marked with a status of [DONE]
Traceback (most recent call last):
  File "/media/Scripts/nzbToCouchPotato.py", line 247, in <module>
    result = nzbToMedia.main(sys.argv, section)
  File "/media/Scripts/nzbToMedia.py", line 691, in main
    download_id='')
  File "/media/Scripts/nzbToMedia.py", line 602, in process
    cleanDir(inputDirectory, sectionName, inputCategory)
  File "/media/Scripts/nzbtomedia/nzbToMediaUtil.py", line 688, in cleanDir
    num_files = len(listMediaFiles(path, minSize=minSize, delete_ignored=delete_ignored))
  File "/media/Scripts/nzbtomedia/nzbToMediaUtil.py", line 878, in listMediaFiles
    if is_sample(fullCurFile) or not is_minSize(fullCurFile, minSize):
  File "/media/Scripts/nzbtomedia/nzbToMediaUtil.py", line 185, in is_minSize
    inputSize = os.path.getsize(inputName)
  File "/usr/local/lib/python2.7/genericpath.py", line 49, in getsize
    return os.stat(filename).st_size
OSError: [Errno 2] No such file or directory: '/media/NZBs/Movies/Transformers Age of Extinction 2014 720p BluRay x264-BLOW REPOST cp(tt2109248)/blow-transformers.age.of.extinction.2014.720p.bluray.x264.mkv'
Exception AttributeError: "'NoneType' object has no attribute 'path'" in <bound method PosixProcess.__del__ of <nzbtomedia.nzbToMediaUtil.PosixProcess instance at 0x802024fc8>> ignored

The movie was successfully renamed and moved, but there is still a directory with the ignored files in it where the file was moved from (/media/NZBs/Movies).

I checked Couchpotato, and it was set to scan a download directory and rename itself. But, it was set to scan a different directory, /media/Torrents/Downloads. So, I don't think that should be the issue.

clinton-hall commented 10 years ago

Sorry, I forgot to mention that the fix above is only available in nightly branch, not master.

can you show CouchPotato logs from 07:26 to 07:28? Is CouchPotato on the same system?

FlexibleToast commented 10 years ago 
10-29 07:26:46INFO[tato.core.plugins.renamer] Scanning media folder /media/NZBs/Movies/Transformers Age of Extinction 2014 720p BluRay x264-BLOW REPOST cp(tt2109248)...
10-29 07:26:46INFO[tato.core.plugins.scanner] Found 1 movies in the folder /media/NZBs/Movies
10-29 07:26:46INFO[hpotato.core.plugins.base] Opening url: get http://api.themoviedb.org/3/movie/tt2109248?api_key=xxx&append_to_response=alternative_titles, data: []
10-29 07:26:46INFO[hpotato.core.plugins.base] Opening url: get http://www.omdbapi.com/?i=tt2109248, data: []
10-29 07:26:46INFO[hpotato.core.plugins.base] Opening url: get https://api.couchpota.to/info/tt2109248/, data: []
10-29 07:26:46INFO[ie.providers.info.omdbapi] Found: Transformers: Age of Extinction (2014)
10-29 07:26:46INFO[tato.core.plugins.renamer] Linking "/media/NZBs/Movies/Transformers Age of Extinction 2014 720p BluRay x264-BLOW REPOST cp(tt2109248)/blow-transformers.age.of.extinction.2014.720p.bluray.x264.mkv" to "/media/Movies/Transformers Age of Extinction (2014)/Transformers Age of Extinction720p.mkv"
10-29 07:27:30INFO[hpotato.core.plugins.base] Opening url: post https://api.pushbullet.com/v2/pushes, data: ['device_iden', 'body', 'type', 'title']
10-29 07:27:30INFO[hpotato.core.plugins.base] Opening url: get http://192.168.0.103:32400/library/sections, data: []
10-29 07:27:30INFO[hpotato.core.plugins.base] Opening url: get http://192.168.0.103:32400/library/sections/1/refresh, data: []
10-29 07:27:30INFO[o.core.notifications.plex] Failed to send notification to client "". Client list is stale, updating the client list and retrying.
10-29 07:27:30INFO[notifications.plex.server] Searching for clients on Plex Media Server
10-29 07:27:30INFO[hpotato.core.plugins.base] Opening url: get http://192.168.0.103:32400/clients, data: []
10-29 07:27:35INFO[hpotato.core.plugins.base] Opening url: get https://api.couchpota.to/search/transformers+age+of+extinction720p+2014/?limit=1, data: []
10-29 07:27:35INFO[hpotato.core.plugins.base] Opening url: get http://api.themoviedb.org/3/search/movie?api_key=xxx&search_type=phrase&query=transformers+age+of+extinction720p&year=2014, data: []
10-29 07:27:35INFO[hpotato.core.plugins.base] Opening url: get http://www.omdbapi.com/?y=2014&t=transformers+age+of+extinction720p, data: []
10-29 07:27:36INFO[hpotato.core.plugins.base] Opening url: get http://api.themoviedb.org/3/search/movie?api_key=xxx&search_type=phrase&query=transformers+age+of+extinction&year=2014, data: []
10-29 07:27:36INFO[hpotato.core.plugins.base] Opening url: get http://www.omdbapi.com/?y=2014&t=transformers+age+of+extinction, data: []
10-29 07:27:36INFO[hpotato.core.plugins.base] Opening url: get https://api.couchpota.to/search/Transformers+Age+of+Extinction+2014/?limit=1, data: []
10-29 07:27:36INFO[ie.providers.info.omdbapi] Found: Transformers: Age of Extinction (2014)
10-29 07:27:36INFO[hpotato.core.plugins.base] Opening url: get http://api.themoviedb.org/3/movie/91314?api_key=xxx&append_to_response=alternative_titles, data: []
10-29 07:27:36INFO[providers.info.themoviedb] Found: [u'Transformers: Age of Extinction (2014)']
10-29 07:27:36INFO[tato.core.plugins.scanner] Found 1 movies in the folder /media/Movies/Transformers Age of Extinction (2014)

No, they're all in separate jails. I'm running them all in FreeNAS plugins. They all have the relevant directories mapped the same and permissions to what they need. Looks like the jail running SABnzbd doesn't have git though. At least that's why I'm assuming it is failing to find git and search for a new release of nzbToMedia.

clinton-hall commented 10 years ago

Ok... Any reason you have CouchPotato, settings, renamer, Default File Action = Link?

Can I also assume you have settings, Downlaoders, SABnzbd Remove NZB enabled?

I think this is a combination of settings that wasn't anticipated... essentially it links the files, then when nzbToMedia goes to clean up it finds files that suddenly disappear (CouchPotato cleans them up at the same time)...

Can you try setting Default File Action = Move?

FlexibleToast commented 10 years ago

I think I had set it to link for torrents. I wasn't sure exactly what action it was taking, but my understanding was that hard links were being created so the movie could continue to seed. I didn't know it was also applying these settings when your script calls it. Also, it appears that Transmission doesn't call your script until its seed goal has been met anyway. So, it seems as though that setting is largely unnecessary for myself. I changed it back to move. Still gave me an error though.

[09:57:34] [INFO]::MAIN: #########################################################
[09:57:34] [INFO]::MAIN: ## ..::[nzbToMedia.pyc]::.. ##
[09:57:34] [INFO]::MAIN: #########################################################
[09:57:34] [INFO]::MAIN: Script triggered from SABnzbd
[09:57:35] [INFO]::MAIN: Auto-detected SECTION:CouchPotato
[09:57:35] [INFO]::MAIN: Calling CouchPotato:movies to post-process:2008 Big Buck Bunny XVID ohne PW.nzb
[09:57:35] [INFO]::MAIN: Attemping imdbID lookup for 2008 Big Buck Bunny XVID ohne PW.nzb
[09:57:35] [INFO]::MAIN: Searching folder and file names for imdbID ...
[09:57:35] [INFO]::MAIN: Searching IMDB for imdbID ...
[09:57:35] [INFO]::MAIN: Found imdbID [tt1254207]
[09:57:35] [POSTPROCESS]::COUCHPOTATO: Starting renamer scan for 2008 Big Buck Bunny XVID ohne PW.nzb
[09:57:35] [POSTPROCESS]::COUCHPOTATO: SUCCESS: Finished renamer scan for folder /media/NZBs/Movies/2008 Big Buck Bunny XVID ohne PW
[09:57:35] [POSTPROCESS]::COUCHPOTATO: Checking for status change, please stand by ...
[09:57:55] [POSTPROCESS]::COUCHPOTATO: Checking for status change, please stand by ...
[09:58:15] [POSTPROCESS]::COUCHPOTATO: Checking for status change, please stand by ...
[09:58:35] [POSTPROCESS]::COUCHPOTATO: Checking for status change, please stand by ...
[09:58:55] [POSTPROCESS]::COUCHPOTATO: Checking for status change, please stand by ...
[09:59:15] [POSTPROCESS]::COUCHPOTATO: Checking for status change, please stand by ...
[09:59:35] [WARNING]::COUCHPOTATO: 2008 Big Buck Bunny XVID ohne PW.nzb does not appear to have changed status after 2 minutes, Please check your logs.
[09:59:35] [ERROR]::MAIN: A problem was reported in the /media/Scripts/nzbToCouchPotato.py script.
CouchPotato: Failed to post-process - No change in status!

And the file didn't actually get moved/renamed at all. I had to enable renamer in CP so I could change link to move, then I disabled it again. Now it doesn't seem to rename or move anything. CP doesn't even create any logs beyond the initial snatch.

clinton-hall commented 10 years ago

CouchPotato's renamer needs to be enabled, this script actually calls the renamer... The difference is the renamer has run_every and force_every set to 0 so that it doesn't run automatically, but only when called from this script.

https://github.com/clinton-hall/nzbToMedia/wiki/couchpotato

So in renamer there are two "actions" the default and torrent.... certainly default should be move as there is no need to link nzb downloads. For Torrents, this script will do the linking and then ask CouchPotato to scan the folder... so generally set both actions to Move.

The script should run when Transmission finishes downloading (not when seeding complete). https://github.com/clinton-hall/nzbToMedia/wiki/transmission

FlexibleToast commented 10 years ago

Turned on the renamer and it seems to be working now. Thanks for the help.