search

mdhiggins / sickbeard_mp4_automator

Automatically convert video files to a standardized format with metadata tagging to create a beautiful and uniform media library
MIT License
1.53k stars 200 forks source link

SB: "trying to post process an existing show directory" again. #168

Closed Geordon closed 10 years ago

Geordon commented 10 years ago

I have posted this to the SB forum as well (http://sickbeard.com/forums/viewtopic.php?f=4&t=9462&p=45462#p45462) . I'm not sure what I messed up, but I'm pretty sure that it was something that I changed, I just don't know what.

Here's the log :

Valid file detected: /Users/<USERNAME>/TV/Falling Skies/Season 3/Falling Skies - 3x09 - Journey to Xibalba.mkv
Video codec detected: h264
Audio stream detected: flac eng [Stream 1]
Creating dual audio channels for iOS compatability for this stream
Subtitle stream detected: subrip eng [Stream 2]
/Users/<USERNAME>/TV/Falling Skies/Season 3/Falling Skies - 3x09 - Journey to Xibalba.mp4 created
/Users/<USERNAME>/TV/Falling Skies/Season 3/Falling Skies - 3x09 - Journey to Xibalba.mkv deleted
Valid file detected: /Users/<USERNAME>/TV/Falling Skies/Season 3/snow-falling.skies.s03e09.1080p.bluray.x264-sample.mkv
Video codec detected: h264
Audio stream detected: flac eng [Stream 1]
Creating dual audio channels for iOS compatability for this stream
Subtitle stream detected: subrip eng [Stream 2]
/Users/<USERNAME>/TV/Falling Skies/Season 3/snow-falling.skies.s03e09.1080p.bluray.x264-sample.mp4 created
/Users/<USERNAME>/TV/Falling Skies/Season 3/snow-falling.skies.s03e09.1080p.bluray.x264-sample.mkv deleted
Loading config from /usr/local/etc/SAB_post_process/sickbeard_mp4_automator-master/autoProcess.ini
Opening URL: http://localhost:8081/home/postprocess/processEpisode?nzbName=Falling.Skies.S03E09.1080p.BluRay.x264-SNOW.nzb&quiet=1&dir=%2FUsers%<USERNAME>%2FTV%2FFalling+Skies%2FSeason+3
Processing folder: /Users/<USERNAME>/TV/Falling Skies/Season 3

You're trying to post process an existing show directory: /Users/<USERNAME>/TV/Falling Skies/Season 3

I'm using SAB to download to temp folder ~/Downloads/incomplete Completed DL folder is set as ~/Downloads/complete

My SB config locations are:

root_dirs = 0|/Users/<USERNAME>/TV
tv_download_dir = /Users/<USERNAME>/Downloads/complete

I'm post-processing with sickbeard_mp4_automator-master and for some reason that I can't figure out, TV shows have started to post-process in the folder mentioned above. I know I changed something, because I have several seasons of several shows that processed correctly, and the process stopped working in the past couple of days. I've been trying to figure out what I messed up, but I'm stuck.

Any help would be GREATLY appreciated.

mdhiggins commented 10 years ago

So this seems to be more an issue with something setup in Sickbeard, not my script. Where is this log from btw? Is this a "Manual Post Processing" log from SB?

Geordon commented 10 years ago

The log came out of SAB, after running the sabToSickbeardwithConverter.py script. Evertyhing that I've uncovered suggests that it is a problem with the SB config, as it's a SB message (several questions asked about it over on the SB forum), but it seems to be happening at the interface of SB and the postprocess script. I didn't know if you had any ideas about it, which is why I posted here, too.

Thanks!

mdhiggins commented 10 years ago

Could you post your autoProcess.ini settings?

Geordon commented 10 years ago

Sure!

autoProcess.ini:

[SickBeard]
host=localhost
port=8081
username=
password=
web_root=
ssl=0
api_key= <REDACTED>

[NzbDrone]
host=localhost
port=8082
web_root=
ssl=0
api_key=

[MP4]
ffmpeg=/usr/local/bin/ffmpeg
ffprobe=/usr/local/bin/ffprobe
output_directory=
copy_to=
move_to=/Users/<USERNAME>/Music/iTunes/iTunes Media/Automatically Add to iTunes.localized/
output_extension=mp4
output_format=mp4
delete_original=True
relocate_moov=True
audio-codec=ac3
video-codec=h264,x264
ios-audio=True
max-audio-channels=2
audio-language=eng
audio-default-language=eng
subtitle-language=eng
subtitle-default-language=eng
fullpathguess=True
convert-mp4=False
tagfile=True
download-artwork=True
download-subs=False
embed-subs=True
sub-providers=addic7ed,podnapisi,thesubdb,opensubtitles

[CouchPotato]
host = localhost
port = 5050
username = 
password =
web_root =
ssl = 0
apikey = <REDACTED>
delay = 65
method = renamer
delete_failed = 0

[uTorrent]
label=
mdhiggins commented 10 years ago

I guess just to rule out my script causing an issue that I'm not able to identify, have you tried downloading a file without using the SABtoSickbeardwithConverter.py file? (just use the one Sickbeard comes with, SABtoSickbeard.py) Let me know if you still get the same error.

Geordon commented 10 years ago

I left a couple of SB downloads running when I went to bed last night, because I thought that I had maybe found what the problem was (SAB > Config > Sorting> Enable TV sorting) and turned it off, to see if that helped the problem. Well, sort of. HEre's the log out of SAB:

Valid file detected: /Users/<USERNAME>/TV/Game of Thrones S03E01 WEB-DL 1080P/Game of Thrones S03E01 WEB-DL 1080P.mkv
Video codec detected: h264
Audio stream detected: ac3 eng [Stream 1]
Creating dual audio channels for iOS compatability for this stream
Subtitle stream detected: subrip dut [Stream 2]
/Users/<USERNAME>/TV/Game of Thrones S03E01 WEB-DL 1080P/Game of Thrones S03E01 WEB-DL 1080P.mp4 created
/Users/<USERNAME>/TV/Game of Thrones S03E01 WEB-DL 1080P/Game of Thrones S03E01 WEB-DL 1080P.mkv deleted
Loading config from /usr/local/etc/SAB_post_process/sickbeard_mp4_automator-master/autoProcess.ini
Opening URL: http://localhost:8081/home/postprocess/processEpisode?nzbName=Game+of+Thrones+S03E01+WEB-DL+1080P.nzb&quiet=1&dir=%2FUsers%2F<USERNAME>%2FTV%2FGame+of+Thrones+S03E01+WEB-DL+1080P
Processing folder: /Users/<USERNAME>/TV/Game of Thrones S03E01 WEB-DL 1080P

Recursively processing a folder: /Users/<USERNAME>/TV/Game of Thrones S03E01 WEB-DL 1080P/reclame

Processing folder: /Users/<USERNAME>/TV/Game of Thrones S03E01 WEB-DL 1080P/reclame

Recursively processing a folder: /Users/<USERNAME>/TV/Game of Thrones S03E01 WEB-DL 1080P/reclame/Belangrijk lees dit eerst!!

Processing folder: /Users/<USERNAME>/TV/Game of Thrones S03E01 WEB-DL 1080P/reclame/Belangrijk lees dit eerst!!

There are no videofiles in folder: /Users/<USERNAME>/TV/Game of Thrones S03E01 WEB-DL 1080P/reclame/Belangrijk lees dit eerst!!

There are no videofiles in folder: /Users/<USERNAME>/TV/Game of Thrones S03E01 WEB-DL 1080P/reclame

Processing /Users/<USERNAME>/TV/Game of Thrones S03E01 WEB-DL 1080P/Game of Thrones S03E01 WEB-DL 1080P.mp4 (Game of Thrones S03E01 WEB-DL 1080P.nzb)

Found result in history: (121361, 3, [], 64)

Parsed Game of Thrones S03E01 WEB-DL 1080P into Game of Thrones - S3E1 - WEB-DL 1080P [ABD: False]

Looking up Game of Thrones in the DB

Lookup successful, using tvdb id 121361

Loading show object for tvdb_id 121361

Retrieving episode object for 3x1

Snatch history had a quality in it, using that: 1080p WEB-DL

Sick Beard snatched this episode, marking it safe to replace

This download is marked as safe to replace existing file

Destination folder for this episode: /Users/<USERNAME>/TV/Game of Thrones

Moving file from /Users/<USERNAME>/TV/Game of Thrones S03E01 WEB-DL 1080P/Game of Thrones S03E01 WEB-DL 1080P.mp4 to /Users/<USERNAME>/TV/Game of Thrones/Game of Thrones - 3x01 - Valar Dohaeris.mp4

Executing command ['/usr/bin/python', '/usr/local/etc/SAB_post_process/sickbeard_mp4_automator-master/postConversion.py', '/Users/<USERNAME>/TV/Game of Thrones/Game of Thrones - 3x01 - Valar Dohaeris.mp4', '/Users/<USERNAME>/TV/Game of Thrones S03E01 WEB-DL 1080P/Game of Thrones S03E01 WEB-DL 1080P.mp4', '121361', '3', '1', '2013-03-31']

Script result: Tagging file :/Users/<USERNAME>/TV/Game of Thrones/Game of Thrones - 3x01 - Valar Dohaeris.mp4

Trying to write tags

Traceback (most recent call last):

  File "/usr/local/etc/SAB_post_process/sickbeard_mp4_automator-master/postConversion.py", line 26, in <module>

    tagmp4.writeTags(output['output'], settings.artwork)

  File "/usr/local/etc/SAB_post_process/sickbeard_mp4_automator-master/tvdb_mp4.py", line 98, in writeTags

    video.save()

  File "/usr/local/etc/SAB_post_process/sickbeard_mp4_automator-master/mutagen/__init__.py", line 134, in save

    return self.tags.save(filename, **kwargs)

  File "/usr/local/etc/SAB_post_process/sickbeard_mp4_automator-master/mutagen/mp4.py", line 299, in save

    self.__save_existing(fileobj, atoms, path, data)

  File "/usr/local/etc/SAB_post_process/sickbeard_mp4_automator-master/mutagen/mp4.py", line 352, in __save_existing

    insert_bytes(fileobj, delta, offset)

  File "/usr/local/etc/SAB_post_process/sickbeard_mp4_automator-master/mutagen/_util.py", line 226, in insert_bytes

    map = mmap.mmap(fobj.fileno(), filesize + size)

OverflowError: cannot fit 'long' into an index-sized integer

Processing succeeded for /Users/<USERNAME>/TV/Game of Thrones S03E01 WEB-DL 1080P/Game of Thrones S03E01 WEB-DL 1080P.mp4

The file doesn't seem to be post-processed, since it's still in the show folder.

<USERNAME>@Enso ~/TV/Game of Thrones $ pwd
/Users/<USERNAME>/TV/Game of Thrones
<USERNAME>@Enso ~/TV/Game of Thrones $ ls -ltr
total 4696896
-rw-r--r--  1 <USERNAME>  staff  2404806847 Sep 29 06:02 Game of Thrones - 3x01 - Valar Dohaeris.mp4
Geordon commented 10 years ago

And in another followup, that set of errors did NOT happen with an additional file passed through your script. My setup just finished another download/process and the log generated looks to be clean. The file is tagged correctly and was moved into my iTunes import folder as desired. Not sure what happened with the Game of Thrones file above, though.

Valid file detected: /Users/<USERNAME>/TV/The Strain S01E12 720p WEB-DL DD5 1 H 264-CtrlHD/The.Strain.S01E12.Last.Rites.720p.WEB-DL.DD5.1.H.264-CtrlHD.mkv
Video codec detected: h264
Audio stream detected: ac3 eng [Stream 1]
Creating dual audio channels for iOS compatability for this stream
Subtitle stream detected: subrip eng [Stream 2]
/Users/<USERNAME>/TV/The Strain S01E12 720p WEB-DL DD5 1 H 264-CtrlHD/The.Strain.S01E12.Last.Rites.720p.WEB-DL.DD5.1.H.264-CtrlHD.mp4 created
/Users/<USERNAME>/TV/The Strain S01E12 720p WEB-DL DD5 1 H 264-CtrlHD/The.Strain.S01E12.Last.Rites.720p.WEB-DL.DD5.1.H.264-CtrlHD.mkv deleted
Loading config from /usr/local/etc/SAB_post_process/sickbeard_mp4_automator-master/autoProcess.ini
Opening URL: http://localhost:8081/home/postprocess/processEpisode?nzbName=The.Strain.S01E12.720p.WEB-DL.DD5.1.H.264-CtrlHD.nzb&quiet=1&dir=%2FUsers%2F<USERNAME>%2FTV%2FThe+Strain+S01E12+720p+WEB-DL+DD5+1+H+264-CtrlHD
Processing folder: /Users/<USERNAME>/TV/The Strain S01E12 720p WEB-DL DD5 1 H 264-CtrlHD

Processing /Users/<USERNAME>/TV/The Strain S01E12 720p WEB-DL DD5 1 H 264-CtrlHD/The.Strain.S01E12.Last.Rites.720p.WEB-DL.DD5.1.H.264-CtrlHD.mp4 (The.Strain.S01E12.720p.WEB-DL.DD5.1.H.264-CtrlHD.nzb)

Found result in history: (276564, 1, [], 32)

Parsed The.Strain.S01E12.720p.WEB-DL.DD5.1.H.264-CtrlHD into The Strain - S1E12 - 720p.WEB-DL.DD5.1.H.264 (CtrlHD) [ABD: False]

Looking up The Strain in the DB

Lookup successful, using tvdb id 276564

Loading show object for tvdb_id 276564

Retrieving episode object for 1x12

Snatch history had a quality in it, using that: 720p WEB-DL

Sick Beard snatched this episode, marking it safe to replace

This download is marked as safe to replace existing file

Found release name The.Strain.S01E12.720p.WEB-DL.DD5.1.H.264-CtrlHD

Destination folder for this episode: /Users/<USERNAME>/TV/The Strain

Moving file from /Users/<USERNAME>/TV/The Strain S01E12 720p WEB-DL DD5 1 H 264-CtrlHD/The.Strain.S01E12.Last.Rites.720p.WEB-DL.DD5.1.H.264-CtrlHD.mp4 to /Users/gvantass/TV/The Strain/The Strain - 1x12 - Last Rites.mp4

Moving file from /Users/<USERNAME>/TV/The Strain S01E12 720p WEB-DL DD5 1 H 264-CtrlHD/The.Strain.S01E12.Last.Rites.720p.WEB-DL.DD5.1.H.264-CtrlHD.nfo to /Users/gvantass/TV/The Strain/The Strain - 1x12 - Last Rites.nfo-orig

Moving file from /Users/<USERNAME>/TV/The Strain S01E12 720p WEB-DL DD5 1 H 264-CtrlHD/The.Strain.S01E12.Last.Rites.720p.WEB-DL.DD5.1.H.264-CtrlHD.nzb to /Users/gvantass/TV/The Strain/The Strain - 1x12 - Last Rites.nzb

Moving file from /Users/<USERNAME>/TV/The Strain S01E12 720p WEB-DL DD5 1 H 264-CtrlHD/The.Strain.S01E12.Last.Rites.720p.WEB-DL.DD5.1.H.264-CtrlHD.srr to /Users/gvantass/TV/The Strain/The Strain - 1x12 - Last Rites.srr

Executing command ['/usr/bin/python', '/usr/local/etc/SAB_post_process/sickbeard_mp4_automator-master/postConversion.py', '/Users/<USERNAME>/TV/The Strain/The Strain - 1x12 - Last Rites.mp4', '/Users/<USERNAME>/TV/The Strain S01E12 720p WEB-DL DD5 1 H 264-CtrlHD/The.Strain.S01E12.Last.Rites.720p.WEB-DL.DD5.1.H.264-CtrlHD.mp4', '276564', '1', '12', '2014-09-28']

Script result: Tagging file :/Users/<USERNAME>/TV/The Strain/The Strain - 1x12 - Last Rites.mp4

Trying to write tags

Tags written successfully

Relocating MOOV atom to start of file

Temporary file /Users/<USERNAME>/TV/The Strain/The Strain - 1x12 - Last Rites.mp4 deleted

File moved to /Users/<USERNAME>/Music/iTunes/iTunes Media/Automatically Add to iTunes.localized

The Strain has queued to be refreshed

{}

success

Deleted folder: /Users/<USERNAME>/TV/The Strain S01E12 720p WEB-DL DD5 1 H 264-CtrlHD

Processing succeeded for /Users/<USERNAME>/TV/The Strain S01E12 720p WEB-DL DD5 1 H 264-CtrlHD/The.Strain.S01E12.Last.Rites.720p.WEB-DL.DD5.1.H.264-CtrlHD.mp4
mdhiggins commented 10 years ago

I think this is related to the tagging failing (index into long error). Are you 100% certain you're running in 64 bit python on those files that fail?

Update: I just updated the script with a little reporter that will print out a "32" or "64" at the beginning of the script so your log file will indicate which mode it is running in. Please send me a new log with this update.

Geordon commented 10 years ago

Will do. Being a Python idiot, I don't know how to either check for OR force 64-bit operation. Downloading the update.

Geordon commented 10 years ago

Ok, it looks like python is defaulting to 32-bit. I've updated my Python install to include 3.0 and 2.7.8 from Python.org. Trying to figure out how to force 64-bit mode on OS X.

mdhiggins commented 10 years ago

Just to catch you before you do a lot of work, this script does NOT work on Python 3 or above, it requires 2.7, so make sure that's what you install

Geordon commented 10 years ago

Ah, good to know. Adjusting fire. :D And thank you for your indulgence, I appreciate it. I really like the script, and think you did a damned fine job. I’m just out of my depth.

Geordon VanTassle about.me/hurricanedad411

On Sep 29, 2014, at 4:01 PM, Michael Higgins notifications@github.com wrote:

Just to catch you before you do a lot of work, this script does NOT work on Python 3 or above, it requires 2.7, so make sure that's what you install

— Reply to this email directly or view it on GitHub.

mdhiggins commented 10 years ago

No worries, we'll get it working eventually. If I used OSX more I would try to give you a hand but alas I don't do much Python work on the mac and don't know how to fix that off the top of my head.

Geordon commented 10 years ago

Woohoo! I think I finally got Python to default to 64-bit. Short answer, it looked like things were defaulting to Python 2.3 or 2.5, which are both 32-bit. I've had this Mac for a couple of versions of OS X :) . Running a manual process now, details and log to follow.

Geordon commented 10 years ago 
python /usr/local/etc/SAB_post_process/sickbeard_mp4_automator-master/manual.py -i ./Game\ of\ Thrones\ -\ 3x01\ -\ Valar\ Dohaeris.mp4 -a
64
Matched TV episode as Game of Thrones (TVDB ID:121361) S03E01
Processing Game of Thrones Season 03 Episode 01 - Valar Dohaeris
Tagging file :./Game of Thrones - 3x01 - Valar Dohaeris.mp4
Trying to write tags
Tags written successfully
Relocating MOOV atom to start of file
File moved to /Users/gvantass/Music/iTunes/iTunes Media/Automatically Add to iTunes.localized

Thank you so much for helping me get this figured out! As for the initial problem, it looks like there is an issue with SB and the "enable TV sorting" option in SAB, at least that's what I changed back to default and it started processing again. SB didn't like the "Enable TV Sorting" option being turned on.

mdhiggins commented 10 years ago

Glad to hear everything is working. Lemme know if you run into any further issues