search

clinton-hall / nzbToMedia

Provides NZB and Torrent postprocessing To CouchPotatoServer, SickBeard/SickRage, HeadPhones, Mylar and Gamez
GNU General Public License v3.0
672 stars 176 forks source link

New Function Request #185

Closed Jorman closed 10 years ago

Jorman commented 11 years ago

Hi, I've a little request, I think that is more personal, but maybe is usefull for another people.

I made a little script that in combination with watcher, monitorize my "various" download folder

So, for example, I call the script for each emule (uncategorized) files ... and for "uncategorized" torrent

So it's possible to add a function like this:

[user_script]

Use user_script for uncategorized download?

Note, is better that the default torrent download dir is located in a different place

user_script = 1

Specify the path of the script

user_script_path = /media/test/script/script.sh

Specify the argument passed to script

for example $FN for full name with path

So the result is /media/test/script/script.sh $FN

user_script_param = $FN

Clean after? Note that delay function is used to prevent possible mistake :) Delay is intended as seconds

user_script_clean = 1 delay = 120

So, only for uncategorized download, the script process the files like a normal use, after that call another "user" script and pass the full path After the delay time it check if something is changed, if not exit elif all the "processed files" are processed, up one level and delete the folder that is used for processing purpose (you know, like now)

What do you think about it?

J

clinton-hall commented 11 years ago

This sounds logical, effective and rather simple.

It is just a requirement that when it then calls the script (for categorized downloads) it needs to pass through the appropriate variables.

Did you want to try and add that script into this repo, or just link it here? Or are you wanting me to try and write a script to do this?

Jorman commented 11 years ago

This sounds logical, effective and rather simple.

It is just a requirement that when it then calls the script (for categorized downloads) it needs to pass through the appropriate variables.

Did you want to try and add that script into this repo, or just link it here? Or are you wanting me to try and write a script to do this?

Tnx, I think that is better to "apply" at the uncategorized download, I mean CP HP SB and others are already managed by your script, uncategorized don't. So for all the other downloads :)

About the variables I think that the fullpath, comprensive the name of the file are more than sufficent An example dir structure is that: /media/download/torrent/music /media/download/torrent/movie /media/download/torrent/series /media/download/torrent/everythingelse

The default torrent location is: /media/download/torrent/everythingelse

Every program can force a downloading dir, so the script process only the "uncategorized" files, so the one in the /media/download/torrent/everythingelse

For the development, like I said, I can read the code but I'm not able to programm :( So if you think that is a good option to have, when you've time, maybe can you implement it :)

Btw, what program I need to use to start "programming" in that way?

J

clinton-hall commented 11 years ago

Ok.. for "programming" you can fork this on github and then edit files within your web browser. But testing is not easy (as you have to git pull all changes).

So usually it is easiest to edit files on your local repository and then do a git push once you ahve the code working.

As far as tools to edit locally. well on a NAS etc I use WinSCP.to edit over ssh. you can also use ssh and then nano or vi to edit.

If you want to use a more visual interface to edit the files, Sublime Text 2 is a great bit of software that highlites code as you go and shows when the code is structured correctly....

Now, As far as the script goes, I can do most of this, but I'm not sure what you want to happen before and after the other script is called....

Do you want this to do the same copy/link and extract to the outputDirectory (or another directory) and flattened? Once the external script is called what changes are we looking for before commencing cleanup?

Jorman commented 11 years ago

Now, As far as the script goes, I can do most of this, but I'm not sure what you want to happen before and after the other script is called....

Do you want this to do the same copy/link and extract to the outputDirectory (or another directory) and flattened? Once the external script is called what changes are we looking for before commencing cleanup?

Tnx, I'll try!

For the implementatio, my goal is "simple" implement nzbtomedia and manage all the download that is manually added :) Not even, CP SB HP, find the download, sometimes we had to add it manually, or we decide to download a film or other that is not on "list" So until now, I use a simple method, I watch every single files on uncategorized directory and if is a multimedia files I lanuch filebot and I process every single file

I try to explain what I think must be the external script implementation On configuration side: [user_script]

Use user_script for uncategorized download?

Note, is better that the default torrent download dir is located in a different place

user_script = 1

Do you want to menage compressed files?

user_manage_compressed = 1

What extension do you want to process? Specify all the extension, or use 1 to ALL

user_script_mediaExtensions = .mkv,.avi,.divx,.xvid,.mov,.wmv,.mp4,.mpg,.mpeg

Specify the path of the script

user_script_path = /media/test/script/script.sh

Specify the argument passed to script

for example $FN for full name with path

So the result is /media/test/script/script.sh $FN

user_script_param = $FN

Clean after? Note that delay function is used to prevent possible mistake :) Delay is intended as seconds

user_script_clean = 1 delay = 120

Let's now make some example, ther's 3 possibility

  1. Donwload a single file
  2. Download multiple files
  3. Download compressed file/s

On every possibility I assume that the user script is activated and configurated like above. I think that is better to use the default procedure, so use the outputfolder, in move mode

  1. I manually add, to torrent, a file called TEST.avi So once is download, the script check the file, is a media file, is not compressed so move on outputfolder and call the user_script, the output will be something like that ./media/test/script/script.sh /nzbtomedia_outputfolder/TEST.avi After 120 sec, if the file is still present - move back the file on original folder and quit
  2. I manually add, to torrent, that contains multiple files So once is download, the script move the entire folder, flatten the folder, check the file, is there is a media file call the user_script, the output will be something like that ./media/test/script/script.sh /nzbtomedia_outputfolder/folderdownloadedname/1.avi ./media/test/script/script.sh /nzbtomedia_outputfolder/folderdownloadedname/2.mkv ./media/test/script/script.sh /nzbtomedia_outputfolder/folderdownloadedname/3.mp4 After 120 sec, if the files is still present - move back the remaining media files on original folder, if not clean up all and exit
  3. I manually add, to torrent, a file called TEST.zip Is all similar at the point 2. but before the script extract all in a folder, in this case is name is TEST

Is everighing right?

J

Jorman commented 11 years ago

Hi, do you have interest / time to help me implement that? I'm trying :) for now I've put only the settings, the hardest part is to write down the autoProcessUserScript.py LOL

J

clinton-hall commented 11 years ago

I do have the interest, the time is what I struggle with.

I'll try and set up a new branch with these options. I'll code in the autoprocess side and set up a call to the external script...

Jorman commented 11 years ago

No rush :)

Tell me if I can help somehow!

clinton-hall commented 11 years ago

ok... slightly different, but hopefully this works...

My script always did extract and link files for all categories (including "no-category").

Now, this allows you to specify what categories to launch the external script for. These are entered as comma separated list, OR you can USE "ALL" and "UNCAT" where UNCAT is used for no-defined category. To disable this external script (default) this is set to "NONE".

The cleanup will only happen if the script returns successful, the number of files with the process extension is 0 after the delay period, and the option to cleanup is set.

I hope this performs as you want... There is no-doubt some cleanup of this code required... please test this from the dev-TPB branch (I pushed to this one by mistake, but I'd rather keep this level of change away from the main dev branch anyway... I'll merge as soon as it is verified to be working.

Jorman commented 11 years ago

Ok, many many tnx, I'll try soon your mod :)

clinton-hall commented 11 years ago

ok... I just made a small change... I did a test in the wrong order, so this external script would have been skipped... in theory it should work now.

Jorman commented 11 years ago

I'm setting up the dev-TPB to test the external script. Only one question, if I set ALL, for the script, what happens?

I mean what is the 1st script called? I'm not sure that is good for categorized download :P

clinton-hall commented 11 years ago

If you select all, it will process for un-categorized and non HP, CP, SB, MY, GZ categories.

For CP, SB, GZ, MY, HP it will still call the usual autoprocess script.

Jorman commented 11 years ago

ok, so I don't really understand the difference between ALL and UNCAT

clinton-hall commented 11 years ago

So ALL = UNCAT + random = everything - (HP, MY, GZ, SB, CP) UNCAT is used here to shown when there is no category. random could be any other category e.g pictures, software etc...

Depending on how you have your categories set up, ALL and UNCAT may be the same, but I tried to build in options.

Not sure if it is the same with Transmission and uTorrent etc, but with NZBGet, if I set my newznab site to send an nzb it actually sends it with a category that isn't defined in my system... E.g it sends HD-TV where I only have category TV set up.

Jorman commented 11 years ago

Ok, perfect, now I understand!

I'll try and I tell you my experiment :)

Jorman commented 11 years ago

09:18:44|INFO TorrentToMedia V8.6 09:18:44|INFO MAIN: Loading config from /home/xbmc/scripts/nzbToMedia/autoProcessMedia.cfg 09:18:44|DEBUG arg 0 is: /home/xbmc/scripts/nzbToMedia/TorrentToMedia.py 09:18:44|DEBUG MAIN: Received Directory: /media/TwoTera/Download/torrent/vari | Name: The.Cloth.2013.iTALiAN.Subbed.HDRiP.XViD.NeWZoNe.avi | Category: 09:18:44|ERROR SEARCH: Could not identify category and torrent name from the directory structure. Please check downloader settings. Exiting

Something wrong ...

clinton-hall commented 11 years ago

sorry about that. I was bound to have missed something.

Please update and try again.

Jorman commented 11 years ago

Lol, not a problem :)

Jorman commented 11 years ago

Here the log: 16:17:04|INFO TorrentToMedia V8.6 16:17:04|INFO MAIN: Loading config from /home/xbmc/scripts/nzbToMedia/autoProcessMedia.cfg 16:17:04|DEBUG arg 0 is: /home/xbmc/scripts/nzbToMedia/TorrentToMedia.py 16:17:04|DEBUG MAIN: Received Directory: /media/TwoTera/Download/torrent/vari | Name: Zombie.Massacre.2013.iTALiAN.AC3.1080p.BluRay.x264.TrTd_TeaM | Category: 16:17:04|INFO SEARCH: Could not identify Category or Torrent Name from the directory structure. 16:17:04|INFO SEARCH: We assume the directory passed is the root directory for your downlaoder 16:17:04|WARNING SEARCH: You should change settings to download torrents to their own directory if possible 16:17:04|INFO SEARCH: We will try and determine which files to process, individually 16:17:04|INFO SEARCH: Files will be linked and will only be processed by the userscript if enabled for UNCAT or ALL 16:17:04|DEBUG MAIN: Scanning files in directory: /media/TwoTera/Download/torrent/vari

Configuration: [UserScript]

Use user_script for uncategorized download?

Set the categories to use external script, comma separated.

Use "UNCAT" to process non-category downloads, and "ALL" for all. Set to "NONE" to disable external script.

user_script_categories = ALL

What extension do you want to process? Specify all the extension, or use "ALL" to process all files.

user_script_mediaExtensions = .mkv,.avi,.divx,.xvid,.mov,.wmv,.mp4,.mpg,.mpeg

Specify the path of the script

user_script_path = /home/xbmc/scripts/filebot_script.sh

Specify the argument(s) passed to script, comma separated in order.

for example FP,FN,DN for file path (absolute file name with path), file anme, absolute directory name (with path).

So the result is /media/test/script/script.sh FP FN DN

user_script_param = FP

Clean after? Note that delay function is used to prevent possible mistake :) Delay is intended as seconds

user_script_clean = 1 delay = 120

I've some consideration, let me know if we think the same

  1. I think that if the complete download is a single file, the script can run the immediately (obviously if is a filtered file)
  2. If the download is a dir, the working dir must be the downloaded dir, I take the log as example so the working dir is /media/TwoTera/Download/torrent/vari/Zombie.Massacre.2013.iTALiAN.AC3.1080p.BluRay.x264.TrTd_TeaM Now, if everything goes as it should, the script up one level and delete the Zombie.Massacre.2013.iTALiAN.AC3.1080p.BluRay.x264.TrTd_TeaM dir I'm referring to that: 16:17:04|DEBUG MAIN: Scanning files in directory: /media/TwoTera/Download/torrent/vari

In that way there's no problem to cleaning up, and no problem if I use the root torrent dir without make another dir for uncat download :)

J

Jorman commented 11 years ago

Another little issue with the nzb and sickbeard ...

00:01:15|INFO nzbToSickBeard V8.6 00:01:15|INFO Script triggered from SABnzbd, starting autoProcessTV... 00:01:15|INFO Loading config from /home/xbmc/scripts/nzbToMedia/autoProcessMedia.cfg 00:01:15|INFO The download succeeded. Sending process request to SickBeard's TPB branch 00:01:15|INFO Waiting for 65.0 seconds to allow SB to process newly extracted files 00:02:20|DEBUG Opening URL: http://localhost:8081/home/postprocess/processEpisode?nzbName=The.X.Factor.UK.S10E17.720p.HDTV.x264-FTP.nzb&quiet=1&dir=%2Fmedia%2FTwoTera%2FDownload%2Fnzb%2Fserietv%2FThe_X_Factor_UK_S10E17_720p_HDTV_x264-FTP 00:02:20|INFO 00:02:20|INFO 00:02:20|INFO 00:02:20|INFO 404 00:02:20|INFO 00:02:20|INFO 00:02:20|INFO 00:02:20|INFO
00:02:20|INFO 00:02:20|INFO 00:02:20|INFO 00:02:20|INFO MAIN: The autoProcessTV script completed successfully.

Maybe I've mis-configured something, this is my configuration, I use the schumi2004 fork [SickBeard]

autoProcessing for TV Series

sbCategory - category that gets called for post-processing with SB

sbCategory = serietv host = localhost port = 8081 username = password =

ADVANCED USE - ONLY EDIT IF YOU KNOW WHAT YOU'RE DOING

web_root = ssl = 0 delay = 65 watch_dir = fork = TPB delete_failed = 1

Ps, there is a way to "re-force" the scan process on completed download?

clinton-hall commented 11 years ago

for schumi2004 fork set fork = failed

NZBGet supports re-postprocessing, but SABnzbd does not. to run the renamer you will need to call nzbToSickBeard and then pass in the arguments in order... personally I would just use SickBeard's manual postprocess.

Did Zombie.Massacre complete correctly, or did the logs end as shown above?

Jorman commented 11 years ago

Tnx ...

Unfortunately manual post-process don't works :( I'll move on my watched folder

Zombie was completed succeffully

clinton-hall commented 11 years ago

Manual postprocess doesn't work?

This is the postpocess called from within the. SickBeard Web UI? Home, manual Post-Processing. Then browse to the folder and hit "Process"

So the call to external script works... You are just suggesting a quicker/cleaner method of processing single files and directories?

Jorman commented 11 years ago

Don't works, I see that there's already an issue on shumi fork, but I read that he stop develop this version, I think I switch on mr-orange version, in that case fork = failed, right?

The call to external script, unless I've done something wrong, don't seems to works, but I don't have more information, the logs don't tell other info, the last line for the log is MAIN: Scanning files in directory: /media/TwoTera/Download/torrent/vari

And yes, I also suggest a method to process single files and directories.

J

clinton-hall commented 11 years ago

mr-orange fork of SickBeard has a few different branches, but the default branch is TPB... so autoProcessMedia.cfg [SickBeard] fork = TPB

The external script section should in theory be called now with these last changes.

I have added the method to process directory, and it will check for existence of a single file, depending on the information that is passed to the script.

Jorman commented 11 years ago

I switched up to mr-orange and set TPB on settings script. Now I'll try and I tell you.

Tnx, J

Jorman commented 11 years ago

Nope, I report some error:

  1. No logs are made, so no postprocess
  2. Music works, but the torrent is not stopped and deleted
  3. User script not called, but maybe is only my fault about configuration

ps I need to use user_script_categories = ALL or user_script_categories = "ALL"

clinton-hall commented 11 years ago

use user_script_categories = ALL

are you saying there are no logfiles being produced at all?

Jorman commented 11 years ago

Ok, I've the right configuration, and yep, no logs at all

clinton-hall commented 11 years ago

ok... no logs were a result of these two errors... try again now... sorry.

Jorman commented 11 years ago

Ok, I update and I try again :)

clinton-hall commented 11 years ago

For future reference. No log at all means the script failed to run.... This is usually the result of a python formatting error (I.e. My stuff up) :D

Jorman commented 11 years ago

Ok, postprocess.log is back!

For UserScript: Nothing happens, the log stop at scanning files

06:18:08|INFO TorrentToMedia V8.6 06:18:08|INFO MAIN: Loading config from /home/xbmc/scripts/nzbToMedia/autoProcessMedia.cfg 06:18:08|DEBUG arg 0 is: /home/xbmc/scripts/nzbToMedia/TorrentToMedia.py 06:18:08|DEBUG MAIN: Received Directory: /media/TwoTera/Download/torrent/vari | Name: Red.2.2013.iTALiAN.MD.DUAL.HDSCR.480p.x264-TrTd_TeaM | Category: 06:18:08|INFO SEARCH: Found torrent directory Red.2.2013.iTALiAN.MD.DUAL.HDSCR.480p.x264-TrTd_TeaM in input directory directory /media/TwoTera/Download/torrent/vari 06:18:08|INFO SEARCH: Setting inputDirectory to /media/TwoTera/Download/torrent/vari/Red.2.2013.iTALiAN.MD.DUAL.HDSCR.480p.x264-TrTd_TeaM 06:18:08|INFO SEARCH: Files appear to be in their own directory 06:18:08|DEBUG SEARCH: Continuing scan to determin category. 06:18:08|INFO SEARCH: Could not find a category in the directory structure 06:18:08|INFO SEARCH: Files will be linked and will only be processed by the userscript if enabled for UNCAT or ALL 06:18:08|DEBUG MAIN: Scanning files in directory: /media/TwoTera/Download/torrent/vari/Red.2.2013.iTALiAN.MD.DUAL.HDSCR.480p.x264-TrTd_TeaM

This my configuration for UserScript [UserScript]

Use user_script for uncategorized download?

Set the categories to use external script, comma separated.

Use "UNCAT" to process non-category downloads, and "ALL" for all. Set to "NONE" to disable external script.

user_script_categories = ALL

What extension do you want to process? Specify all the extension, or use "ALL" to process all files.

user_script_mediaExtensions = .mkv,.avi,.divx,.xvid,.mov,.wmv,.mp4,.mpg,.mpeg

Specify the path of the script

user_script_path = /home/xbmc/scripts/filebot_script.sh

Specify the argument(s) passed to script, comma separated in order.

for example FP,FN,DN for file path (absolute file name with path), file anme, absolute directory name (with path).

So the result is /media/test/script/script.sh FP FN DN

user_script_param = FP

Clean after? Note that delay function is used to prevent possible mistake :) Delay is intended as seconds

user_script_clean = 1 delay = 120

For HP and CP: Now the torrent is deleted, only one strange behavior on postprocess.log, multiple line that say the same :)

04:13:21|INFO TorrentToMedia V8.6 04:13:21|INFO MAIN: Loading config from /home/xbmc/scripts/nzbToMedia/autoProcessMedia.cfg 04:13:21|DEBUG arg 0 is: /home/xbmc/scripts/nzbToMedia/TorrentToMedia.py 04:13:21|DEBUG MAIN: Received Directory: /media/TwoTera/Download/torrent/film | Name: Dark.Skies.Oscure.Presenze.2013.iTALiAN.MD.DUAL.1080p.BrRiP.x264-TrTd_TeaM | Category: 04:13:21|INFO SEARCH: Found torrent directory Dark.Skies.Oscure.Presenze.2013.iTALiAN.MD.DUAL.1080p.BrRiP.x264-TrTd_TeaM in input directory directory /media/TwoTera/Download/torrent/film 04:13:21|INFO SEARCH: Setting inputDirectory to /media/TwoTera/Download/torrent/film/Dark.Skies.Oscure.Presenze.2013.iTALiAN.MD.DUAL.1080p.BrRiP.x264-TrTd_TeaM 04:13:21|INFO SEARCH: Files appear to be in their own directory 04:13:21|DEBUG SEARCH: Continuing scan to determin category. 04:13:21|DEBUG SEARCH: Found Category: film in directory structure 04:13:21|INFO SEARCH: Determined Category to be: film 04:13:21|DEBUG SEARCH: Found Category: film in directory structure 04:13:21|DEBUG SEARCH: Found Category: film in directory structure 04:13:21|DEBUG SEARCH: Found Category: film in directory structure 04:13:21|DEBUG SEARCH: Found Category: film in directory structure 04:13:21|DEBUG SEARCH: Found Category: film in directory structure 04:13:21|DEBUG SEARCH: Found Category: film in directory structure 04:13:21|DEBUG SEARCH: Found Category: film in directory structure 04:13:21|DEBUG SEARCH: Found Category: film in directory structure 04:13:21|INFO MAIN: Download is a directory 04:13:21|DEBUG MAIN: Scanning files in directory: /media/TwoTera/Download/torrent/film/Dark.Skies.Oscure.Presenze.2013.iTALiAN.MD.DUAL.1080p.BrRiP.x264-TrTd_TeaM 04:13:21|DEBUG MAIN: Ignoring unknown filetype .txt for file /media/TwoTera/Download/torrent/film/Dark.Skies.Oscure.Presenze.2013.iTALiAN.MD.DUAL.1080p.BrRiP.x264-TrTd_TeaM/TrTd_TeaM.info.txt 04:13:21|INFO MAIN: Found video file .mkv in /media/TwoTera/Download/torrent/film/Dark.Skies.Oscure.Presenze.2013.iTALiAN.MD.DUAL.1080p.BrRiP.x264-TrTd_TeaM/Dark.Skies.Oscure.Presenze.2013.iTALiAN.MD.DUAL.1080p.BrRiP.x264-TrTd_TeaM.mkv 04:13:21|INFO CREATE DESTINATION: Creating destination folder: /media/TwoTera/Download/torrent/tmp/film/Dark.Skies.Oscure.Presenze.2013.iTALiAN.MD.DUAL.1080p.BrRiP.x264-TrTd_TeaM 04:13:21|DEBUG Moving /media/TwoTera/Download/torrent/film/Dark.Skies.Oscure.Presenze.2013.iTALiAN.MD.DUAL.1080p.BrRiP.x264-TrTd_TeaM/Dark.Skies.Oscure.Presenze.2013.iTALiAN.MD.DUAL.1080p.BrRiP.x264-TrTd_TeaM.mkv to /media/TwoTera/Download/torrent/tmp/film/Dark.Skies.Oscure.Presenze.2013.iTALiAN.MD.DUAL.1080p.BrRiP.x264-TrTd_TeaM/Dark.Skies.Oscure.Presenze.2013.iTALiAN.MD.DUAL.1080p.BrRiP.x264-TrTd_TeaM.mkv 04:13:21|INFO FLATTEN: Flattening directory: /media/TwoTera/Download/torrent/tmp/film/Dark.Skies.Oscure.Presenze.2013.iTALiAN.MD.DUAL.1080p.BrRiP.x264-TrTd_TeaM 04:13:21|INFO REMOVER: Removing empty folders in: /media/TwoTera/Download/torrent/tmp/film/Dark.Skies.Oscure.Presenze.2013.iTALiAN.MD.DUAL.1080p.BrRiP.x264-TrTd_TeaM 04:13:21|DEBUG MAIN: Connecting to transmission: http://localhost:9091 04:13:21|DEBUG MAIN: Stoping torrent Dark.Skies.Oscure.Presenze.2013.iTALiAN.MD.DUAL.1080p.BrRiP.x264-TrTd_TeaM in transmission while processing 04:13:26|DEBUG MAIN: Calling autoProcess script for successful download. 04:13:26|INFO MAIN: Calling CouchPotatoServer to post-process: Dark.Skies.Oscure.Presenze.2013.iTALiAN.MD.DUAL.1080p.BrRiP.x264-TrTd_TeaM 04:13:26|INFO Loading config from /home/xbmc/scripts/nzbToMedia/autoProcessMedia.cfg 04:13:26|WARNING Could not find an imdb id in directory or name 04:13:26|INFO Postprocessing will continue, but the movie may not be identified correctly by CouchPotato 04:13:26|DEBUG Opening URL: http://localhost:5050/api/28ceb7bbfab54dd6854d19b5d6e7d442/movie.list/?status=active 04:13:26|DEBUG Opening URL: http://localhost:5050/api/28ceb7bbfab54dd6854d19b5d6e7d442/movie.get/?id=80 04:13:26|DEBUG Opening URL: http://localhost:5050/api/28ceb7bbfab54dd6854d19b5d6e7d442/movie.get/?id=27 04:13:26|DEBUG Opening URL: http://localhost:5050/api/28ceb7bbfab54dd6854d19b5d6e7d442/movie.get/?id=15 04:13:27|DEBUG Opening URL: http://localhost:5050/api/28ceb7bbfab54dd6854d19b5d6e7d442/movie.get/?id=157 04:13:27|INFO Found movie id 157 in database via download_id fb9fecee5046a88a443d944c7044b9423e37e0c1 04:13:27|DEBUG Looking for status of movie: 157 - with release sent to clientAgent: transmission and download_id: fb9fecee5046a88a443d944c7044b9423e37e0c1 04:13:27|DEBUG Opening URL: http://localhost:5050/api/28ceb7bbfab54dd6854d19b5d6e7d442/movie.get/?id=157 04:13:27|DEBUG This movie is marked as status active in CouchPotatoServer 04:13:27|DEBUG Found a single release with download_id: fb9fecee5046a88a443d944c7044b9423e37e0c1 for clientAgent: Transmission. Release status is: snatched 04:13:27|INFO Waiting for 65.0 seconds to allow CPS to process newly extracted files 04:14:32|DEBUG Opening URL: http://localhost:5050/api/28ceb7bbfab54dd6854d19b5d6e7d442/renamer.scan/?movie_folder=/media/TwoTera/Download/torrent/tmp/film/Dark.Skies.Oscure.Presenze.2013.iTALiAN.MD.DUAL.1080p.BrRiP.x264-TrTd_TeaM&downloader=Transmission&download_id=fb9fecee5046a88a443d944c7044b9423e37e0c1 04:14:38|INFO CouchPotatoServer returned {u'success': True} 04:14:38|INFO renamer scan started on CouchPotatoServer for Dark.Skies.Oscure.Presenze.2013.iTALiAN.MD.DUAL.1080p.BrRiP.x264-TrTd_TeaM 04:14:38|DEBUG Looking for status of movie: 157 - with release sent to clientAgent: Transmission and download_id: fb9fecee5046a88a443d944c7044b9423e37e0c1 04:14:38|DEBUG Opening URL: http://localhost:5050/api/28ceb7bbfab54dd6854d19b5d6e7d442/movie.get/?id=157 04:14:38|DEBUG This movie is marked as status done in CouchPotatoServer 04:14:38|DEBUG Found a single release with download_id: fb9fecee5046a88a443d944c7044b9423e37e0c1 for clientAgent: Transmission. Release status is: downloaded 04:14:38|INFO SUCCESS: This movie is now marked as status done in CouchPotatoServer 04:14:38|DEBUG MAIN: Deleting torrent Dark.Skies.Oscure.Presenze.2013.iTALiAN.MD.DUAL.1080p.BrRiP.x264-TrTd_TeaM from transmission 04:14:43|INFO All files have been processed. Cleaning outputDirectory /media/TwoTera/Download/torrent/tmp/film/Dark.Skies.Oscure.Presenze.2013.iTALiAN.MD.DUAL.1080p.BrRiP.x264-TrTd_TeaM 04:14:43|INFO MAIN: All done.

20:32:42|INFO TorrentToMedia V8.6 20:32:42|INFO MAIN: Loading config from /home/xbmc/scripts/nzbToMedia/autoProcessMedia.cfg 20:32:42|DEBUG arg 0 is: /home/xbmc/scripts/nzbToMedia/TorrentToMedia.py 20:32:42|DEBUG MAIN: Received Directory: /media/TwoTera/Download/torrent/musica | Name: Nickelback - Dark Horse [2008][CD+SkidVid_XviD+Cov]320Kbps | Category: 20:32:42|INFO SEARCH: Found torrent directory Nickelback - Dark Horse [2008][CD+SkidVid_XviD+Cov]320Kbps in input directory directory /media/TwoTera/Download/torrent/musica 20:32:42|INFO SEARCH: Setting inputDirectory to /media/TwoTera/Download/torrent/musica/Nickelback - Dark Horse [2008][CD+SkidVid_XviD+Cov]320Kbps 20:32:42|INFO SEARCH: Files appear to be in their own directory 20:32:42|DEBUG SEARCH: Continuing scan to determin category. 20:32:42|DEBUG SEARCH: Found Category: musica in directory structure 20:32:42|INFO SEARCH: Determined Category to be: musica 20:32:42|DEBUG SEARCH: Found Category: musica in directory structure 20:32:42|DEBUG SEARCH: Found Category: musica in directory structure 20:32:42|DEBUG SEARCH: Found Category: musica in directory structure 20:32:42|DEBUG SEARCH: Found Category: musica in directory structure 20:32:42|DEBUG SEARCH: Found Category: musica in directory structure 20:32:42|DEBUG SEARCH: Found Category: musica in directory structure 20:32:42|DEBUG SEARCH: Found Category: musica in directory structure 20:32:42|DEBUG SEARCH: Found Category: musica in directory structure 20:32:42|DEBUG MAIN: Scanning files in directory: /media/TwoTera/Download/torrent/musica/Nickelback - Dark Horse [2008][CD+SkidVid_XviD+Cov]320Kbps 20:32:42|DEBUG MAIN: Connecting to transmission: http://localhost:9091 20:32:42|DEBUG MAIN: Stoping torrent Nickelback - Dark Horse [2008][CD+SkidVid_XviD+Cov]320Kbps in transmission while processing 20:32:47|DEBUG MAIN: Calling autoProcess script for successful download. 20:32:47|INFO MAIN: Calling HeadPhones to post-process: Nickelback - Dark Horse [2008][CD+SkidVid_XviD+Cov]320Kbps 20:32:47|INFO Loading config from /home/xbmc/scripts/nzbToMedia/autoProcessMedia.cfg 20:32:47|INFO Waiting for 65.0 seconds to allow HeadPhones to process newly extracted files 20:33:52|DEBUG Opening URL: http://localhost:8181/api?apikey=26acff2128eb5a839f8efd58b73ea040&cmd=forceProcess 20:33:52|INFO HeaPhones returned ['OK'] 20:33:52|INFO forceProcess started on HeadPhones for Nickelback - Dark Horse [2008][CD+SkidVid_XviD+Cov]320Kbps 20:34:52|INFO This album should have completed processing. Please check HeadPhones Logs 20:34:52|DEBUG MAIN: Deleting torrent Nickelback - Dark Horse [2008][CD+SkidVid_XviD+Cov]320Kbps from transmission 20:34:57|INFO MAIN: All done.

J

clinton-hall commented 11 years ago

Ok... I (should have) stopped the constant looping and repeating the same line in logs as shown for music and film category.

Also, I discovered that the outputDirectory was not being correctly assigned when no category! hence the script would exit with no error when no category.... This is now fixed and hopefully getting closer to calling the user script.

So many little issues when trying to add a simple external script. ;)

Jorman commented 11 years ago

Let's try :)

Jorman commented 11 years ago

Ok, I'm here :)

So this's the log 17:28:01|INFO TorrentToMedia V8.6 17:28:01|INFO MAIN: Loading config from /home/xbmc/scripts/nzbToMedia/autoProcessMedia.cfg 17:28:01|DEBUG arg 0 is: /home/xbmc/scripts/nzbToMedia/TorrentToMedia.py 17:28:01|DEBUG MAIN: Received Directory: /media/TwoTera/Download/torrent/vari | Name: L.Evocazione.The.Conjuring.2013.iTALiAN.AC3.DUAL.1080p.BrRiP.x264-TrTd_TeaM | Category: 17:28:01|INFO SEARCH: Found torrent directory L.Evocazione.The.Conjuring.2013.iTALiAN.AC3.DUAL.1080p.BrRiP.x264-TrTd_TeaM in input directory directory /media/TwoTera/Download/torrent/vari 17:28:01|INFO SEARCH: Setting inputDirectory to /media/TwoTera/Download/torrent/vari/L.Evocazione.The.Conjuring.2013.iTALiAN.AC3.DUAL.1080p.BrRiP.x264-TrTd_TeaM 17:28:01|INFO SEARCH: Files appear to be in their own directory 17:28:01|DEBUG SEARCH: Continuing scan to determin category. 17:28:01|INFO SEARCH: Could not find a category in the directory structure 17:28:01|INFO SEARCH: Files will be linked and will only be processed by the userscript if enabled for UNCAT or ALL 17:28:01|INFO MAIN: Download is a directory 17:28:01|INFO MAIN: Output directory set to: /media/TwoTera/Download/torrent/tmp/L.Evocazione.The.Conjuring.2013.iTALiAN.AC3.DUAL.1080p.BrRiP.x264-TrTd_TeaM 17:28:01|DEBUG MAIN: Scanning files in directory: /media/TwoTera/Download/torrent/vari/L.Evocazione.The.Conjuring.2013.iTALiAN.AC3.DUAL.1080p.BrRiP.x264-TrTd_TeaM 17:28:01|DEBUG MAIN: Looking for L.Evocazione.The.Conjuring.2013.iTALiAN.AC3.DUAL.1080p.BrRiP.x264-TrTd_TeaM in: TrTd_TeaM.info.txt 17:28:01|DEBUG MAIN: Looking for L.Evocazione.The.Conjuring.2013.iTALiAN.AC3.DUAL.1080p.BrRiP.x264-TrTd_TeaM in: L.Evocazione.The.Conjuring.2013.iTALiAN.AC3.DUAL.1080p.BrRiP.x264-TrTd_TeaM.nfo 17:28:01|DEBUG MAIN: Found file L.Evocazione.The.Conjuring.2013.iTALiAN.AC3.DUAL.1080p.BrRiP.x264-TrTd_TeaM.nfo that matches Torrent Name L.Evocazione.The.Conjuring.2013.iTALiAN.AC3.DUAL.1080p.BrRiP.x264-TrTd_TeaM 17:28:01|INFO MAIN: Found file /media/TwoTera/Download/torrent/vari/L.Evocazione.The.Conjuring.2013.iTALiAN.AC3.DUAL.1080p.BrRiP.x264-TrTd_TeaM/L.Evocazione.The.Conjuring.2013.iTALiAN.AC3.DUAL.1080p.BrRiP.x264-TrTd_TeaM.nfo for category 17:28:01|INFO CREATE DESTINATION: Creating destination folder: /media/TwoTera/Download/torrent/tmp/L.Evocazione.The.Conjuring.2013.iTALiAN.AC3.DUAL.1080p.BrRiP.x264-TrTd_TeaM 17:28:01|DEBUG Moving /media/TwoTera/Download/torrent/vari/L.Evocazione.The.Conjuring.2013.iTALiAN.AC3.DUAL.1080p.BrRiP.x264-TrTd_TeaM/L.Evocazione.The.Conjuring.2013.iTALiAN.AC3.DUAL.1080p.BrRiP.x264-TrTd_TeaM.nfo to /media/TwoTera/Download/torrent/tmp/L.Evocazione.The.Conjuring.2013.iTALiAN.AC3.DUAL.1080p.BrRiP.x264-TrTd_TeaM/L.Evocazione.The.Conjuring.2013.iTALiAN.AC3.DUAL.1080p.BrRiP.x264-TrTd_TeaM.nfo 17:28:01|DEBUG MAIN: Found file L.Evocazione.The.Conjuring.2013.iTALiAN.AC3.DUAL.1080p.BrRiP.x264-TrTd_TeaM.mkv that matches Torrent Name L.Evocazione.The.Conjuring.2013.iTALiAN.AC3.DUAL.1080p.BrRiP.x264-TrTd_TeaM 17:28:01|INFO MAIN: Found file /media/TwoTera/Download/torrent/vari/L.Evocazione.The.Conjuring.2013.iTALiAN.AC3.DUAL.1080p.BrRiP.x264-TrTd_TeaM/L.Evocazione.The.Conjuring.2013.iTALiAN.AC3.DUAL.1080p.BrRiP.x264-TrTd_TeaM.mkv for category 17:28:01|DEBUG Moving /media/TwoTera/Download/torrent/vari/L.Evocazione.The.Conjuring.2013.iTALiAN.AC3.DUAL.1080p.BrRiP.x264-TrTd_TeaM/L.Evocazione.The.Conjuring.2013.iTALiAN.AC3.DUAL.1080p.BrRiP.x264-TrTd_TeaM.mkv to /media/TwoTera/Download/torrent/tmp/L.Evocazione.The.Conjuring.2013.iTALiAN.AC3.DUAL.1080p.BrRiP.x264-TrTd_TeaM/L.Evocazione.The.Conjuring.2013.iTALiAN.AC3.DUAL.1080p.BrRiP.x264-TrTd_TeaM.mkv 17:28:01|INFO FLATTEN: Flattening directory: /media/TwoTera/Download/torrent/tmp/L.Evocazione.The.Conjuring.2013.iTALiAN.AC3.DUAL.1080p.BrRiP.x264-TrTd_TeaM 17:28:01|INFO REMOVER: Removing empty folders in: /media/TwoTera/Download/torrent/tmp/L.Evocazione.The.Conjuring.2013.iTALiAN.AC3.DUAL.1080p.BrRiP.x264-TrTd_TeaM 17:28:01|DEBUG MAIN: Connecting to transmission: http://localhost:9091 17:28:01|DEBUG MAIN: Stoping torrent L.Evocazione.The.Conjuring.2013.iTALiAN.AC3.DUAL.1080p.BrRiP.x264-TrTd_TeaM in transmission while processing 17:28:06|INFO MAIN: Processing user script /home/xbmc/scripts/filebot_script.sh. 17:28:06|INFO Running script ['/home/xbmc/scripts/filebot_script.sh', '/media/TwoTera/Download/torrent/tmp/L.Evocazione.The.Conjuring.2013.iTALiAN.AC3.DUAL.1080p.BrRiP.x264-TrTd_TeaM/L.Evocazione.The.Conjuring.2013.iTALiAN.AC3.DUAL.1080p.BrRiP.x264-TrTd_TeaM.mkv'] on file /media/TwoTera/Download/torrent/tmp/L.Evocazione.The.Conjuring.2013.iTALiAN.AC3.DUAL.1080p.BrRiP.x264-TrTd_TeaM/L.Evocazione.The.Conjuring.2013.iTALiAN.AC3.DUAL.1080p.BrRiP.x264-TrTd_TeaM.mkv.

I don't know why but the script is called, but something strange happen, I made a little modification to check what is passed to the script and seems to be correct... after some consideration I paste my script, maybe is only my mistake

I think that must be inserted a kind of revert function in case of failure, I try to explain my vision :)

One download is completed "categorized or not is the same" The script make what it need to do, so if is a torrent stop it, move it to a tmp location, extract if must be extracted, try to execute the script, user or nzb or torrent. And than, only If all is ok, clean all the traces, remove the torrent, in case is configured to to that, and all folders. But if an error happens? Actually left all in the tmp directory, IMHO, the best think is to revert all, so move back all (what happens if the original file is compressed?) and restart the torrent. Maybe the best initial solution is to copy all in a tmp dir, not to move. But copy with a low priority option :)

Ps, if I manually run ./filebot_script.sh /media/TwoTera/Download/torrent/tmp/L.Evocazione.The.Conjuring.2013.iTALiAN.AC3.DUAL.1080p.BrRiP.x264-TrTd_TeaM/L.Evocazione.The.Conjuring.2013.iTALiAN.AC3.DUAL.1080p.BrRiP.x264-TrTd_TeaM.mkv

works fine :)

my script:

!/bin/bash

filename=$1 dest_dir="/media/TwoTera/Multimedia" priority="nice -19"

file=$(basename "$filename")

echo $filename >> /home/xbmc/scripts/richiami.txt

$priority filebot -script fn:amc "$filename" --output "$dest_dir" --action move --conflict fail -non-strict --def music=n --log-file amc.log --def artwork=n --def "seriesFormat=Serie Tv/{n}/{n} - {s.pad(2)}x{es.collect{ it.pad(2) }.join('-')} - {t}" "movieFormat=Film/{n} ({y}){' - '+fn.match(/(?i)cd+[0-9]/).toUpperCase()}{' - '+fn.match(/(?i)cd +[0-9]/).toUpperCase()}" --lang it

if [[ "$(tail -3 /home/xbmc/.filebot/logs/amc.log | head -1)" == already ]] then /home/xbmc/scripts/pushover.sh -a filebot -m "Filebot ha cercato di rinominare $file, ma il file di destinazione esiste già. Nulla è stato rinominato!!" exit 0 fi

if [[ "$(tail -1 /home/xbmc/.filebot/logs/amc.log)" == Fail ]] then /home/xbmc/scripts/pushover.sh -a filebot -m "Filebot ha cercato di rinominare $file, ma qualcosa e' andato storto, il file non e' stato rinominato!" else From=$(tail /home/xbmc/.filebot/history.xml | grep -oP '(?<=from=").(?=" to)' | tail -1) To=$(tail /home/xbmc/.filebot/history.xml | grep -oP '(?<=to="/media/TwoTera/Multimedia/).(?="/>)' | tail -1) /home/xbmc/scripts/pushover.sh -a filebot -m "Filebot ha appena rinominato: $From --> $To" fi

J

clinton-hall commented 11 years ago

Ok.. As far as keeping the original torrent untouched, this should be the case if you set autoProcessMedia.cfg [Torrent] useLink = hard

At least the script is now trying to call the external script... I'll just need to play with the way the script is called and arguments are passed.

clinton-hall commented 11 years ago

ok... I have changed the way the Call is made. so this should launch the external script more reliably (and wait for completion before continuing).

Also, usually I would expect a return code of 0 when successful and anything else would be "failed"... however I have allowed for a new parameter "user_script_successCodes" to be a list of successful output codes so that this should work with a number of external scripts.

please let me know if this is now working, or otherwise what other changes are required.

Jorman commented 11 years ago

Ok, I'll try.

A question, I'm not so expert but, if I use hard links, the file remain on the download folder and is linked for processing purpose, so on the "multimedia" folder there is only the link and not the entire file. Right?

clinton-hall commented 11 years ago

What you have described is a sym-link. Essentially a shortcut.

A hard link is essentially a file. It looks and acts the same as copying, except it doesn't use additional hard drive space.

Files are made up of data and headers, the header (file name and properties etc) points to the data. A hard link creates a new header that points to the same data. If the actual data is edited, it is changed for both "files". But if one file name is changed, the other is not impacted. If one file is deleted, the data remains and the other filename is still ok. Only when all files (headers) are deleted will the data be deleted.

Now... All of this only works on the same hard disk, not across different drives. If across different drives, use symlink. This still works fine, but as you said is just a shortcut with a new name.

Jorman commented 11 years ago

ok tnx, much clear :) For my use I think that I'll use a move function, because I want to have the file only in the right position "multimedia" folder, so the "download" folder is only temporary, so no risk of wrong action :) I've to figure out how to make it works

clinton-hall commented 11 years ago

A move (on the same drive ) is actually a hard-link to new location and an unlink of the original location....

So, actually setting the option for uselink=hard and deleteOriginal=1 this actually does the same as a move, but the difference being that it only deletes the original files once we are sure the processing of the moved file has been successful.

I would recommend this method (at least to test and see if this all works as you want).

Jorman commented 11 years ago

ok, you've convinced me :) I'll try this combination, as I said my goal is to post-process the download, if everything is ok, delete the original and clean up all (not the root that maybe contains other files), if not delete the junk tmp files, reactivate the torrent and signal a possible error.

clinton-hall commented 11 years ago

When you say it like that it does sound rather complicated..... But yes, I expect the script should do as you described.

Possibly a few small tweaks to be made, but I really need to see some logs to indicate what happens with the external script before I know what else needs to happen.

Jorman commented 11 years ago

I'm changing my script and I've one question, if I need to pass more than the full path of the file, can I add my param to the script? Like that user_script_param = -f FP

J

clinton-hall commented 11 years ago

ok... I merged this to dev branch yesterday, and this change was just made in dev branch (I'm not using dev-TPB anymore as this will soon be deleted).

You should be able to specify params as

user_script_param = -f,FP
Jorman commented 11 years ago

ok, tnx, I switched up :)

Jorman commented 11 years ago

Ok, time to report:

Configuration: [UserScript] user_script_categories = ALL user_script_mediaExtensions = .mkv,.avi,.divx,.xvid,.mov,.wmv,.mp4,.mpg,.mpeg user_script_path = /home/xbmc/scripts/filebot_script.sh user_script_param = -f,FP user_script_successCodes = 0 user_script_clean = 1 delay = 120

On my script I use the getopts, so after the -f I put the filename that must be processed, now the script is called but, I don't know why the file is not processed, I inserted one trigger to my script to see what is passed after the -f

In theory this is what is passed to the script: /media/TwoTera/Download/torrent/tmp/The last ride 2004/The last ride 2004.avi

But the file is not processed, but If I run ./filebot_script.sh -f "/media/TwoTera/Download/torrent/tmp/The last ride 2004/The last ride 2004.avi" all works! Maybe the ""????

J

clinton-hall commented 11 years ago

Ok... I have quoted the Params... Please update and test again. If this works I'll actually add this as an option (probably userscript_quoteParams, or similar) so that this can be used with multiple scripts.

Jorman commented 11 years ago

ok, I'll try, but the strange think is that I already use the "" on my script...