Multiple Categories, Sonarr and NzbGet

Describe the bug Not sure where else to ask questions, this is not really a bug I don't think I am using Sonarr and sending different categories to NzbGet based on folder they come from so I can process differently. I am not sure how to set that up as the nzbtodrone config only has a place for one category. I set up the second one under nzbtomedia...in the drone section, but then I had to manually modify the autoprocessmedia.cfg and add the category.

Is this the right way to do it or am I missing something? I assumed it would just know the category (nzbget recognizes it) but it didnt.

Technical Specs

Running on (Windows, Linux, NAS Model etc) '....' Ubuntu 18.04
Python version '....' 3.8
Download Client (NZBget, SABnbzd, Transmission) '....' nzbget
Intended Media Management (SickChill, CouchPotoato, Radarr, Sonarr) '....' Sonar

Expected behavior A clear and concise description of what you expected to happen.

Log Please provide an extract, or full debug log that indicates the issue.

Multiple categories can be supported.

By default, in NZBGet ui you can only specify 1 category for each media type in nzbToMedia.py (e.g. you can set 1 movie category, 1 tv category and 1 music category). You can manually edit the autoProcessMedia.cfg to add more.

The other way of doing this is just set the multiple categories in NZBGet to call nzbToNZBDrone.py directly. When configuring this script it doesn't ask for the category, and just passes through the category that calls the script. So this will allow you to call the same script (with the same config settings) for multiple categories.

Thanks for the reply. I found a post you made someplace that explained the second category...which worked once I added it. Now I updated it to a single category like you said and it still works.

I am getting an error (has been there since the start, but everything appears to work OK). I will paste that at the end here...let me know please what it means and if I should care about it. As a side note, I was hoping you could either point to a good explanation or provide one here as to how it all works from a high level. What I mean is, I would like to know a bit about how the communication between the 3 pieces works:

Sonarr sends the request through an API directly to NzbGet...Nzbget talks directly with Sonarr as far as I can tell (Sonarr gets updates as the download progresses, so must be). At this point, nzbToMedia is not even in the picture right?

Once the download completes, NzbGet unpacks it from the interim folder to the final download folder, then runs a few scripts to clean things up (and I have one for language correction, etc.). Then Nzbget calls nzbToNzbDrone.py and then what happens? Does nzbToMedia talk to Sonarr or only back to Nzbget once it is done?

Here's the log file, note the error in the middle there about the fork. Thanks very much for the help.

2023-06-13 02:09:13 INFO 2023-06-13 02:10:04 INFO 2023-06-13 02:10:04 INFO 2023-06-13 02:10:04 INFO 2023-06-13 02:10:05 INFO 2023-06-13 02:10:05 INFO 2023-06-13 02:10:05 2023-06-13 02:10:05 2023-06-13 02:10:05 INFO 2023-06-13 02:10:05 INFO 2023-06-13 02:10:05 INFO 2023-06-13 02:10:05 INFO 2023-06-13 02:10:05 INFO 2023-06-13 02:10:05 INFO 2023-06-13 02:10:05 INFO 2023-06-13 02:10:05 2023-06-13 02:10:05 INFO 2023-06-13 02:10:05 INFO 2023-06-13 02:10:05 INFO 2023-06-13 02:10:05 INFO 2023-06-13 02:10:05 INFO 2023-06-13 02:10:05 2023-06-13 02:12:05 INFO ::MAIN: The /opt/nzbget/scripts/nzbToMedia/nzbToNzbDrone.py script completed successfully. ::MAIN: Python v3.8 will reach end of life in 489 days. ::MAIN: Checking database structure... ::MAIN: Checking if git needs an update ::MAIN: No update needed ::MAIN: nzbToMedia Version:c4cc554ea1e9b342538853a8108964bde7929e2f Branch:master (Linux 4.15.0-212-generic) WARNING ::MAIN: Failed to locate 7zip. Transcoding of disk images and extraction of .7z files will not be possible! WARNING ::MAIN: Failed to locate par2. Repair and rename using par files will not be possible! ::MAIN: ######################################################### ::MAIN: ## ..::[nzbToMedia.py]::.. ## ::MAIN: ######################################################### ::MAIN: Script triggered from NZBGet Version 21.1. ::MAIN: Auto-detected SECTION:NzbDrone ::MAIN: Calling NzbDrone:sonarr-tv to post-process:Halo.Nightfall.2014.S01.720p.BluRay.DD5.1.x264.Hifi ::MAIN: Attempting to verify sonarr-tv fork WARNING ::MAIN: Connection to NzbDrone:sonarr-tv failed! Check your configuration ::MAIN: NzbDrone:sonarr-tv fork set to default ::MAIN: NzbDrone:sonarr-tv Could not create a fork object for default. Probaly class not added yet. ::MAIN: FLATTEN: Flattening directory: /disk3/nzbget-dl/Halo.Nightfall.2014.S01.720p.BluRay.DD5.1.x264.Hifi ::TRANSCODER: Checking [Halo.Nightfall.2014.S01.720p.BluRay.DD5.1.x264.Hifi.mkv] for corruption, please stand by ... ::TRANSCODER: SUCCESS: [Halo.Nightfall.2014.S01.720p.BluRay.DD5.1.x264.Hifi.mkv] has no corruption. POSTPROCESS::NZBDRONE: SUCCESS: The download succeeded, sending a post-process request ::MAIN: The /opt/nzbget/scripts/nzbToMedia/nzbToNzbDrone.py script completed successfully.

2 warnings in the log are around par2, and 7zip. These can be installed to get rid of the errors, but these are only required if you need to do extra par repair checks (these are done by nzbget as standard) and/or want to do transcoding of disk images.

The other warning is around a failed connection to sonarr. This could be the api key, port, host, or could even be that the version on Sonarr isn't compatible with the version of nzbToMedia... but this doesn't really matter as per the below:

The way the communication works is largely as you stated;

Sonarr sends the request through an API directly to NzbGet
Sonarr gets updates from NZBget as the download progresses... so this is Sonarr communicating with NZBget. NZBGet doesn't initiate any communication with Sonarr.... but that is really just a technicality.
At this point, nzbToMedia is not in the picture.
On completion of downloading (and after any postprocess scripts have finished running) Sonarr will ask NZBGet for the status, identify the download has completed, and do any renaming/moving.

So with this, and indeed with any of the other media management tools (SickBeard, CouchPotato etc) the nzbToMedia scripts are not needed, and the whole management can certainly be done without these scripts.

The reason these scripts started was to reduce the workload that would cause a NAS (or low processing power system) to slow down when trying to download using SABnzbd and having SickBeard/CouchPotato constantly asking SABnzbd what the status was on each download.... Basically the python processes used by SAB were pushing the limits a bit and every time something would interrupt SABnzbd to ask for the status, the systems would get overloaded and slow right down... These scripts stopped that process and instead would tell SickBeard/CouchPotato when the download was completed and have the files processed. The key benefit here is only realised by actually stopping the constant update request (or significantly reducing the frequency of this).

Since then other additional features were added that can be beneficial (but not necessarily needed).

Transcoding (converting the files to another format that is playable on a specific device)
Failed download handing (now standard on most SickBeard/Sonarr/Radarr etc)
Corruption checks (actually uses ffmpeg/ffprobe to verify that the media is a valid playable file)
safe renaming (checks for the rename .sh/.bat files and rebuilds the rename commands to rename the download file... found with obfuscated file names) . The key here is that it will only rename the download files using valid rename commands and is NOT asking you to actually run a downloaded .sh/.bat file which can be very dangerous. Even here, this is usually not a requirement when Sonarr knows what download was requested and therefore knows what to rename to...
Download additional Subs (this can also now be achieved through other avenues)
Failed/corrupt notification back to the indexer (dognzb.cr)

The script itself will do these particular functions and then tell Sonarr, via api, that the download is complete and where it is (if you used transcoder this will likely be a different location to what NZBGet downloaded to).

Even if this communication to Sonarr fails, when Sonarr asks for the status and see the download is complete it will find the files are rename them (unless you used transcoding options as above).

So. if you don't need/want the extra features and you have enough processing power, removing nzbToMedia might make it a cleaner process for you.

If you have low processing power, or like to keep your usage low, these scripts can reduce the workload, but you also need to "tune" your Sonarr to stop/reduce the polling of NZBGet.

Otherwise these scripts can add some additional features as above, but there are also some other scripts that will do some of these features as well before allowing Sonarr to do its thing. e.g. https://github.com/clinton-hall/GetScripts, https://github.com/nzbget/FakeDetector, https://github.com/nzbget/FailureLink

Thanks for the thorough update...much appreciated. This explains a few things for me.

I would like to use the script and stop Sonarr from the polling. I think it would be faster too in the end (as of now it takes a minute for Sonarr to pick up on the file).

To that end, I am using Sonarr V4, which is the development branch. Is that likely why the API is failing like that? I have the API, port, etc. all set right (I will verify again) but still get that failure. I assume this is why when the script ends, Sonarr does not import right away...it only picks it up in the polling scan.

Thanks. Eric

On Wed, Jun 14, 2023 at 6:04 AM Clinton Hall @.***> wrote:

2 warnings in the log are around par2, and 7zip. These can be installed to get rid of the errors, but these are only required if you need to do extra par repair checks (these are done by nzbget as standard) and/or want to do transcoding of disk images.

The other warning is around a failed connection to sonarr. This could be the api key, port, host, or could even be that the version on Sonarr isn't compatible with the version of nzbToMedia... but this doesn't really matter as per the below:

The way the communication works is largely as you stated;

Sonarr sends the request through an API directly to NzbGet

Sonarr gets updates from NZBget as the download progresses... so this is Sonarr communicating with NZBget. NZBGet doesn't initiate any communication with Sonarr.... but that is really just a technicality.

At this point, nzbToMedia is not in the picture.

On completion of downloading (and after any postprocess scripts have finished running) Sonarr will ask NZBGet for the status, identify the download has completed, and do any renaming/moving.

So with this, and indeed with any of the other media management tools (SickBeard, CouchPotato etc) the nzbToMedia scripts are not needed, and the whole management can certainly be done without these scripts.

The reason these scripts started was to reduce the workload that would cause a NAS (or low processing power system) to slow down when trying to download using SABnzbd and having SickBeard/CouchPotato constantly asking SABnzbd what the status was on each download.... Basically the python processes used by SAB were pushing the limits a bit and every time something would interrupt SABnzbd to ask for the status, the systems would get overloaded and slow right down... These scripts stopped that process and instead would tell SickBeard/CouchPotato when the download was completed and have the files processed. The key benefit here is only realised by actually stopping the constant update request (or significantly reducing the frequency of this).

Since then other additional features were added that can be beneficial (but not necessarily needed).

Transcoding (converting the files to another format that is playable on a specific device)

Failed download handing (now standard on most SickBeard/Sonarr/Radarr etc)

Corruption checks (actually uses ffmpeg/ffprobe to verify that the media is a valid playable file)

safe renaming (checks for the rename .sh/.bat files and rebuilds the rename commands to rename the download file... found with obfuscated file names) . The key here is that it will only rename the download files using valid rename commands and is NOT asking you to actually run a downloaded .sh/.bat file which can be very dangerous. Even here, this is usually not a requirement when Sonarr knows what download was requested and therefore knows what to rename to...

Download additional Subs (this can also now be achieved through other avenues)

Failed/corrupt notification back to the indexer (dognzb.cr)

The script itself will do these particular functions and then tell Sonarr, via api, that the download is complete and where it is (if you used transcoder this will likely be a different location to what NZBGet downloaded to).

Even if this communication to Sonarr fails, when Sonarr asks for the status and see the download is complete it will find the files are rename them (unless you used transcoding options as above).

So. if you don't need/want the extra features and you have enough processing power, removing nzbToMedia might make it a cleaner process for you.

If you have low processing power, or like to keep your usage low, these scripts can reduce the workload, but you also need to "tune" your Sonarr to stop/reduce the polling of NZBGet.

Otherwise these scripts can add some additional features as above, but there are also some other scripts that will do some of these features as well before allowing Sonarr to do its thing. e.g. https://github.com/clinton-hall/GetScripts, https://github.com/nzbget/FakeDetector, https://github.com/nzbget/FailureLink

— Reply to this email directly, view it on GitHub https://github.com/clinton-hall/nzbToMedia/issues/1972#issuecomment-1590891163, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQDOIDUHPJXD5QIKXU3Q5DXLGECRANCNFSM6AAAAAAY322MVE . You are receiving this because you authored the thread.Message ID: @.***>

The version you have should be compatible with Sonarr V4.

are you able to edit the autoprocessmedia.cfg and set

[General]
log_debug = 1

then the next time the script runs there should be more details in the log. Hopefully these will help identify the issue.

Will do. I also noticed that no matter what I set the check for update flag to in the config, it always checks for an update.

One other thing I will mention is that in the autoprocessconfig file, there is always a section called tv. I reinstalled and then reset the config in nzbget, changing the name to sonarr-tv in the category...thinking that would start fresh with only one section in there...but it just added it to the bottom as a new section. Just wanted to be sure that was normal.

I will post back with the log file.

Here's the log file. Looks like the API is working fine at the end there...so perhaps the errors listed about the fork are inconsequential?

I did a test and removed nzbToMedia script from the category and it did process fine...but it seems at the end, it takes longer as it has to wait for Sonarr to poll the folder.

Let me know your thoughts. I am also wondering why it checks for update required, even though I have that set to not do that in the nzbget setting for nzbtodrone. Its not a huge deal, more curious than anything. But I am processing a lot of my stuff right now, so no need to keep checking for updates at the moment.

I removed the IP address, but it is the same in all places in the log file:

2023-06-15 12:25:47 INFO ::MAIN: Python v3.8 will reach end of life in 487 days. 2023-06-15 12:25:47 INFO ::MAIN: Checking database structure... 2023-06-15 12:25:47 DEBUG ::MAIN: Checking Initial Schema database upgrade 2023-06-15 12:25:47 DEBUG ::MAIN: InitialSchema upgrade not required 2023-06-15 12:25:47 DEBUG ::MAIN: Checking if we can use git commands: git version 2023-06-15 12:25:47 DEBUG ::MAIN: Executing git version with your shell in /opt/nzbget/scripts/nzbToMedia 2023-06-15 12:25:47 DEBUG ::MAIN: git version : returned successful 2023-06-15 12:25:47 DEBUG ::MAIN: Using: git 2023-06-15 12:25:47 DEBUG ::MAIN: Executing git symbolic-ref -q HEAD with your shell in /opt/nzbget/scripts/nzbToMedia 2023-06-15 12:25:47 DEBUG ::MAIN: git symbolic-ref -q HEAD : returned successful 2023-06-15 12:25:47 INFO ::MAIN: Checking if git needs an update 2023-06-15 12:25:47 DEBUG ::MAIN: Executing git rev-parse HEAD with your shell in /opt/nzbget/scripts/nzbToMedia 2023-06-15 12:25:47 DEBUG ::MAIN: git rev-parse HEAD : returned successful 2023-06-15 12:25:47 DEBUG ::MAIN: Executing git fetch origin with your shell in /opt/nzbget/scripts/nzbToMedia 2023-06-15 12:25:47 DEBUG ::MAIN: git fetch origin : returned successful 2023-06-15 12:25:47 DEBUG ::MAIN: Executing git rev-parse --verify --quiet '@{upstream}' with your shell in /opt/nzbget/scripts/nzbToMedia 2023-06-15 12:25:47 DEBUG ::MAIN: git rev-parse --verify --quiet '@{upstream}' : returned successful 2023-06-15 12:25:47 DEBUG ::MAIN: Executing git rev-list --left-right '@{upstream}'...HEAD with your shell in /opt/nzbget/scripts/nzbToMedia 2023-06-15 12:25:48 DEBUG ::MAIN: git rev-list --left-right '@{upstream}'...HEAD : returned successful 2023-06-15 12:25:48 DEBUG ::MAIN: cur_commit = c4cc554ea1e9b342538853a8108964bde7929e2f % (newest_commit)= c4cc554ea1e9b342538853a8108964bde7929e2f, num_commits_behind = 0, num_commits_ahead = 0 2023-06-15 12:25:48 INFO ::MAIN: No update needed 2023-06-15 12:25:48 INFO ::MAIN: nzbToMedia Version:c4cc554ea1e9b342538853a8108964bde7929e2f Branch:master (Linux 4.15.0-212-generic) 2023-06-15 12:25:48 WARNING ::MAIN: Failed to locate 7zip. Transcoding of disk images and extraction of .7z files will not be possible! 2023-06-15 12:25:48 WARNING ::MAIN: Failed to locate par2. Repair and rename using par files will not be possible! 2023-06-15 12:25:48 INFO ::MAIN: ######################################################### 2023-06-15 12:25:48 INFO ::MAIN: ## ..::[nzbToMedia.py]::.. ## 2023-06-15 12:25:48 INFO ::MAIN: ######################################################### 2023-06-15 12:25:48 DEBUG ::MAIN: Options passed into nzbToMedia: ['/opt/nzbget/scripts/nzbToMedia/nzbToNzbDrone.py'] 2023-06-15 12:25:48 INFO ::MAIN: Script triggered from NZBGet Version 21.1. 2023-06-15 12:25:48 DEBUG ::MAIN: Adding NZB download info for directory /disk3/nzbget-dl/Hey.Arnold.S05E01.Monkeyman.Buses.Bikes.Subways.480p.HULU.WEB-DL.AAC2.0.H.264-ETHiCS to database 2023-06-15 12:25:48 INFO ::MAIN: Auto-detected SECTION:NzbDrone 2023-06-15 12:25:48 INFO ::MAIN: Calling NzbDrone:sonarr-tv to post-process:Hey.Arnold.S05E01.Monkeyman.Buses.Bikes.Subways.480p.HULU.WEB-DL.AAC2.0.H.264-ETHiCS 2023-06-15 12:25:48 DEBUG ::SERVER: Attempting to connect to server at http://IPREMOVED:8989 2023-06-15 12:25:48 DEBUG ::SERVER: Server responded at http://IPREMOVED:8989 2023-06-15 12:25:48 INFO ::MAIN: Attempting to verify sonarr-tv fork 2023-06-15 12:25:48 WARNING ::MAIN: Connection to NzbDrone:sonarr-tv failed! Check your configuration 2023-06-15 12:25:48 INFO ::MAIN: NzbDrone:sonarr-tv fork set to default 2023-06-15 12:25:48 DEBUG ::MAIN: Create object for fork default 2023-06-15 12:25:48 INFO ::MAIN: NzbDrone:sonarr-tv Could not create a fork object for default. Probaly class not added yet. 2023-06-15 12:25:48 INFO ::MAIN: FLATTEN: Flattening directory: /disk3/nzbget-dl/Hey.Arnold.S05E01.Monkeyman.Buses.Bikes.Subways.480p.HULU.WEB-DL.AAC2.0.H.264-ETHiCS 2023-06-15 12:25:48 DEBUG ::MAIN: Checking for empty folders in:/disk3/nzbget-dl/Hey.Arnold.S05E01.Monkeyman.Buses.Bikes.Subways.480p.HULU.WEB-DL.AAC2.0.H.264-ETHiCS 2023-06-15 12:25:48 DEBUG ::MAIN: calling command: /usr/bin/ffprobe -v quiet -print_format json -show_format -show_streams -show_error /opt/nzbget/scripts/nzbToMedia/tests/test.mp4 2023-06-15 12:25:48 INFO ::TRANSCODER: Checking [Hey.Arnold.S05E01.Monkeyman.Buses.Bikes.Subways.480p.HULU.WEB-DL.AAC2.0.H.264-ETHiCS.mkv] for corruption, please stand by ... 2023-06-15 12:25:48 DEBUG ::MAIN: calling command: /usr/bin/ffprobe -v quiet -print_format json -show_format -show_streams -show_error /disk3/nzbget-dl/Hey.Arnold.S05E01.Monkeyman.Buses.Bikes.Subways.480p.HULU.WEB-DL.AAC2.0.H.264-ETHiCS/Hey.Arnold.S05E01.Monkeyman.Buses.Bikes.Subways.480p.HULU.WEB-DL.AAC2.0.H.264-ETHiCS.mkv 2023-06-15 12:25:48 INFO ::TRANSCODER: SUCCESS: [Hey.Arnold.S05E01.Monkeyman.Buses.Bikes.Subways.480p.HULU.WEB-DL.AAC2.0.H.264-ETHiCS.mkv] has no corruption. 2023-06-15 12:25:48 POSTPROCESS::NZBDRONE: SUCCESS: The download succeeded, sending a post-process request 2023-06-15 12:25:48 DEBUG ::NZBDRONE: path: /disk3/nzbget-dl/Hey.Arnold.S05E01.Monkeyman.Buses.Bikes.Subways.480p.HULU.WEB-DL.AAC2.0.H.264-ETHiCS 2023-06-15 12:25:48 DEBUG ::NZBDRONE: Opening URL: http://IPREMOVED:8989/api/v3/command with data: {"name": "DownloadedEpisodesScan", "path": "/disk3/nzbget-dl/Hey.Arnold.S05E01.Monkeyman.Buses.Bikes.Subways.480p.HULU.WEB-DL.AAC2.0.H.264-ETHiCS", "downloadClientId": "7c4fedd04c044b89aa60d243505f9e1f", "importMode": "Move"} 2023-06-15 12:25:48 DEBUG ::NZBDRONE: Scan started with id: 92445 2023-06-15 12:25:58 DEBUG ::NZBDRONE: The Scan command return status: completed 2023-06-15 12:25:58 DEBUG ::NZBDRONE: The directory /disk3/nzbget-dl/Hey.Arnold.S05E01.Monkeyman.Buses.Bikes.Subways.480p.HULU.WEB-DL.AAC2.0.H.264-ETHiCS has been removed. Renaming was successful. 2023-06-15 12:25:58 INFO ::MAIN: The /opt/nzbget/scripts/nzbToMedia/nzbToNzbDrone.py script completed successfully.

I think from memory (without delving into the code) that the version check still happens even when the updater is disabled. This part can probably be disabled by setting version_notify = 0

The tv subsection is copied from the default .cfg template. so this will re-create but since it doesn't match a category that calls the script it isn't an issue so much as untidy config... something I should ideally try and clean-up at my end.

Yes, processing seems to be working. The fork warning is a result of the need to continue with creating sub-modules for the script (again, more tidy-up at my end)

Great, thanks for all the info. Seems to be working well. I appreciate the work on this. I am going to try and tweak Sonarr to stop the polling...I think that is a simple database edit, but then it overwrites on updates (which for me in Sev, is often) so not sure its worth it....it is pretty quick to act now, but the polling is taking processing power

clinton-hall / nzbToMedia

Multiple Categories, Sonarr and NzbGet #1972