search

ManiMatter / decluttarr

Watches radarr, sonarr, lidarr, readarr and whisparr download queues and removes downloads if they become stalled or no longer needed.
GNU General Public License v3.0
99 stars 15 forks source link

Not removing any torrents #56

Closed Shane091490 closed 3 months ago

Shane091490 commented 3 months ago

Hi again, so I've been running with the below docker compose for 19+hrs. I have 223 torrents in my Radarr queue and Decluttarr still has not removed any of the torrents.

My understanding of this is that this is supposed to loop every 10mins and check for stalled, failed, missing files, orphaned, metadate missing, unmonitored and/or slow torrents and remove and/or block them after 3 checks, which means after 30mins it should have removed the torrents right? I have slow torrents turned off because most of my torrents are slow, less than 100kbs and a lot of those do eventually download.

What am I missing?

Radarr version: 5.3.6.8612 qBittorrent image: markusmcnugen/qbittorrentvpn

_radarr-decluttarr_logs.txt

version: "3.3" services: radarr-decluttarr: image: ghcr.io/manimatter/decluttarr:latest container_name: radarr-decluttarr restart: always environment:

ManiMatter commented 3 months ago

IGNORE_PRIVATE_TRACKERS=True

could that be it?

Shane091490 commented 3 months ago

I don't have/use any private trackers though.

Shane091490 commented 3 months ago

Wait is that referring to indexers? I don't use any private indexers.

Shane091490 commented 3 months ago

Just re-deployed with IGNORE_PRIVATE_TRACKERS=False and changed the timer to 2 and after 4 checks it still did not remove any torrents. So it's not that setting....

ManiMatter commented 3 months ago

Can you please 1)switch to debug mode 2) turn off all remove functions but one where you expect a removal 3) let it run until the supposed remove dhould be trigged 4) paste the full logs to a pastebin 5) paste a screenshot of qbit & sonarr to a pasteimg

will take a look later thx

Shane091490 commented 3 months ago

_radarr-decluttarr_logs (2).txt

Screenshot 2024-03-14 105134

Screenshot 2024-03-14 105030

version: "3.3" services: radarr-decluttarr: image: ghcr.io/manimatter/decluttarr:latest container_name: radarr-decluttarr restart: always environment:

Shane091490 commented 3 months ago

image

ManiMatter commented 3 months ago

What I find interesting is that the logs suggest that all your torrents are labelled as "Don't Kill", which per your qBit screenshot is not the case.

Not sure what's going on. What the tool does is it creates the no-kill tag in qbit if it doesnt exist when decluttarr starts. Let's check if that works as expected. If you go to qbit setting "Tags and Categories", do you see it under "Available Tags"?

Do you have any idea why this could be happening? Do you have any special setup potentially with your network configuration or so that could be causing this?

Shane091490 commented 3 months ago

image

Shane091490 commented 3 months ago

No, I have no idea. The tag is created but i have not set the tag on any of the torrents myself.

ManiMatter commented 3 months ago

ok, we'll need to investigate that. i'll try to find time on the weekend to create a dev version that could help us going to the bottom of this; would you be ok to help me a bit with back & forth of sharing the logs what's then going on?

Shane091490 commented 3 months ago

Sure, I'll help with what ever you need.

Shane091490 commented 3 months ago

Here is a basic diagram of the network setup. They run on separate VMs, not sure if that is a problem but decluttarr does connect to both Radarr/qBit.

image

ManiMatter commented 3 months ago

Any reason you dont run decluttarr on vm A? Would it work if you did?

Shane091490 commented 3 months ago

No, I suppose I can. I will try it.

Shane091490 commented 3 months ago

1) Forgot I had the UFW enabled 2) After moving it to the same VM, it would not even connect to qbit anymore, now errors and exits. 3) Disabled UFW and tried again but still not working. 4) Tested and confirmed my qBit user/pass is correct valid. 5) Radarr, qBit, Decluttarr are all on same VM and docker network with UFW disabled.

image Screenshot 2024-03-14 135709

ManiMatter commented 3 months ago

Loggin forbidden is what you get after too many bad attempts; qbit blocks the ip of deckuttarr.

pw / username of qbit correct?

if on the same network, you can disable login for local connections on qbit as a remedy, too

to unblock i think you need to restart qbit container

Shane091490 commented 3 months ago

Ok, got qbit to connect again, had to change from using IP address to using my domain address and restarted qbit container. But it's still is not removing the torrents...

Tested with UFW enabled and disabled. No change....

ManiMatter commented 3 months ago

Ok lets leave it as it is and try to find the cause on the weekend…

can you try one thing pls: go to radarr/queue and remove the affected torrent there (and ban the torrent). Does it work?

because that is what the script does..

ManiMatter commented 3 months ago

I've just pushed a version to /dev that gives us more granular logs. you can pull the :dev image to get it. also - pls try what I wrote above , i'm curious if that works. because if it doesn't, then it's something where radarr is lacking permissions to delete on qbit, and not a decluttar issue:

can you try one thing pls: go to radarr/queue and remove the affected torrent there (and ban the torrent). Does it work?

Shane091490 commented 3 months ago

Hi,

I removed and blocked a stalled torrent from Radarr and it did remove it from qbittorrent successfully.

I will pull the dev image now.

Shane091490 commented 3 months ago

Ok, I have a dozzle instance that I don't mind opening for sharing my logs for this purpose. you can go to https://logs.myrealm.site to view real time logs and download the logs as well if needed.

ManiMatter commented 3 months ago

Thx; did you try this? can you try one thing pls: go to radarr/queue and remove the affected torrent there (and ban the torrent). Does it work?

Shane091490 commented 3 months ago

Yes, I already did that. image

ManiMatter commented 3 months ago

Thx apols missed your reply on this. Will check the logs later

ManiMatter commented 3 months ago

Just checked out the dozzle logs. Unfortunately I can't go back all the way, only the more recent logs seem to be displayed.

Would you mind just restarting dozzle, and once the cleaning has been executed that supposedly should remove the movie stop decluttar once again and post the logs? Will then have a look the "traditional way".

Cheers and thanks for your help

Shane091490 commented 3 months ago

Are you wanting to see the logs at the start of decluttarr?

[DEBUG]: Using selector: EpollSelector [DEBUG]: Starting new HTTPS connection (1): radarr.myrealm.site:443 [DEBUG]: https://radarr.myrealm.site:443 "GET /api/v3/system/status HTTP/1.1" 200 None

INFO: Decluttarr - Application Started! INFO: INFO: Current Settings INFO: Version: dev INFO: Commit: 527d6b0 INFO: INFO: False | Removing failed downloads INFO: False | Removing downloads missing metadata INFO: False | Removing downloads missing files INFO: False | Removing orphan downloads INFO: False | Removing slow downloads INFO: True | Removing stalled downloads INFO: False | Removing downloads belonging to unmonitored TV shows/movies INFO: INFO: Running every: 0 days 0 hours 2.0 minutes INFO: Permitted number of times before stalled/missing metadata/slow downloads are removed: 3 INFO: Downloads with this tag will be skipped: "Don't Kill" INFO: Private Trackers will be skipped: False INFO: INFO: Configured Instances INFO: Radarr: https://radarr.myrealm.site/api/v3 INFO: qBittorrent: https://rqbit.myrealm.site/api/v2 INFO: INFO: Check Instances [DEBUG]: Starting new HTTPS connection (1): radarr.myrealm.site:443 [DEBUG]: https://radarr.myrealm.site:443 "GET /api/v3/system/status HTTP/1.1" 200 None [DEBUG]: Starting new HTTPS connection (1): radarr.myrealm.site:443 [DEBUG]: https://radarr.myrealm.site:443 "GET /api/v3/system/status HTTP/1.1" 200 None INFO: OK | Radarr [DEBUG]: Starting new HTTPS connection (1): rqbit.myrealm.site:443 [DEBUG]: https://rqbit.myrealm.site:443 "POST /api/v2/auth/login HTTP/1.1" 200 3 [DEBUG]: Starting new HTTPS connection (1): rqbit.myrealm.site:443 [DEBUG]: https://rqbit.myrealm.site:443 "GET /api/v2/app/version HTTP/1.1" 200 6 INFO: OK | qBittorrent INFO:

INFO: [DEBUG]: Starting new HTTPS connection (1): rqbit.myrealm.site:443 [DEBUG]: https://rqbit.myrealm.site:443 "GET /api/v2/torrents/tags HTTP/1.1" 200 14

[DEBUG]: Starting new HTTPS connection (1): rqbit.myrealm.site:443 [DEBUG]: https://rqbit.myrealm.site:443 "GET /api/v2/torrents/info?tag=Don%27t+Kill HTTP/1.1" 200 52225 [DEBUG]: main/protectedDowloadItems: [{'added_on': 1709167335, 'amount_left': 0, 'auto_tmm': False, 'availability': 0, 'category': 'Movies', 'completed': 0, 'completion_on': 0, 'content_path': '', 'dl_limit': -1, 'dlspeed': 0, 'downloaded': 0, 'downloaded_session': 0, 'eta': 8640000, 'f_l_piece_prio': False, 'force_start': False, 'hash': 'e89b272f5d1698ca346c4db4131eb0ee3d499c00', 'last_activity': 1709167335, 'magnet_uri': 'magnet:?xt=urn:btih:e89b272f5d1698ca346c4db4131eb0ee3d499c00&dn=Code%20of%20Honor%202016%201080p%20BluRay%20x264%20ROVERS%20ORARBG&tr=udp%3a%2f%2ftracker.mirrorbay.org%3a6969%2fannounce&tr=udp%3a%2f%2fopentracker.i2p.rocks%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.openbittorrent.com%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.therarbg.to%3a6969%2fannounce&tr=udp%3a%2f%2fp4p.arenabg.com%3a1337%2fannounce&tr=udp%3a%2f%2ftracker.theoks.net%3a6969%2fannounce&tr=udp%3a%2f%2fexplodie.org%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.t-rb.org%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.torrent.eu.org%3a451%2fannounce&tr=udp%3a%2f%2ftracker.tiny-vps.com%3a6969%2fannounce&tr=udp%3a%2f%2fmovies.zsw.ca%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.therarbg.com%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.moeking.me%3a6969%2fannounce&tr=udp%3a%2f%2fopen.demonii.com%3a1337%2fannounce&tr=udp%3a%2f%2fexodus.desync.com%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.opentrackr.org%3a1337%2fannounce&tr=udp%3a%2f%2ftracker1.bt.moack.co.kr%3a80%2fannounce&tr=udp%3a%2f%2fopen.stealth.si%3a80%2fannounce', 'max_ratio': -1, 'max_seeding_time': -1, 'name': 'Code of Honor 2016 1080p BluRay x264 ROVERS ORARBG', 'num_complete': 3, 'num_incomplete': 3, 'num_leechs': 0, 'num_seeds': 0, 'priority': 4, 'progress': 0, 'ratio': 0, 'ratio_limit': -2, 'save_path': '/mnt/media/downloads/', 'seeding_time': 0, 'seeding_time_limit': -2, 'seen_complete': 0, 'seq_dl': False, 'size': 0, 'state': 'metaDL', 'super_seeding': False, 'tags': '', 'time_active': 1442153, 'total_size': -1, 'tracker': 'udp://open.stealth.si:80/announce', 'trackers_count': 18, 'up_limit': -1, 'uploaded': 0, 'uploaded_session': 0, 'upspeed': 0}, {'added_on': 1709165560, 'amount_left': 5012234944, 'auto_tmm': False, 'availability': 0.33500000834465027, 'category': 'Movies', 'completed': 2533359616, 'completion_on': 0, 'content_path': '/mnt/media/downloads/A.Dennis.the.Menace.Christmas.2007.1080p.AMZN.WEBRip.AAC2.0.x264-FGT', 'dl_limit': -1, 'dlspeed': 0, 'downloaded': 2537159210, 'downloaded_session': 0, 'eta': 8640000, 'f_l_piece_prio': False, 'force_start': False, 'hash': '3ebd27601468516f1995736bfde54a8f5c09737d', 'last_activity': 1710522522, 'magnet_uri': 'magnet:?xt=urn:btih:3ebd27601468516f1995736bfde54a8f5c09737d&dn=A.Dennis.the.Menace.Christmas.2007.1080p.AMZN.WEBRip.AAC2.0.x264-FGT&tr=udp%3a%2f%2ftracker.mirrorbay.org%3a6969%2fannounce&tr=udp%3a%2f%2fopentracker.i2p.rocks%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.openbittorrent.com%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.therarbg.to%3a6969%2fannounce&tr=udp%3a%2f%2fp4p.arenabg.com%3a1337%2fannounce&tr=udp%3a%2f%2ftracker.theoks.net%3a6969%2fannounce&tr=udp%3a%2f%2fexplodie.org%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.t-rb.org%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.torrent.eu.org%3a451%2fannounce&tr=udp%3a%2f%2ftracker.tiny-vps.com%3a6969%2fannounce&tr=udp%3a%2f%2fmovies.zsw.ca%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.therarbg.com%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.moeking.me%3a6969%2fannounce&tr=udp%3a%2f%2fopen.demonii.com%3a1337%2fannounce&tr=udp%3a%2f%2fexodus.desync.com%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.opentrackr.org%3a1337%2fannounce&tr=udp%3a%2f%2ftracker1.bt.moack.co.kr%3a80%2fannounce&tr=udp%3a%2f%2fopen.stealth.si%3a80%2fannounce', 'max_ratio': -1, 'max_seeding_time': -1, 'name': 'A Dennis the Menace Christmas 2007 1080p AMZN WEBRip AAC2 0 x264 FGT ORARBG', 'num_complete': 0, 'num_incomplete': 9, 'num_leechs': 4, 'num_seeds': 0, 'priority': 50, 'progress': 0.33574022508837265, 'ratio': 0.6419441170977993, 'ratio_limit': -2, 'save_path': '/mnt/media/downloads/', 'seeding_time': 0, 'seeding_time_limit': -2, 'seen_complete': 0, 'seq_dl': False, 'size': 7545594560, 'state': 'stalledDL', 'super_seeding': False, 'tags': '', 'time_active': 1443929, 'total_size': 7545594560, 'tracker': 'udp://movies.zsw.ca:6969/announce', 'trackers_count': 18, 'up_limit': -1, 'uploaded': 1628714429, 'uploaded_session': 10366662, 'upspeed': 0}, {'added_on': 1709169454, 'amount_left': 0, 'auto_tmm': False, 'availability': 0, 'category': 'Movies', 'completed': 0, 'completion_on': 0, 'content_path': '', 'dl_limit': -1, 'dlspeed': 0, 'downloaded': 0, 'downloaded_session': 0, 'eta': 8640000, 'f_l_piece_prio': False, 'force_start': False, 'hash': 'ca2b6924b6c5aae04d32f0e6e5af623e59f95406', 'last_activity': 1709169454, 'magnet_uri': 'magnet:?xt=urn:btih:ca2b6924b6c5aae04d32f0e6e5af623e59f95406&dn=A%20Nightmare%20On%20Elm%20Street%20-%20The%20Dream%20Child%20(1989%20-%20DVD%205)%20-%202Lions-Team&tr=http%3a%2f%2finferno.demonoid.com%3a3408%2fannounce&tr=http%3a%2f%2ftracker.ex.ua%2fannounce&tr=http%3a%2f%2fnemesis.1337x.org%2fannounce&tr=http%3a%2f%2fgenesis.1337x.org%3a1337%2fannounce&tr=http%3a%2f%2fexodus.1337x.org%2fannounce&tr=udp%3a%2f%2ftracker.openbittorrent.com%3a80%2fannounce&tr=udp%3a%2f%2ftracker.publicbt.com%3a80%2fannounce&tr=http%3a%2f%2f9.rarbg.com%3a2710%2fannounce&tr=http%3a%2f%2fbt1.the9.com%3a6969%2fannounce&tr=udp%3a%2f%2fdenis.stalker.h3q.com%3a6969%2fannounce&tr=http%3a%2f%2ftracker.publicbt.com%2fannounce&tr=http%3a%2f%2ftracker.openbittorrent.com%2fannounce&tr=udp%3a%2f%2ftracker.1337x.org%3a80%2fannounce&tr=udp%3a%2f%2ftracker.opentrackr.org%3a1337%2fannounce&tr=http%3a%2f%2ftracker.openbittorrent.com%3a80%2fannounce&tr=udp%3a%2f%2fopentracker.i2p.rocks%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.internetwarriors.net%3a1337%2fannounce&tr=udp%3a%2f%2ftracker.leechers-paradise.org%3a6969%2fannounce&tr=udp%3a%2f%2fcoppersurfer.tk%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.zer0day.to%3a1337%2fannounce', 'max_ratio': -1, 'max_seeding_time': -1, 'name': 'A Nightmare On Elm Street - The Dream Child (1989 - DVD 5) - 2Lions-Team', 'num_complete': 0, 'num_incomplete': 1, 'num_leechs': 0, 'num_seeds': 0, 'priority': 66, 'progress': 0, 'ratio': 0, 'ratio_limit': -2, 'save_path': '/mnt/media/downloads/', 'seeding_time': 0, 'seeding_time_limit': -2, 'seen_complete': 0, 'seq_dl': False, 'size': 0, 'state': 'metaDL', 'super_seeding': False, 'tags': '', 'time_active': 1439759, 'total_size': -1, 'tracker': 'udp://tracker.opentrackr.org:1337/announce', 'trackers_count': 20, 'up_limit': -1, 'uploade

Shane091490 commented 3 months ago

You can get the very beginning by downloading the logs. Go to the top right corner and click the dots >> Download: Mine opens in Notepad and begins at the first log. image

ManiMatter commented 3 months ago

I can see where the issue is, but I have no clue what causes it.

Check this out: https://rqbit.myrealm.site/api/v2/torrents/info?tag=Don%27t+Kill

This returns a massive list right now, but the list is wrong. It is supposed to be filtered by those downloads that are tagged with "Don't Kill", but they are included in the response eventhough they are not tagged.

It is as if the "tag" parameter was completely ignored as it happens with parameters unknown to qbit. See below as example: this unknown parameter is ignored and the entire list is returned; my suspicion is that somehow the "tag" parameter in your version is not recognized and hence ignored. https://rqbit.myrealm.site/api/v2/torrents/info?UNKOWNPARAMETER=RANDOMVALUE

I just tested the corresponding URL on my own setup; works perfectly. So for some reason, in your setup the "?tag=Don%27t+Kill" is ignored.

Any ideas why that could be the case? We need to get that fixed, else everything will be marked as "protected" in your case and hence nothing is removed.

for reference - here's the relevant documentation for the qbit API: https://github.com/qbittorrent/qBittorrent/wiki/WebUI-API-(qBittorrent-4.1)#get-torrent-list

tag 
Get torrents with the given tag (empty string means "without tag"; no "tag" parameter means "any tag". Remember to URL-encode the category name. For example, My tag becomes My%20tag
Shane091490 commented 3 months ago

Sorry, but I'm not an api expert. I have no clue either.

Shane091490 commented 3 months ago

There is a difference in the qbittorrent versions. You are running a newer version than me probably.

Shane091490 commented 3 months ago

I have qbittorrent version: 4.3.8 Looks like the latest version is 4.6 something. This image has not been updated in 2+yrs. I use this one because of it's open vpn functions bundled into the image, and it was only one I was able to get working honestly... I can try another more recently updated qbittorrent image with open vpn functions. Looks like there may be a couple other options to try. They may have a newer version of qbittorrent which may fix this api tag filtering issue.

Shane091490 commented 3 months ago

I have changed to the following and restarted decluttarr. Image: dyonr/qbittorrentvpn Version: 4.6.0

Shane091490 commented 3 months ago

Looks like that resolved it! it just removed the torrents it went from 200+ down to 60~ and then Radarr searched other sources and added some more.

Shane091490 commented 3 months ago

So in conclusion we now know that Decluttarr does not work with qbittorrent version 4.3.8.

Add that to your repo for others.

Thank you so much for taking the time to look at this with me.

Shane091490 commented 3 months ago

I'm going to turn on the other options now and let it check and remove all the rest of the torrents.

Shane091490 commented 3 months ago

Alright well I'm happy with that. If you don't have anything else then I'll switch back over to latest branch and we can mark this resolved.

I'm going to upgrade my Sonarr qbittorrent instance and get Decluttarr up on it as well. Hope all goes well there.

Thanks again.

ManiMatter commented 3 months ago

No probs thanks for your help I don‘t think though that qbit 4.3.8 is the problem; according to specs the filter should work since 2.8.3.

Looks to me rather that the version of qbit vpn you used has a problem with get-parameters.

glad you found an alternative that works