ManiMatter / decluttarr

Watches radarr, sonarr, lidarr, readarr and whisparr download queues and removes downloads if they become stalled or no longer needed.
GNU General Public License v3.0
99 stars 15 forks source link

Downloads in moving state are seen as slow #101

Closed SixFive7 closed 1 month ago

SixFive7 commented 1 month ago

When there is a super fast download everything goes perfect. But when this download is finished there is a moment when qbittorrent is busy moving the data from the incomplete downloads to the final download directory. Normally this is fast, but if the download is large or the move is between two different filesystem it can take a while. During this move the download state is moving before transitioning to seeding or completed. Decluttarr should ignore downloads in this intermediate state. Instead it's concluding these (already completed) downloads are slow and incorrectly starts removing them.

ManiMatter commented 1 month ago

hi @SixFive7

thank you for the input.

Could you please paste 1) What you see in the /queue API response (open network tab in your browser, go to your *arr app, queue-tab, and look for "queue" in the network response), at the moment when such import is happening? 2) what you see in the /info API response from qBit

SixFive7 commented 1 month ago

Hi,

Thank you for your time in this awesome tool!

The /queue API response from Radarr:

{
  "movieId": 363,
  "languages": [
    {
      "id": 1,
      "name": "English"
    }
  ],
  "quality": {
    "quality": {
      "id": 19,
      "name": "Bluray-2160p",
      "source": "bluray",
      "resolution": 2160,
      "modifier": "none"
    },
    "revision": {
      "version": 1,
      "real": 0,
      "isRepack": false
    }
  },
  "customFormats": [],
  "customFormatScore": 0,
  "size": 77532326,
  "title": "REDACTED",
  "sizeleft": 0,
  "added": "2024-05-12T15:54:00Z",
  "status": "downloading", <- huh?
  "trackedDownloadStatus": "ok",
  "trackedDownloadState": "downloading",
  "statusMessages": [],
  "downloadId": "REDACTED",
  "protocol": "torrent",
  "downloadClient": "qBittorrent",
  "downloadClientHasPostImportCategory": false,
  "indexer": "REDACTED",
  "id": "REDACTED",
},

and the /info API response from qBit:

{
  "added_on": 1715529240,
  "amount_left": 0,
  "auto_tmm": false,
  "availability": -1,
  "category": "radarr",
  "completed": 7677532326,
  "completion_on": 1715546586,
  "content_path": "/partials/REDACTED",
  "dl_limit": 0,
  "dlspeed": 0,
  "download_path": "/partials",
  "downloaded": 7677544724,
  "downloaded_session": 7677544724,
  "eta": 8640000,
  "f_l_piece_prio": false,
  "force_start": false,
  "hash": "b7602c93f4729d49f44a37eef3f9c6def0fa79cc",
  "inactive_seeding_time_limit": -2,
  "infohash_v1": "REDACTED",
  "infohash_v2": "",
  "last_activity": 1715546656,
  "magnet_uri":  "REDACTED",
  "max_inactive_seeding_time": -1,
  "max_ratio": -1,
  "max_seeding_time": 1,
  "name": "REDACTED",
  "num_complete": 2,
  "num_incomplete": 26,
  "num_leechs": 0,
  "num_seeds": 0,
  "priority": 0,
  "progress": 1,
  "ratio": 1.9321961921794205,
  "ratio_limit": -2,
  "save_path": "/media/qbittorrent",
  "seeding_time": 71,
  "seeding_time_limit": -2,
  "seen_complete": 1715546586,
  "seq_dl": false,
  "size": 77532326,
  "state": "moving", <- huh?
  "super_seeding": false,
  "tags": "",
  "time_active": 17403,
  "total_size": 77532326,
  "tracker": "",
  "trackers_count": 6,
  "up_limit": 0,
  "uploaded": 7156977957,
  "uploaded_session": 7157124988,
  "upspeed": 0
}

Interesting is that Radarr is reporting the state as downloading whilst qBit is reporting it as moving. Maybe moving isn't implemented on the Radarr side and it just wait for completed? Both are up to date and otherwise performing admirably. In any case, would it be enough to add a condition to remove a download from decluttarr's target list if its qBit state is moving?

ManiMatter commented 1 month ago

Probably the arr apps consider the "moving" part of the "downloading"-process. Ie. the definition of the arr apps of what is "downloading"-status is wider than the one by qbit (which is more detailed).

A simple workaround is to increase the frequency by which you run decluttarr (e.g. from 10min to, dunno, 30min), so that you are sure things copy successfully from one file system to another before decluttarr tries to remove them).

Another would be that the *arr apps add the information that things are still moving to their API response (could, for instance, be added to "statusMessage"). For that, you'd have to raise a ticket with them though..

If they were to provide that information, happy to then adjust the logic accordingly of decluttarr to exclude the moving ones from slowness-check

ManiMatter commented 1 month ago

Here's another idea...

"size": 7677532326, "total_size": 7677532326,

If those two are identical, that means everything has downloaded, I'd assume? If yes, we could exclude those from the slowness check, which should solve the problem.

What do you think

SixFive7 commented 1 month ago

Yes that sounds like the perfect language agnostic way to fix this. Will this solution overlap with any commits already present from https://github.com/ManiMatter/decluttarr/pull/97?

ManiMatter commented 1 month ago

No this will not overlap. #97 is about SAB downloads that have not even started and are queued, but the status shows "downloading".

here, we are talking about downloads that have finished downloading and are in "moving" state according to qbit, but the status reported by *arr is "downloading". What we'll now use is the download sizes to detect which ones are done downloading and skip them (despite them showing still "downloading" status)

ManiMatter commented 1 month ago

Just merged a fix to the dev-branch. Can you pls pull the dev-image and see if it works for you?

SixFive7 commented 1 month ago

Hi, I just tested the dev build. Could not test the move detection yet as I need to wait for a large download that takes at least 1 min. (minimal interval of declutarr) to move between disks. But I did notice the following new errors. Not sure if they are related or not. But since this has never happenend on the normal builds I'd figure I at least share it. Will keep testing though.

[INFO]: *** Check Instances ***
[INFO]: OK | Radarr
[INFO]: OK | Sonarr
[INFO]: OK | Lidarr
[INFO]: OK | Readarr
[INFO]: OK | qBittorrent
[INFO]: 
[INFO]: ##################################################
[INFO]: LOG_LEVEL = INFO: Only logging changes (switch to VERBOSE for more info)
/app/src/jobs/remove_slow.py:18: RuntimeWarning: coroutine 'rest_get' was never awaited
  qBitConnectionStatus = await rest_get(settingsDict['QBITTORRENT_URL']+'/sync/maindata', cookies=settingsDict['QBIT_COOKIE'])['server_state']['connection_status']
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
[WARNING]: >>> Queue cleaning failed on Radarr. (File: remove_slow.py / Line: 18 / Error Message: 'coroutine' object is not subscriptable / Error Type: <class 'TypeError'>)
[WARNING]: >>> Queue cleaning failed on Sonarr. (File: remove_slow.py / Line: 18 / Error Message: 'coroutine' object is not subscriptable / Error Type: <class 'TypeError'>)
[WARNING]: >>> Queue cleaning failed on Radarr. (File: remove_slow.py / Line: 18 / Error Message: 'coroutine' object is not subscriptable / Error Type: <class 'TypeError'>)
[WARNING]: >>> Queue cleaning failed on Sonarr. (File: remove_slow.py / Line: 18 / Error Message: 'coroutine' object is not subscriptable / Error Type: <class 'TypeError'>)
>>> keeps on going a long time >>>
SixFive7 commented 1 month ago

Ok, before I let the large download start moving across disks I first quickly tested if the dev build would still recognize a real slow download. But it seems broken. Decluttarr v1.41.2 is still working fine though.

ManiMatter commented 1 month ago

thanks for highlighting this. fixed it. can you pls retry with dev?

SixFive7 commented 1 month ago

Wow you work fast!

Just re pulled the image. Got commit 881aaf3 now. It's still giving an error but detection seems to work. Should I continue with the large file (and slow to reproduce) test or is this error an indication my test might say nothing?

[INFO]: ##################################################
[INFO]: Decluttarr - Application Started!
[INFO]: 
[INFO]: *** Current Settings ***
[INFO]: Version: dev
[INFO]: Commit: 881aaf3
[INFO]: 
[INFO]: True | Removing failed downloads
[INFO]: True | Removing downloads missing metadata
[INFO]: True | Removing downloads missing files
[INFO]: True | Removing downloads that fail on import (no format upgrade)
[INFO]: True | Removing orphan downloads
[INFO]: True | Removing slow downloads
[INFO]: True | Removing stalled downloads
[INFO]: True | Removing downloads belonging to unmonitored items
[INFO]: 
[INFO]: Running every: 0 days 0 hours 1.0 minutes
[INFO]: Minimum speed enforced: 100 KB/s
[INFO]: Permitted number of times before stalled/missing metadata/slow downloads are removed: 10
[INFO]: Downloads with this tag will be skipped: "Don't Kill"
[INFO]: Private Trackers will be skipped: True
[INFO]: 
[INFO]: *** Configured Instances ***
[INFO]: Radarr: http://media-radarr:7878/api/v3
[INFO]: Sonarr: http://media-sonarr:8989/api/v3
[INFO]: Lidarr: http://media-lidarr:8686/api/v1
[INFO]: Readarr: http://media-readarr:8787/api/v1
[INFO]: qBittorrent: http://media-qbittorrent:8080/api/v2
[INFO]: 
[INFO]: *** Check Instances ***
[INFO]: OK | Radarr
[INFO]: OK | Sonarr
[INFO]: OK | Lidarr
[INFO]: OK | Readarr
[INFO]: OK | qBittorrent
[INFO]: 
[INFO]: ##################################################
[INFO]: LOG_LEVEL = INFO: Only logging changes (switch to VERBOSE for more info)
/app/src/jobs/remove_slow.py:19: RuntimeWarning: coroutine 'rest_get' was never awaited
  qBitConnectionStatus = await rest_get(settingsDict['QBITTORRENT_URL']+'/sync/maindata', cookies=settingsDict['QBIT_COOKIE'])['server_state']['connection_status']
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
[WARNING]: >>> Queue cleaning failed on Radarr. (File: remove_slow.py / Line: 19 / Error Message: 'coroutine' object is not subscriptable / Error Type: <class 'TypeError'>)
[WARNING]: >>> Queue cleaning failed on Sonarr. (File: remove_slow.py / Line: 19 / Error Message: 'coroutine' object is not subscriptable / Error Type: <class 'TypeError'>)
[INFO]: >>> Detected stalled download (1 out of 10 permitted times): REDACTED
ManiMatter commented 1 month ago

gnah. I fixed it didnt commit the fix when pushing :D

Should be fixed now, pls could you retest

SixFive7 commented 1 month ago

Almost there.

The previous error are gone. I also did the test with the large move transfer. Everything worked, there were 2 cycles during the move where it did not pick it up as slow. So all good. What I can't test easily is what happens if the download is detected as slow (but not yet removed) just before it goes to the moving state. Here I'm just assuming the filter logic is disconnected from that. Does this need to be tested?

What is new however is an error in the detection cycle just after the download went to completed in qbit. Not sure what the state in radarr was at that point. Already had trouble keeping my eyes on two thing within that second.

2024-05-13T15:39:47.382185067Z [INFO]: ##################################################
2024-05-13T15:39:47.382236682Z [INFO]: Decluttarr - Application Started!
2024-05-13T15:39:47.382247508Z [INFO]: 
2024-05-13T15:39:47.382256517Z [INFO]: *** Current Settings ***
2024-05-13T15:39:47.382291020Z [INFO]: Version: dev
2024-05-13T15:39:47.382320424Z [INFO]: Commit: f9f46c5
2024-05-13T15:39:47.382333625Z [INFO]: 
2024-05-13T15:39:47.382345009Z [INFO]: True | Removing failed downloads
2024-05-13T15:39:47.382356324Z [INFO]: True | Removing downloads missing metadata
2024-05-13T15:39:47.382372877Z [INFO]: True | Removing downloads missing files
2024-05-13T15:39:47.382390827Z [INFO]: True | Removing downloads that fail on import (no format upgrade)
2024-05-13T15:39:47.382402281Z [INFO]: True | Removing orphan downloads
2024-05-13T15:39:47.382424003Z [INFO]: True | Removing slow downloads
2024-05-13T15:39:47.382437273Z [INFO]: True | Removing stalled downloads
2024-05-13T15:39:47.382464163Z [INFO]: True | Removing downloads belonging to unmonitored items
2024-05-13T15:39:47.382480297Z [INFO]: 
2024-05-13T15:39:47.382520038Z [INFO]: Running every: 0 days 0 hours 1.0 minutes
2024-05-13T15:39:47.382533308Z [INFO]: Minimum speed enforced: 100 KB/s
2024-05-13T15:39:47.382553144Z [INFO]: Permitted number of times before stalled/missing metadata/slow downloads are removed: 10
2024-05-13T15:39:47.382572560Z [INFO]: Downloads with this tag will be skipped: "Don't Kill"
2024-05-13T15:39:47.382584434Z [INFO]: Private Trackers will be skipped: True
2024-05-13T15:39:47.382593863Z [INFO]: 
2024-05-13T15:39:47.382619426Z [INFO]: *** Configured Instances ***
2024-05-13T15:39:47.382631718Z [INFO]: Radarr: http://media-radarr:7878/api/v3
2024-05-13T15:39:47.382646385Z [INFO]: Sonarr: http://media-sonarr:8989/api/v3
2024-05-13T15:39:47.382654836Z [INFO]: Lidarr: http://media-lidarr:8686/api/v1
2024-05-13T15:39:47.382678513Z [INFO]: Readarr: http://media-readarr:8787/api/v1
2024-05-13T15:39:47.382692273Z [INFO]: qBittorrent: http://media-qbittorrent:8080/api/v2
2024-05-13T15:39:47.382708407Z [INFO]: 
2024-05-13T15:39:47.382718255Z [INFO]: *** Check Instances ***
2024-05-13T15:39:47.387358813Z [INFO]: OK | Radarr
2024-05-13T15:39:47.392409983Z [INFO]: OK | Sonarr
2024-05-13T15:39:47.397218935Z [INFO]: OK | Lidarr
2024-05-13T15:39:47.401490859Z [INFO]: OK | Readarr
2024-05-13T15:39:47.406419104Z [INFO]: OK | qBittorrent
2024-05-13T15:39:47.406439358Z [INFO]: 
2024-05-13T15:39:47.407971940Z [INFO]: ##################################################
2024-05-13T15:39:47.407989820Z [INFO]: LOG_LEVEL = INFO: Only logging changes (switch to VERBOSE for more info)
2024-05-13T15:41:48.722359697Z [INFO]: >>> Detected stalled download (1 out of 10 permitted times): REDACTED-A
2024-05-13T15:41:48.811988060Z [INFO]: >>> Detected slow download (1 out of 10 permitted times): REDACTED-B
2024-05-13T15:41:48.823512215Z [INFO]: >>> Detected stalled download (1 out of 10 permitted times): REDACTED-C
2024-05-13T15:42:49.108220511Z [INFO]: >>> Detected stalled download (2 out of 10 permitted times): REDACTED-A
2024-05-13T15:42:49.202590287Z [INFO]: >>> Detected slow download (2 out of 10 permitted times): REDACTED-B
2024-05-13T15:42:49.214008908Z [INFO]: >>> Detected stalled download (2 out of 10 permitted times): REDACTED-C
2024-05-13T15:43:49.445720795Z [INFO]: >>> Detected slow download (1 out of 10 permitted times): REDACTED-A
2024-05-13T15:43:49.454599685Z [INFO]: >>> Download no longer marked as stalled: REDACTED-A
2024-05-13T15:43:49.567936928Z [INFO]: >>> Detected slow download (1 out of 10 permitted times): REDACTED-D
2024-05-13T15:43:49.567972339Z [INFO]: >>> Detected slow download (3 out of 10 permitted times): REDACTED-B
2024-05-13T15:43:49.580139963Z [INFO]: >>> Detected stalled download (3 out of 10 permitted times): REDACTED-C
2024-05-13T15:44:49.849764739Z [WARNING]: >>> Queue cleaning failed on Radarr. (File: remove_slow.py / Line: 32 / Error Message: local variable 'speed' referenced before assignment / Error Type: <class 'UnboundLocalError'>)
2024-05-13T15:44:50.001106008Z [INFO]: >>> Download no longer marked as slow: REDACTED-D
2024-05-13T15:44:50.001126821Z [INFO]: >>> Detected slow download (4 out of 10 permitted times): REDACTED-B
2024-05-13T15:44:50.012608859Z [INFO]: >>> Detected stalled download (4 out of 10 permitted times): REDACTED-C
ManiMatter commented 1 month ago

What is new however is an error in the detection cycle just after the download went to completed in qbit. Not sure what the state in radarr was at that point. Already had trouble keeping my eyes on two thing within that second.

Can you pls try again?

what happens if the download is detected as slow (but not yet removed) just before it goes to the moving state.

If the torrent gets caught before (say, 2 times out of 3), and then finishes downloading and gets into moving state, then it will be skipped. But if, of course, the download is at 99.999% and caught 3/3 times, it will be removed (since not completely downloaded yet)

SixFive7 commented 1 month ago

Yes, I just tried again on commit 9f8d196. Also tested the large download on it.

Everything looks perfect! 🥳

[INFO]: ##################################################
[INFO]: Decluttarr - Application Started!
[INFO]: 
[INFO]: Like this app? Thanks for giving it a ⭐️ on GitHub!
[INFO]: https://github.com/ManiMatter/decluttarr/
[INFO]: 
[INFO]: *** Current Settings ***
[INFO]: Version: dev
[INFO]: Commit: 9f8d196
[INFO]: 
[INFO]: True | Removing failed downloads
[INFO]: True | Removing downloads missing metadata
[INFO]: True | Removing downloads missing files
[INFO]: True | Removing downloads that fail on import (no format upgrade)
[INFO]: True | Removing orphan downloads
[INFO]: True | Removing slow downloads
[INFO]: True | Removing stalled downloads
[INFO]: True | Removing downloads belonging to unmonitored items
[INFO]: 
[INFO]: Running every: 0 days 0 hours 1.0 minutes
[INFO]: Minimum speed enforced: 100 KB/s
[INFO]: Permitted number of times before stalled/missing metadata/slow downloads are removed: 10
[INFO]: Downloads with this tag will be skipped: "Don't Kill"
[INFO]: Private Trackers will be skipped: True
[INFO]: 
[INFO]: *** Configured Instances ***
[INFO]: Radarr: http://media-radarr:7878/api/v3
[INFO]: Sonarr: http://media-sonarr:8989/api/v3
[INFO]: Lidarr: http://media-lidarr:8686/api/v1
[INFO]: Readarr: http://media-readarr:8787/api/v1
[INFO]: qBittorrent: http://media-qbittorrent:8080/api/v2
[INFO]: 
[INFO]: *** Check Instances ***
[INFO]: OK | Radarr
[INFO]: OK | Sonarr
[INFO]: OK | Lidarr
[INFO]: OK | Readarr
[INFO]: OK | qBittorrent
[INFO]: 
[INFO]: ##################################################
[INFO]: LOG_LEVEL = INFO: Only logging changes (switch to VERBOSE for more info)
[INFO]: >>> Detected missing metadata download (1 out of 10 permitted times): REDACTED-D
[INFO]: >>> Download no longer marked as missing metadata: REDACTED-D
[INFO]: >>> Detected slow download (1 out of 10 permitted times): REDACTED-A
[INFO]: >>> Detected slow download (1 out of 10 permitted times): REDACTED-B
[INFO]: >>> Detected slow download (2 out of 10 permitted times): REDACTED-A
[INFO]: >>> Detected slow download (2 out of 10 permitted times): REDACTED-B
[INFO]: >>> Detected slow download (3 out of 10 permitted times): REDACTED-A
[INFO]: >>> Detected slow download (3 out of 10 permitted times): REDACTED-B
[INFO]: >>> Detected slow download (4 out of 10 permitted times): REDACTED-A
[INFO]: >>> Detected slow download (4 out of 10 permitted times): REDACTED-B
[INFO]: >>> Detected slow download (5 out of 10 permitted times): REDACTED-A
[INFO]: >>> Detected slow download (5 out of 10 permitted times): REDACTED-B
[INFO]: >>> Download no longer marked as slow: REDACTED-A
[INFO]: >>> Download no longer marked as slow: REDACTED-B
[INFO]: >>> Detected slow download (1 out of 10 permitted times): REDACTED-B
[INFO]: >>> Detected slow download (2 out of 10 permitted times): REDACTED-B
[INFO]: >>> Detected slow download (3 out of 10 permitted times): REDACTED-B
[INFO]: >>> Detected slow download (4 out of 10 permitted times): REDACTED-B
[INFO]: >>> Detected slow download (5 out of 10 permitted times): REDACTED-B
[INFO]: >>> Detected slow download (6 out of 10 permitted times): REDACTED-B
[INFO]: >>> Detected slow download (7 out of 10 permitted times): REDACTED-B
[INFO]: >>> Detected slow download (8 out of 10 permitted times): REDACTED-B
[INFO]: >>> Detected slow download (9 out of 10 permitted times): REDACTED-B
[INFO]: >>> Detected slow download (10 out of 10 permitted times): REDACTED-B
[INFO]: >>> Detected slow download too many times (11 out of 10 permitted times): REDACTED-B
[INFO]: >>> Removing slow download: REDACTED-B
[INFO]: >>> Detected slow download too many times (12 out of 10 permitted times): REDACTED-B
[INFO]: >>> Removing slow download: REDACTED-B
[INFO]: >>> Detected missing metadata download (1 out of 10 permitted times): REDACTED-C
[INFO]: >>> Detected slow download too many times (13 out of 10 permitted times): REDACTED-B
[INFO]: >>> Removing slow download: REDACTED-B
[INFO]: >>> Detected missing metadata download (2 out of 10 permitted times): REDACTED-C
[INFO]: >>> Detected slow download too many times (14 out of 10 permitted times): REDACTED-B
[INFO]: >>> Removing slow download: REDACTED-B
[INFO]: >>> Detected missing metadata download (3 out of 10 permitted times): REDACTED-C
[INFO]: >>> Detected slow download too many times (15 out of 10 permitted times): REDACTED-B
[INFO]: >>> Removing slow download: REDACTED-B
[INFO]: >>> Download no longer marked as missing metadata: REDACTED-C
[INFO]: >>> Detected slow download too many times (16 out of 10 permitted times): REDACTED-B
[INFO]: >>> Removing slow download: REDACTED-B
ManiMatter commented 1 month ago

I'm not happy yet.

I'm missing a message that a given download has been completed (but is likely moving). Fixed that in the version I just pushed to dev. Can you pls re-run?

Also, what I don't understand is why we have "16 out of 10 permitted times", since we are not in a test-run. Have you followed this advice per README?

If you see strange errors such as "found 10 / 3 times", consider turning on the setting "Reject Blocklisted Torrent Hashes While Grabbing" on indexer-level (available in the nightly versions of the *arr apps)

SixFive7 commented 1 month ago

Oh sorry, completely overlooked that. First time I have seen "16 out of 10". I'll look into switching to nightly versions. I'm still trying to test with a large download. But it's difficult to time. I did spot something else though. When a download has just been added and it's still searching for metadata is is logging this:

[INFO]: >>> Detected slow download that has completed downloading - skipping from slow  (likely moving): REDACTED

If I call qbit's /v2/torrents/info at that exact time I get this result:

{
    "added_on": 1715633782,
    "amount_left": 0,
    "auto_tmm": false,
    "availability": 0,
    "category": "radarr",
    "completed": 0,
    "completion_on": 0,
    "content_path": "",
    "dl_limit": 0,
    "dlspeed": 0,
    "download_path": "/partials",
    "downloaded": 0,
    "downloaded_session": 0,
    "eta": 8640000,
    "f_l_piece_prio": false,
    "force_start": false,
    "hash": "REDACTED",
    "inactive_seeding_time_limit": -2,
    "infohash_v1": "REDACTED",
    "infohash_v2": "",
    "last_activity": 1715633782,
    "magnet_uri": "REDACTED",
    "max_inactive_seeding_time": -1,
    "max_ratio": -1,
    "max_seeding_time": 1,
    "name": "REDACTED",
    "num_complete": 7,
    "num_incomplete": 5,
    "num_leechs": 3,
    "num_seeds": 0,
    "priority": 6,
    "progress": 0,
    "ratio": 0,
    "ratio_limit": -2,
    "save_path": "/media/qbittorrent",
    "seeding_time": 0,
    "seeding_time_limit": -2,
    "seen_complete": 0,
    "seq_dl": false,
    "size": 0, <- could be downloading, could be not
    "state": "metaDL", <- not "downloading"
    "super_seeding": false,
    "tags": "",
    "time_active": 22,
    "total_size": -1, <- weird
    "tracker": "REDACTED",
    "trackers_count": 10,
    "up_limit": 0,
    "uploaded": 0,
    "uploaded_session": 0,
    "upspeed": 0
},

Maybe it's best to add an extra language agnostic condition: total_size == -1. Or forget the sizes and fall back to parsing language dependent state by checking state == downloading?

I'll try and setup some docker iops limitations to catch a moving downloaded during a decluttarr cycle.

ManiMatter commented 1 month ago

Already had that "state == downloading" check, but didn't wrap the new code into it.

Try again please :)

Thx for your help with this.

SixFive7 commented 1 month ago

Tested it again on commit 3b4b12d, together with a nightly prowlarr build and a large file system transfer. Everything checks out I think:

[INFO]: ##################################################
[INFO]: Decluttarr - Application Started!
[INFO]: 
[INFO]: Like this app? Thanks for giving it a ⭐️ on GitHub!
[INFO]: https://github.com/ManiMatter/decluttarr/
[INFO]: 
[INFO]: *** Current Settings ***
[INFO]: Version: dev
[INFO]: Commit: 3b4b12d
[INFO]: 
[INFO]: True | Removing failed downloads
[INFO]: True | Removing downloads missing metadata
[INFO]: True | Removing downloads missing files
[INFO]: True | Removing downloads that fail on import (no format upgrade)
[INFO]: True | Removing orphan downloads
[INFO]: True | Removing slow downloads
[INFO]: True | Removing stalled downloads
[INFO]: True | Removing downloads belonging to unmonitored items
[INFO]: 
[INFO]: Running every: 0 days 0 hours 1.0 minutes
[INFO]: Minimum speed enforced: 100 KB/s
[INFO]: Permitted number of times before stalled/missing metadata/slow downloads are removed: 10
[INFO]: Downloads with this tag will be skipped: "Don't Kill"
[INFO]: Private Trackers will be skipped: True
[INFO]: 
[INFO]: *** Configured Instances ***
[INFO]: Radarr: http://media-radarr:7878/api/v3
[INFO]: Sonarr: http://media-sonarr:8989/api/v3
[INFO]: Lidarr: http://media-lidarr:8686/api/v1
[INFO]: Readarr: http://media-readarr:8787/api/v1
[INFO]: qBittorrent: http://media-qbittorrent:8080/api/v2
[INFO]: 
[INFO]: *** Check Instances ***
[INFO]: OK | Radarr
[INFO]: OK | Sonarr
[INFO]: OK | Lidarr
[INFO]: OK | Readarr
[INFO]: OK | qBittorrent
[INFO]: 
[INFO]: ##################################################
[INFO]: LOG_LEVEL = INFO: Only logging changes (switch to VERBOSE for more info)
[INFO]: >>> Detected missing metadata download (1 out of 10 permitted times): REDACTED-A
[INFO]: >>> Detected missing metadata download (1 out of 10 permitted times): REDACTED-B
[INFO]: >>> Detected missing metadata download (1 out of 10 permitted times): REDACTED-C
[INFO]: >>> Detected missing metadata download (1 out of 10 permitted times): REDACTED-D
[INFO]: >>> Detected missing metadata download (2 out of 10 permitted times): REDACTED-A
[INFO]: >>> Detected missing metadata download (2 out of 10 permitted times): REDACTED-B
[INFO]: >>> Detected missing metadata download (2 out of 10 permitted times): REDACTED-C
[INFO]: >>> Detected missing metadata download (2 out of 10 permitted times): REDACTED-D
[INFO]: >>> Download no longer marked as missing metadata: REDACTED-C
[INFO]: >>> Detected missing metadata download (3 out of 10 permitted times): REDACTED-A
[INFO]: >>> Detected missing metadata download (3 out of 10 permitted times): REDACTED-B
[INFO]: >>> Detected missing metadata download (3 out of 10 permitted times): REDACTED-D
[INFO]: >>> Detected slow download (1 out of 10 permitted times): REDACTED-E
[INFO]: >>> Detected missing metadata download (4 out of 10 permitted times): REDACTED-A
[INFO]: >>> Detected missing metadata download (4 out of 10 permitted times): REDACTED-B
[INFO]: >>> Detected missing metadata download (4 out of 10 permitted times): REDACTED-D
[INFO]: >>> Detected slow download (2 out of 10 permitted times): REDACTED-E
[INFO]: >>> Download no longer marked as missing metadata: REDACTED-A
[INFO]: >>> Detected missing metadata download (5 out of 10 permitted times): REDACTED-B
[INFO]: >>> Detected missing metadata download (5 out of 10 permitted times): REDACTED-D
[INFO]: >>> Detected slow download (3 out of 10 permitted times): REDACTED-E
[INFO]: >>> Detected missing metadata download (6 out of 10 permitted times): REDACTED-B
[INFO]: >>> Detected missing metadata download (6 out of 10 permitted times): REDACTED-D
[INFO]: >>> Detected slow download (4 out of 10 permitted times): REDACTED-E
[INFO]: >>> Download no longer marked as missing metadata: REDACTED-D
[INFO]: >>> Detected missing metadata download (7 out of 10 permitted times): REDACTED-B
[INFO]: >>> Detected slow download (5 out of 10 permitted times): REDACTED-E
[INFO]: >>> Detected missing metadata download (8 out of 10 permitted times): REDACTED-B
[INFO]: >>> Detected slow download (6 out of 10 permitted times): REDACTED-E
[INFO]: >>> Detected missing metadata download (9 out of 10 permitted times): REDACTED-B
[INFO]: >>> Detected slow download (7 out of 10 permitted times): REDACTED-E
[INFO]: >>> Detected missing metadata download (10 out of 10 permitted times): REDACTED-B
[INFO]: >>> Detected slow download (8 out of 10 permitted times): REDACTED-E
[INFO]: >>> Detected missing metadata download too many times (11 out of 10 permitted times): REDACTED-B
[INFO]: >>> Removing missing metadata download: REDACTED-B
[INFO]: >>> Detected slow download (9 out of 10 permitted times): REDACTED-E
[INFO]: >>> Download no longer marked as missing metadata: REDACTED-B
[INFO]: >>> Detected slow download (10 out of 10 permitted times): REDACTED-E
[INFO]: >>> Detected slow download too many times (11 out of 10 permitted times): REDACTED-E
[INFO]: >>> Removing slow download: REDACTED-E
[INFO]: >>> Detected missing metadata download (1 out of 10 permitted times): REDACTED-F
[INFO]: >>> Download no longer marked as slow: REDACTED-E
[INFO]: >>> Download no longer marked as missing metadata: REDACTED-F
  --> This is the point in time where fast download REDACTED-G finished and started moving to another disk.
[INFO]: >>> Detected slow download that has completed downloading - skipping check (torrent files likely in process of being moved): REDACTED-G
[INFO]: >>> Detected slow download (1 out of 10 permitted times): REDACTED-H
[INFO]: >>> Download no longer marked as slow: REDACTED-H
[INFO]: >>> Detected slow download (1 out of 10 permitted times): REDACTED-I
[INFO]: >>> Download no longer marked as slow: REDACTED-I

What do you think?

ManiMatter commented 1 month ago

Yep looks great! Cheers

ps just released to „latest“ image

SixFive7 commented 1 month ago

Excellent. Thank you for all your work on this repository!

ManiMatter commented 1 month ago

quick question: how did you highlight this in color in your post above?

<- huh?

SixFive7 commented 1 month ago

Ha, with the dirtiest hack ever. I wrapped the code in a json code block with ```json at the start of the block. Then just added the comment. Because the comment fails to parse it is highlighted in red. Sometimes you have to tweak the comment so it is no valid json. But it works ;-)

ManiMatter commented 1 month ago
neat 

but how do you write

"total_size": -1, <- weird

And only color „weird“?

SixFive7 commented 1 month ago

I wouldn't know how to do that, sorry. I just posted this:

"total_size": -1, <- weird wrapped inside a ```json block. Resulting in this:

image