Closed delize closed 3 years ago
delize
commented
3 years ago @thalieht is there any additional information you need from me?
This is causing an issue for mainly because I am trying to reduce the time on disk once seed requirements have been met.
I have asked in the irc channel on freenode, if there is any additional information I should include.
Also, is there anything other further troubleshooting steps that I should take? Could this be config related?
delize
commented
3 years ago I was able to grab a log of a torrent that was added to qbittorrent from Sonarr showing any potential differences here.
With the client respecting the tackers and indexers ratio and seedlimits:
sonarr.trace.6.txt:21-3-5 22:51:52.8|Debug|QBittorrent|Downloading torrent for episode 'Pokemon.Journeys.S19E41.Pikachu.Translation.Check.1080p.NF.WEB-DL.DDP2.0.x264-LAZY' finished (12401 bytes from https://broadcasthe.net/torrents.php?action=download&id=1429572&authkey=(removed)&torrent_pass=(removed)
sonarr.trace.6.txt:21-3-5 22:51:52.8|Trace|HttpClient|Req: [POST] http://qbittorrent:8079/api/v2/torrents/add:
sonarr.trace.6.txt:21-3-5 22:51:52.8|Trace|HttpClient|Res: [POST] http://qbittorrent:8079/api/v2/torrents/add: 200.OK (1 ms)
sonarr.trace.6.txt:21-3-5 22:51:52.8|Trace|HttpClient|Req: [POST] http://qbittorrent:8079/api/v2/torrents/resume: hashes=3b865c3317a6de072faea35c16d61ff47fc7a191
sonarr.trace.6.txt:21-3-5 22:51:52.8|Trace|HttpClient|Res: [POST] http://qbittorrent:8079/api/v2/torrents/resume: 200.OK (1 ms)
sonarr.trace.6.txt:21-3-5 22:51:52.8|Trace|HttpClient|Req: [POST] http://qbittorrent:8079/api/v2/torrents/setShareLimits: hashes=3b865c3317a6de072faea35c16d61ff47fc7a191&ratioLimit=1.1&seedingTimeLimit=5080
sonarr.trace.6.txt:21-3-5 22:51:52.8|Trace|HttpClient|Res: [POST] http://qbittorrent:8079/api/v2/torrents/setShareLimits: 200.OK (1 ms)
sonarr.trace.6.txt:21-3-5 22:51:52.8|Info|DownloadService|Report sent to qbittorrent - nas. Pokemon.Journeys.S19E41.Pikachu.Translation.Check.1080p.NF.WEB-DL.DDP2.0.x264-LAZY{"added_on":1614984712,"amount_left":632953701,"auto_tmm":true,"availability":5.328999996185303,"category":"tv/sonarr-qb","completed":245760,"completion_on":21600,"content_path":"/pool/downloads/complete/tv/sonarr-qb/Pokemon.Journeys.S19E41.Pikachu.Translation.Check.1080p.NF.WEB-DL.DDP2.0.x264-LAZY.mkv","dl_limit":-1,"dlspeed":36248,"downloaded":181242,"downloaded_session":267052,"eta":87308,"f_l_piece_prio":false,"force_start":false,"hash":"3b865c3317a6de072faea35c16d61ff47fc7a191","last_activity":1614984717,"magnet_uri":"magnet:?xt=urn:btih:3b865c3317a6de072faea35c16d61ff47fc7a191&dn=Pokemon.Journeys.S19E41.Pikachu.Translation.Check.1080p.NF.WEB-DL.DDP2.0.x264-LAZY.mkv&tr=https%3a%2f%2flandof.tv%(removed)%2fannounce","max_ratio":1.1,"max_seeding_time":5080,"name":"Pokemon.Journeys.S19E41.Pikachu.Translation.Check.1080p.NF.WEB-DL.DDP2.0.x264-LAZY.mkv","num_complete":40,"num_incomplete":10,"num_leechs":1,"num_seeds":5,"priority":2,"progress":0.0003881241459237439,"ratio":0,"ratio_limit":1.1,"save_path":"/pool/downloads/complete/tv/sonarr-qb/","seeding_time_limit":5080,"seen_complete":1614984715,"seq_dl":false,"size":633199461,"state":"downloading","super_seeding":false,"tags":"","time_active":2,"total_size":633199461,"tracker":"https://landof.tv/(removed)/announce","trackers_count":1,"up_limit":-1,"uploaded":0,"uploaded_session":0,"upspeed":0}
A file slightly before this one was grabbed without setting the ratio and seedlimits properly:
sonarr.trace.16.txt:21-3-5 21:48:48.0|Debug|QBittorrent|Downloading torrent for episode 'Pokemon.Journeys.S19E37.That.New.Old.Gang.of.Mine.1080p.NF.WEB-DL.DDP2.0.x264-LAZY' finished (10761 bytes from https://broadcasthe.net/torrents.php?action=download&id=1429555&authkey=(removed)&torrent_pass=(removed)
sonarr.trace.16.txt:21-3-5 21:48:48.0|Trace|HttpClient|Req: [POST] http://qbittorrent:8079/api/v2/torrents/add:
sonarr.trace.16.txt:21-3-5 21:48:48.0|Trace|HttpClient|Res: [POST] http://qbittorrent:8079/api/v2/torrents/add: 200.OK (2 ms)
sonarr.trace.16.txt:21-3-5 21:48:48.0|Trace|HttpClient|Req: [POST] http://qbittorrent:8079/api/v2/torrents/resume: hashes=9e2b10abea7190cd065e30af25c8fa8cdc8d1e3c
sonarr.trace.16.txt:21-3-5 21:48:48.0|Trace|HttpClient|Res: [POST] http://qbittorrent:8079/api/v2/torrents/resume: 200.OK (1 ms)
sonarr.trace.16.txt:21-3-5 21:48:48.0|Trace|HttpClient|Req: [POST] http://qbittorrent:8079/api/v2/torrents/setShareLimits: hashes=9e2b10abea7190cd065e30af25c8fa8cdc8d1e3c&ratioLimit=1.1&seedingTimeLimit=5080
sonarr.trace.16.txt:21-3-5 21:48:48.0|Trace|HttpClient|Res: [POST] http://qbittorrent:8079/api/v2/torrents/setShareLimits: 200.OK (1 ms)
sonarr.trace.16.txt:21-3-5 21:48:48.0|Info|DownloadService|Report sent to qbittorrent - nas. Pokemon.Journeys.S19E37.That.New.Old.Gang.of.Mine.1080p.NF.WEB-DL.DDP2.0.x264-LAZY{"added_on":1614980928,"amount_left":545935831,"auto_tmm":true,"availability":9,"category":"tv/sonarr-qb","completed":1114112,"completion_on":21600,"content_path":"/pool/downloads/complete/tv/sonarr-qb/Pokemon.Journeys.S19E37.That.New.Old.Gang.of.Mine.1080p.NF.WEB-DL.DDP2.0.x264-LAZY.mkv","dl_limit":-1,"dlspeed":118693,"downloaded":614809,"downloaded_session":1185326,"eta":23426,"f_l_piece_prio":false,"force_start":false,"hash":"9e2b10abea7190cd065e30af25c8fa8cdc8d1e3c","last_activity":1614980935,"magnet_uri":"magnet:?xt=urn:btih:9e2b10abea7190cd065e30af25c8fa8cdc8d1e3c&dn=Pokemon.Journeys.S19E37.That.New.Old.Gang.of.Mine.1080p.NF.WEB-DL.DDP2.0.x264-LAZY.mkv&tr=https%3a%2f%2flandof.tv%(removed)%2fannounce","max_ratio":-1,"max_seeding_time":-1,"name":"Pokemon.Journeys.S19E37.That.New.Old.Gang.of.Mine.1080p.NF.WEB-DL.DDP2.0.x264-LAZY.mkv","num_complete":54,"num_incomplete":8,"num_leechs":1,"num_seeds":9,"priority":1,"progress":0.002036581877497792,"ratio":0,"ratio_limit":-2,"save_path":"/pool/downloads/complete/tv/sonarr-qb/","seeding_time_limit":-2,"seen_complete":1614980933,"seq_dl":false,"size":547049943,"state":"downloading","super_seeding":false,"tags":"","time_active":4,"total_size":547049943,"tracker":"https://landof.tv/(removed)/announce","trackers_count":1,"up_limit":-1,"uploaded":0,"uploaded_session":0,"upspeed":0}
delize
commented
3 years ago Is there rate limiting or anything that would prevent successive messages from running separate commands against the API endpoint?
delize
commented
3 years ago Dev from Sonarr/Sonarr is asking the best approach here due to this issue: https://github.com/Sonarr/Sonarr/issues/4360#issuecomment-791987384
Could a dev possibly follow up or respond?
FranciscoPombal
commented
3 years ago @glassez ping
glassez
commented
3 years ago @glassez ping
Please don't ping me with these types of Issues until someone confirms that this is really an qBittorrent web API issue and not some kind of client-side bug.
FranciscoPombal
commented
3 years ago @glassez
Please don't ping me with these types of Issues until someone confirms that this is really an qBittorrent web API issue and not some kind of client-side bug.
I mean, OP clearly did some due diligence already, at least they should be acknowledged, no? I only pinged you specifically because this could be a WebAPI issue, and you prefer handling these.
delize
commented
3 years ago For what it is worth, (without requiring anyone to go the Sonarr comment previously mentioned, I am pasting it below) this is what we are waiting on confirmation of to see if it is a client side bug or due to how the API works, as there is no documentation based around the issue I (and others) are seeing.
Sonarr should not have to guess any internal processing delays. There's no remark in the qbit api documentation that adds are completed asynchronously. So we would require clarification from the qbit staff first.
Sonarr POSTs the torrent via the api call 'add':
https://github.com/qbittorrent/qBittorrent/blob/15f1fdddd991126829714f1a5715d38d0accf8e7/src/webui/api/torrentscontroller.cpp#L673
Followed by a resume call apparently, finally by a setShareLimits. Both of which use:
https://github.com/qbittorrent/qBittorrent/blob/15f1fdddd991126829714f1a5715d38d0accf8e7/src/webui/api/torrentscontroller.cpp#L132
So if setShareLimits fails, then resume likely will have failed as well.
Both of which use BitTorrent Session, inspecting that:
addTorrent: https://github.com/qbittorrent/qBittorrent/blob/master/src/base/bittorrent/session.cpp#L2045
https://github.com/qbittorrent/qBittorrent/blob/master/src/base/bittorrent/session.cpp#L2084
https://github.com/qbittorrent/qBittorrent/blob/master/src/base/bittorrent/session.cpp#L2190
https://github.com/qbittorrent/qBittorrent/blob/master/src/base/bittorrent/session.cpp#L2193
Line 2193 adds it to loadingTorrents, which would cause the torrent to be invisible for a while (since findTorrent doesn't check that list).
However, this is something that was changed in qbit about 3 months ago: qbittorrent/qBittorrent@a93b675#diff-0efd426259b58f47b1f848560cf9c51a92268084060cebf91c7ef22f092da1f7R2239
So it's probably better to ask qbit devs about this, and to see what the correct approach is going forward.Either way, it might be helpful if someone also reviews the whole issue at https://github.com/Sonarr/Sonarr/issues/4360 - as there is also a substantial amount of debugging there to justify a response (in my opinion anyways).
glassez
commented
3 years ago There's no remark in the qbit api documentation that adds are completed asynchronously.
Yes, currently torrents are added asynchronously. Sorry, documentation is one of our weak points.
Sonarr POSTs the torrent via the api call 'add'. Followed by a resume call apparently, finally by a setShareLimits.
Given the above, this can't be considered reliable until you are sure that the torrent has already been added.
So if setShareLimits fails, then resume likely will have failed as well.
Seems Yes.
You can add torrent in "resumed" state by passing "paused=false" parameter. qBittorrent core also provide parameters to set initial values for "share limits" but they're still unavailable from web API (so we probably need to provide them ASAP).
Taloth
commented
3 years ago It used to add them synchronously till 3 months ago. Isn't it an option to show the torrent in a loading state instead? I'm not sure what properties are unavailable while it's being loaded, but it'd certainly make it more predictable.
Coz you're basically saying that we need to poll the info or properties api till the torrent we just added appears.
glassez
commented
3 years ago It used to add them synchronously till 3 months ago
Neither three months ago, nor even three years ago. Apparently, you just haven't encountered it. In fact, the time to add a torrent is quite short, if qBittorrent is not heavily loaded.
Isn't it an option to show the torrent in a loading state instead?
In fact, I've been thinking about this lately (in connection with planning other improvements). But it requires quite a lot of refactoring. At a minimum, I need to implement stubs for most getters/setters of torrent. And I care more about setters. Many of them will not be able to correctly apply the changes until the torrent is fully loaded. So they should either just ignore the impact, or we will have to implement some kind of deferred actions queue (which I would really not like).
Taloth
commented
3 years ago This commit from 3 months ago (I linked to it earlier) https://github.com/qbittorrent/qBittorrent/commit/a93b675cb85e79f7a9666aba816a09f69f655941#diff-0efd426259b58f47b1f848560cf9c51a92268084060cebf91c7ef22f092da1f7L2212 dumps the loading torrent into m_loadingTorrents rather than calling loadTorrents, this means findTorrent doesn't see it in m_torrents. Till loadTorrents is called by the async fileSearchFinished.
I'm not familiar with qbits codebase sufficiently to say with confidence that's the change that really made it async, but it certainly looks like it.
And yes, as a fellow dev I understand the challenges of adding a loading state, but please understand that from our perspective the commit mentioned is a breaking change. It makes the api unpredictable, and the 'conversation' is basically: "pick up this page" "OK" "now flip the page" "What page?!?" The api should either wait for the torrent to appear, or have a loading state.
glassez
commented
3 years ago I'm not familiar with qbits codebase sufficiently to say with confidence that's the change that really made it async, but it certainly looks like it
So why don't you believe the one who is familiar with qBittorrent codebase and moreover who done the changes mentioned above?
The api should either wait for the torrent to appear, or have a loading state.
Sorry, WebAPI is something we care about in one of the last queues due to lack of resources. It is mainly intended to serve the embedded web UI. Yes, it has many disadvantages for general purpose use. Perhaps we will implement a better API in the future, if someone helps us with this. So far, I can only provide #14519.
delize
commented
3 years ago When #14519 is released/made GA in v4.3.4, I will test and report back if that does indeed resolve the issue going forward in the short term. While the answer for the longer term is what you mentioned, longer term development of the WebAPI/API platform.
Thanks very much for the quick fix glassez (not going to @ you). :)
Taloth
commented
3 years ago Thank you, I'll make sure the new api version is handled appropriately in Sonarr.
Taloth
commented
3 years ago I've added the necessary code to Sonarr to call the new api and tested it with the nightly build. For existing qbit versions I've added code to check if the torrent was added successfully before issuing further commands.
Glassez, my apologies for my responses earlier, it was shortsighted of me and I should've addressed you differently. Thank you for your effort.
delize
commented
3 years ago Just following up, on v4.3.4.1 this seems to be working great and gets around any potential lag issues that existed while on 4.3.3.
Closing this as it appears to be fixed unless anyone would like to keep this open (feel free to reopen / or ask me to reopen).
Please provide the following information
qBittorrent version and Operating System
qBittorrent v4.3.3 Web UI (64-bit) / Docker from:
ghcr.io/linuxserver/qbittorrentIf on linux, libtorrent-rasterbar and Qt version
What is the problem
Sonarr sends seedingTimeLimit via API, but qbittorrent is not respecting the details and does not show the configuration properly in JSON query/response to the torrent. This occurs if the following setting(s):
Are checked or unchecked (I have tried both configurations, neither have worked properly).
Sonarr sends the post query to qbittorrent:
You can see that the
ratioLimitandseeding_time_limitare still set to-2.With that said, sometimes the torrent behavior is respected, but it does not seem to be consistent. Within a 24 hour time period, I had 5 torrents submitted on Sunday night, with only one of them actually respecting the configuration. Unfortunately, I do not have a log of that specific torrent - but if it occurs again, I hope to have one.
What is the expected behavior
qbittorrent should respect the seedingTimeLimit behavior sent in the post query, and the post query should override the configuration that is set in the configuration of the WebUI. Setting the
seeding_time_limit,max_seeding_time, orratio_limitSteps to reproduce
Configure Sonarr & qbittorrent with docker-compose file.
Extra info(if any)
I have been troubleshooting this issue with Sonarr support for 3 - 4 weeks, making sure that it isn't configuration issues, time based issues, etc.
https://github.com/Sonarr/Sonarr/issues/4360