search

kiwix / libkiwix

Common code base for all Kiwix ports
https://download.kiwix.org/release/libkiwix/
GNU General Public License v3.0
112 stars 54 forks source link

kiwix::Downloader may miss a quickly finished download #1049

Closed veloman-yunkan closed 4 months ago

veloman-yunkan commented 4 months ago

kiwix::Downloader only intercepts (and caches) downloads while they are in waiting or active state. As a result a small download that is completed before an access to it via kiwix::Downloader is attempted will never be returned.

kelson42 commented 4 months ago

@veloman-yunkan This sounds a very serious problem, even if user reports mostly problems with big downloads. Eventually, this bug should probably be fix ASAP... Or do I miss something?

veloman-yunkan commented 4 months ago

I will fix it together with the other download-related bugs in kiwix-desktop (on which I am working currently).

mgautierfr commented 4 months ago

While you are working on this, you may want to investigate the JSON-RPC over WebSocket of aria2 (https://aria2.github.io/manual/en/html/aria2c.html#rpc-interface)

We would need a client library that support WebSocket, but it would allow aria to send notification back to us, and so we would have to pull information, just wait for notification.

veloman-yunkan commented 4 months ago

Well, this ticket turned out to be a false alarm caused by a problem I was seeing in kiwix-desktop. I was quick to blame it on kiwix::Downloader after looking only at Downloader::getDownload() and noticing that when called with a download-id missing from its cache it only checks active and waiting downloads:

https://github.com/kiwix/libkiwix/blob/1ba588272c91e8ec59847311e1cb083be6a3a316/src/downloader.cpp#L184-L204

Now I see that download-ids are added to the cache as soon as they are obtained from aria in Downloader::startDownload():

https://github.com/kiwix/libkiwix/blob/1ba588272c91e8ec59847311e1cb083be6a3a316/src/downloader.cpp#L169-L182

Since the download cache is never pruned the bug observed in kiwix-desktop should have a different explanation, the most likely one (though also a purely hypothetical one, as now I can't reproduce the bug) being that completion of the download was properly detected but improperly handled - the download id registered with the book was reset but the download kept being queried with that empty invalid id.