search

avluis / Hentoid

Doujinshi Android App
https://discord.gg/QEZ3qk9
Apache License 2.0
1.01k stars 83 forks source link

Can't complete Hitomi download after task added a long time #1024

Open Retia-Adolf opened 1 year ago

Retia-Adolf commented 1 year ago

Hentoid Version: 1.17.0

Device/Android Version: Android 13

Issue details / Repro steps: Hitomi download links seem composed of temporary access token. When I tried downloading from Hitomi via a slow proxy (several hundred kB/s to download a several hundred MBs task), it seems download links became invalid after a long time. While after that I'm still able to access manga webpage both gallery and web reader, and while there's speed shows in queue, the task just stop there and never progress, and error log (below) said "Network error 404" (tried manually access those links in web browser I actually got "403 Forbidden").

While for small size tasks I can just delete it and download from beginning again ("Redownload from scratch" doesn't work in this case for me btw), for large tasks it seems just prone to this link invalid error happen again, if I can't get a quicker network connection.

I think it would be nice if Hentoid could get new links to replace invalid links for queued tasks, then it is possible to continue in-progress tasks without re-try from beginning.

Stack trace / LogCat:

Error log : begin
Hentoid ver: 1.17.0 (595)
API: 33
Device: Phone 2
Start : 2022-12-14T12:27:27.010Z
End : 2022-12-14T12:27:47.717Z (00:20)
-----
Error log for Night of Revenge [2399180@hitomi]: 41 errors
2022-12-14T20:27:27.01 04 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://ba.hitomi.la/webp/1670990402/130/349913986879111c95537d94fee459f5cc136edea242030b8a2ed0286e9e1820.webp
2022-12-14T20:27:27.019 06 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://aa.hitomi.la/webp/1670990402/1017/db3a813f603dcd2e71288fcfebb6bea4d19df3ebf0f8307eb77a341ab87b1f93.webp
2022-12-14T20:27:28.037 07 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://aa.hitomi.la/webp/1670990402/920/77852fb97405c410aa65a4d12c0275da640b2477a42e859185e4223030250983.webp
2022-12-14T20:27:28.044 09 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://aa.hitomi.la/webp/1670990402/3994/a035111e8799ddfb9c4e76ca4b5e8da023c5f38856481eb45592d18da8a829af.webp
2022-12-14T20:27:29.076 11 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://ba.hitomi.la/webp/1670990402/2131/2dd066414641bdb7d7df4110d1f5ced825229c960237b9a1f9f2b1062db26538.webp
2022-12-14T20:27:29.079 10 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://aa.hitomi.la/webp/1670990402/2815/296d44314d5b04f2bd06cf501c04b5b28ba2a34890d43784cb4aba721dbf7ffa.webp
2022-12-14T20:27:30.123 16 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://aa.hitomi.la/webp/1670990402/3151/010714d424b327e68fe6cc7df430275711abdfec0eebef209ded24464f73d4fc.webp
2022-12-14T20:27:30.125 17 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://ba.hitomi.la/webp/1670990402/761/f8b7b81335d8674dfc1acdaef4f49f63a81780f33d1169fd356771411622cf92.webp
2022-12-14T20:27:31.169 18 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://aa.hitomi.la/webp/1670990402/2595/704f3819dec423e751459779ec3cf207ccc400f70ebbc7865156fd7bdf34923a.webp
2022-12-14T20:27:31.176 19 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://ba.hitomi.la/webp/1670990402/1004/3f192dc07b78089fa2f78151b2c9a2d9c28c1caa7be7bd5a8630503fda098ec3.webp
2022-12-14T20:27:32.187 20 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://ba.hitomi.la/webp/1670990402/2121/97f253eda726a2a159857bc78fc72c6ac698644a5749988880027d04b2d8f498.webp
2022-12-14T20:27:32.211 21 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://ba.hitomi.la/webp/1670990402/3528/a1290cfcdb41535290d29d7feb4d781004863289dbfa845a0a0b1b061177cc8d.webp
2022-12-14T20:27:33.2 23 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://ba.hitomi.la/webp/1670990402/1591/4b7ff91201719202d4984652e154c60165f667812587a3a785ad876f341e7376.webp
2022-12-14T20:27:33.22 22 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://ba.hitomi.la/webp/1670990402/1101/2a99ad17ae6f8f865bb1cf72db17f4ee039cecfa61750ba274a7dc99044a64d4.webp
2022-12-14T20:27:34.235 25 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://aa.hitomi.la/webp/1670990402/1511/46fbacdc0355f6df133633514d5564261b29160c7e47797ac1638ead02349e75.webp
2022-12-14T20:27:34.238 24 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://ba.hitomi.la/webp/1670990402/1715/961c4c3311101020e92e7166a75357f3d63af7cc1f306aa2f35da01d377bab36.webp
2022-12-14T20:27:35.279 26 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://aa.hitomi.la/webp/1670990402/3128/ed3bc2684ee8784b7a1ce715145c7e22d06966fac5cf7722a531ad380f87b38c.webp
2022-12-14T20:27:35.281 27 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://ba.hitomi.la/webp/1670990402/3030/e0878b7cdbd0c72cc8edd8628ce09ac24bfb1a06bb59f7e4fbc7e5323e7dcd6b.webp
2022-12-14T20:27:36.299 30 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://ba.hitomi.la/webp/1670990402/1113/202267a4d4713388a398adc5120b701abd6b918fe2a68f673b50682836937594.webp
2022-12-14T20:27:36.302 29 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://ba.hitomi.la/webp/1670990402/314/6f9690e33fa2f300a6a0fe861dcb42bf2e602ad0699aaa46cfb7a5e6c90f83a1.webp
2022-12-14T20:27:37.324 31 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://aa.hitomi.la/webp/1670990402/925/f93aafb76d3a697f92e0a7f6d432737d7798333946cf349999501d58ed76d9d3.webp
2022-12-14T20:27:37.731 32 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://ba.hitomi.la/webp/1670990402/699/3f699cfbf016f29647153f791bb36b7a2d924c5d258ab9ab6108fc3f3

@AVnetWS/admin-team

RobbWatershed commented 1 year ago

Hitomi download links seem composed of temporary access token. When I tried downloading from Hitomi via a slow proxy (several hundred kB/s to download a several hundred MBs task), it seems download links became invalid after a long time

Exactly. Your diagnosis is correct.

While I completely understand your request, what you're asking is very specific.

Best way to implement that would be something like "regenerate missiong pages" instead of "redownload from scratch". I'm adding that to the roadmap in case someone else expressed the same need.

Retia-Adolf commented 1 year ago

While I completely understand your request, what you're asking is very specific.

Does it means that currently only Hitomi has such limit? I only know nhentai urls are not temporary though.

RobbWatershed commented 5 months ago

Does it means that currently only Hitomi has such limit? I only know nhentai urls are not temporary though

Yes, we've only seen that on Hitomi so far