search

avluis / Hentoid

Doujinshi Android App
https://discord.gg/QEZ3qk9
Apache License 2.0
1.03k stars 83 forks source link

Issues redownloading merged books when exported #1104

Closed mish20011 closed 8 months ago

mish20011 commented 10 months ago
So basically I got a new phone, I eported bookmarks and settings while movig my 38gb hentai out of my phone (and having issue puting it in my new phone due to the large file size where it doesn't move files at all so I had to do it bit by bit). So when I imported the books I chose add books even when empty which includes the merged books, so when I try to redownload it (coz why should I move the other books when I can just redownload them, except for the purged ones like from ksk and ehentai). This is where the problem comes, since I merged the books the downloader has problems trying to find the missing pages ie. the book I am trying to download is 20 pages but since I merged its 5 chapters in a total of 100 pages, the download becomes error as it can't find the sources to download the remaining 80 pages. And what's worse is I can't fix it by redownloading from scratch because for some reason it still has the amout of pages when I merged it.

Hentoid Version: 1.18.7

Device/Android Version: Freeyond M5, as this is my new phone

Issue details / Repro steps:

Merge a book with another, export it to another device, Import ad empty book, redownload it from source, get error because no pages found to redownload the other half of the merged book.

Stack trace / LogCat:

paste stack trace and/or log here

Error log : begin Hentoid ver: 1.18.7 (681) API: 31 Device: M5 Start : 2023-10-20T15:48:11.747Z End : 2023-10-20T15:48:19.937Z (00:08)

Error log for Yuuwaku・Imouto #1 Onii-chan chi ni Otomari | Little Sister Temptation #1 Staying at Onii-chan's apartment [70713@Hentaifox] : 101 errors 2023-10-20T23:48:11.747 022 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/22. 2023-10-20T23:48:11.755 024 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/24. 2023-10-20T23:48:11.756 025 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/25. 2023-10-20T23:48:11.796 023 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/23. 2023-10-20T23:48:12.021 026 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/26. 2023-10-20T23:48:12.062 028 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/28. 2023-10-20T23:48:12.073 027 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/27. 2023-10-20T23:48:12.123 029 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/29. 2023-10-20T23:48:12.285 030 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/30. 2023-10-20T23:48:12.325 031 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/31. 2023-10-20T23:48:12.373 032 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/32. 2023-10-20T23:48:12.483 033 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/33. 2023-10-20T23:48:12.555 034 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/34. 2023-10-20T23:48:12.571 036 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/36. 2023-10-20T23:48:12.625 037 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/37. 2023-10-20T23:48:12.816 038 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/38. 2023-10-20T23:48:12.833 039 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/39. 2023-10-20T23:48:12.841 040 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/40. 2023-10-20T23:48:12.955 041 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/41. 2023-10-20T23:48:13.087 042 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/42. 2023-10-20T23:48:13.15 035 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/35. 2023-10-20T23:48:13.152 043 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/43. 2023-10-20T23:48:13.24 044 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/44. 2023-10-20T23:48:13.361 045 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/45. 2023-10-20T23:48:13.435 046 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/46. 2023-10-20T23:48:13.495 047 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/47. 2023-10-20T23:48:13.54 048 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/48. 2023-10-20T23:48:13.63 049 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/49. 2023-10-20T23:48:13.726 050 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/50. 2023-10-20T23:48:13.782 051 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/51. 2023-10-20T23:48:13.868 052 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/52. 2023-10-20T23:48:13.869 053 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/53. 2023-10-20T23:48:14.019 054 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/54. 2023-10-20T23:48:14.145 056 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/56. 2023-10-20T23:48:14.146 057 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/57. 2023-10-20T23:48:14.299 058 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/58. 2023-10-20T23:48:14.416 059 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/59. 2023-10-20T23:48:14.424 060 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/60. 2023-10-20T23:48:14.555 061 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/61. 2023-10-20T23:48:14.603 055 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/55. 2023-10-20T23:48:14.674 062 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/62. 2023-10-20T23:48:14.748 063 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/63. 2023-10-20T23:48:14.802 064 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/64. 2023-10-20T23:48:14.929 065 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/65. 2023-10-20T23:48:15.009 066 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/66. 2023-10-20T23:48:15.01 067 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/67. 2023-10-20T23:48:15.075 068 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/68. 2023-10-20T23:48:15.225 069 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/69. 2023-10-20T23:48:15.279 070 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/70. 2023-10-20T23:48:15.329 071 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/71. 2023-10-20T23:48:15.404 072 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/72. 2023-10-20T23:48:15.491 073 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/73. 2023-10-20T23:48:15.572 074 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/74. 2023-10-20T23:48:15.637 075 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/75. 2023-10-20T23:48:15.699 076 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/76. 2023-10-20T23:48:15.791 077 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/77. 2023-10-20T23:48:15.886 078 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/78. 2023-10-20T23:48:15.94 079 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/79. 2023-10-20T23:48:15.942 080 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/80. 2023-10-20T23:48:16.084 081 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/81. 2023-10-20T23:48:16.16 082 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/82. 2023-10-20T23:48:16.2 083 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/83. 2023-10-20T23:48:16.339 085 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/85. 2023-10-20T23:48:16.341 084 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/84. 2023-10-20T23:48:16.43 086 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/86. 2023-10-20T23:48:16.502 087 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/87. 2023-10-20T23:48:16.603 088 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/88. 2023-10-20T23:48:16.687 089 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/89. 2023-10-20T23:48:16.732 090 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/90. 2023-10-20T23:48:16.74 091 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/91. 2023-10-20T23:48:16.887 092 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/92. 2023-10-20T23:48:16.962 093 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/93. 2023-10-20T23:48:16.976 094 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/94. 2023-10-20T23:48:17.084 095 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/95. 2023-10-20T23:48:17.162 096 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/96. 2023-10-20T23:48:17.243 097 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/97. 2023-10-20T23:48:17.336 098 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/98. 2023-10-20T23:48:17.346 099 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/99. 2023-10-20T23:48:17.499 100 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/100. 2023-10-20T23:48:17.501 101 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/101. 2023-10-20T23:48:17.601 102 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/102. 2023-10-20T23:48:17.648 103 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/103. 2023-10-20T23:48:17.759 105 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/105. 2023-10-20T23:48:17.828 104 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/104. 2023-10-20T23:48:17.999 107 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/107. 2023-10-20T23:48:18.059 108 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/108. 2023-10-20T23:48:18.075 109 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/109. 2023-10-20T23:48:18.275 110 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/110. 2023-10-20T23:48:18.325 111 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/111. 2023-10-20T23:48:18.404 106 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/106. 2023-10-20T23:48:18.411 112 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/112. 2023-10-20T23:48:18.551 113 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/113. 2023-10-20T23:48:18.571 114 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/114. 2023-10-20T23:48:18.708 115 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/115. 2023-10-20T23:48:18.754 116 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/116. 2023-10-20T23:48:18.84 117 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/117. 2023-10-20T23:48:18.882 118 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/118. 2023-10-20T23:48:19.016 119 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i.hentaifox.com/003/1664245/119. 2023-10-20T23:48:19.065 120 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/120. 2023-10-20T23:48:19.079 121 - [Networking]: Network error; HTTP statusCode=404; message=Network error 404 @ https://i2.hentaifox.com/003/1664245/121. 2023-10-20T23:48:19.937 pages - [Parsing]: The number of downloaded images (21) does not match the book's number of pages (121) @ https://hentaifox.com/gallery/70713/ Error log : end

Screenshot_20231020233526

@AVnetWS/admin-team

RobbWatershed commented 10 months ago

We do have an issue about that, yep. Thanks for your feedback !

RobbWatershed commented 9 months ago

I'm currently working on your issue.

To simplify what follows, let's consider you merged books A and B into book C.

Inside book C, A and B are called chapters.

The best case where everything works smoothly is the case where every chapter still exists online, with the same URL as the one you initially downloaded them from. There's nothing specific to say about that.

Problem is a few things can get wrong over time :

...which brings me to a question : What is the app supposed to do when chapter A is online and downloadable but chapter B is nowhere to be found?

Simplest answer is "download what can be downloaded and leave the rest empty" but that would mean the book would be stuck in the errors tab forever, which is not what you want.

Another option I can think of is "drop undownloadable chapters", which would download a valid book, albeit with missing chapters.

Would that be okay to you?

RobbWatershed commented 8 months ago

The case where every chapter still exists online, with the same URL as the one you initially downloaded them from, has been implemented and will be supported by next release.

For any other case, the app will refuse to queue the book for redownloading.

mish20011 commented 8 months ago

I'm currently working on your issue.

To simplify what follows, let's consider you merged books A and B into book C.

Inside book C, A and B are called chapters.

The best case where everything works smoothly is the case where every chapter still exists online, with the same URL as the one you initially downloaded them from. There's nothing specific to say about that.

Problem is a few things can get wrong over time :

* A book can disappear from its original site (e.g. DMCA takedown or site dropping dead)

* The URL of a book and/or its pages can evolve over time (e.g. Hitomi)

...which brings me to a question : What is the app supposed to do when chapter A is online and downloadable but chapter B is nowhere to be found?

Simplest answer is "download what can be downloaded and leave the rest empty" but that would mean the book would be stuck in the errors tab forever, which is not what you want.

Another option I can think of is "drop undownloadable chapters", which would download a valid book, albeit with missing chapters.

Would that be okay to you?

yeah, dropping downloadable chapters is the best option

mish20011 commented 8 months ago

The case where every chapter still exists online, with the same URL as the one you initially downloaded them from, has been implemented and will be supported by next release.

For any other case, the app will refuse to queue the book for redownloading.

thanks!

RobbWatershed commented 8 months ago

Available in today's v1.18.14