ccloli / E-Hentai-Downloader

Download E-Hentai archive as zip file
GNU General Public License v3.0
1.88k stars 138 forks source link

Provide an option to continue downloading even if an exceed limits error is encountered #190

Closed yamada95 closed 3 years ago

yamada95 commented 3 years ago

I'm using multiple proxy servers to access ehentai at the same time, each network request to ehentai is sent through a random proxy server. Each proxy server should have 5000 credits initially, but as the downloading progresses, the servers may vary in credits quite dramatically, some of them may run out of credits while others still remain a considerable amount.

What I would like to see is, if a request is sending to a server that already ran out of credits, there is an option for us to ignore the error and continue the downloading, then the next request will be sent through a different proxy server, hopefully.

ccloli commented 3 years ago

I don't think it's a common feature, so I'm not willing to add it. If you do want it, you can modify the source code by yourself.

Open the source code and search for status: 'Failed! (Exceed Limits)' , and go down for a few lines you'll see a for loop statement, add return failedFetching(index, nodeList); after it, save and reload all the gallery pages, and it should achieve your goal.

image

However, keep in mind that it's not recommend to do this for normal usage, and requesting with the ip that out of viewing limit will still remains to be counted.

yamada95 commented 3 years ago

Thanks for the detailed response! It's totally fine to me to modify the script manually.

I'm using a similar workaround but with the forced param set to true, and it fits my need perfectly:

return failedFetching(index, nodeList, true);

And I wonder what does the forced param mean in the context of an exceed limits failure? Is it safe to ignore this param in this specific context?

ccloli commented 3 years ago

If forced is set to false or not set, the script will retry the image URL as normal. If you don't change the settings, it will try to redownload the image with the same URL for about 3 times, and if all of them are failed, it'll be added to retry queue.

When setting forced to true, the script will stop retrying immediately and adds the image to retry queue.

When the script handles retry queue, the script will try getting a new image URL, but for this time, the script will try requesting the image URL without H@H network by default, which will cost more image limits (+5 more than normal request).

If the script fails immediately, you may need to spend more image viewing limits to get new image URL, but for your case, the failed image URL could be loaded by another IP which doesn't used up limits.

So in theory, setting forced to true or false has no difference in what you want, but setting to true may cost you more limits.

yamada95 commented 3 years ago

I've experienced a little bit with a false value, but encountered a strange issue, the extension shown the number of Downloading images is 1 while I found there was none (all the failed messages are in red).

But never mind, I can keep using a true value or just restart the download.