search

ccloli / E-Hentai-Downloader

Download E-Hentai archive as zip file
GNU General Public License v3.0
1.88k stars 138 forks source link

IP banned because of harvesting software #191

Open maxcmonster opened 3 years ago

maxcmonster commented 3 years ago

My IP had been banned for several times, after using -Hentai-Downloader. I had set up download to just 1x picture for each time, and small size picture. But it seems system still indicate me as harvesting software, which shows: Your IP address has been temporarily banned for excessive pageloads which indicates that you are using automated mirroring/harvesting software. So I have to give up to use the downloader now.

ccloli commented 3 years ago

Sorry to hear that, are you using the latest v1.33.2? As #133 discussed, there's a known bug that the script may send a bunch of requests to get image limits due to Chrome's background tabs optimization, and has been fixed in latest v1.33.2.

The checklist in #133 may help you to figure out what's wrong, and I'll check if v1.33.2 still send multiple requests in the background later today.

maxcmonster commented 3 years ago

Yes, my downloader was auto updated to 1.33.2 BTW: I had set up download to just 1x picture for each time, and only start 1x downloader.

ccloli commented 3 years ago

By opening some galleries and leaving these tabs in the background for about an hour, I don't see the script send any requests. So at least the bug on < 1.33.2 that send many requests has already been fixed, and I've no clue what's wrong.

Maybe you can setting Delay _____ second(s) before requesting next image to prevent the script getting multiple image urls at the same time, and check if you're using a shared IP with others, or if any other devices are also browsing E-Hentai.

maxcmonster commented 3 years ago

Set delay to 1 second, not banned until now. Seems it works. BTW: Could delay time set below 1 second?

ccloli commented 3 years ago

Could delay time set below 1 second?

Yep, you can set it like 0.5.

maxcmonster commented 3 years ago

Banned again, it seems website is not just checking delay time, but also download pages: Your IP address has been temporarily banned for excessive pageloads which indicates that you are using automated mirroring/harvesting software. The ban expires in 2 days and 15 hours

ccloli commented 3 years ago

If it happens when you're downloading, maybe your internet speed is too fast, you can add an extra delay, but if not, then I've no clue what's going on.

As I previous comment on https://github.com/ccloli/E-Hentai-Downloader/issues/133#issuecomment-858825020, for my testing 40 requests per minute should be fine, but it may be vary. As for now I'm still doing the same task every hour on one of my static IP server, and so far I don't see my IP is getting banned.

Some extra notes from E-Hentai offical wiki:

"You are opening pages too fast, thus placing a heavy load on the server. Back down, or your IP address will be automatically banned."

  • This may be triggered by prefetching in Chrome; uncheck Settings -> Show advanced settings -> Privacy -> "Use a prediction service to load pages more quickly."
  • This may be triggered by add-ons/scripts that organize/update bookmarks to galleries automatically; such actions may induce page loads.

"Your IP address has been temporarily banned for using automated mirroring/harvesting software and/or failing to heed the overload warning"

  • You have opened too many pages at once or you have used software that downloaded too many images from the site simultaneously.
  • Wait until the ban expires.
  • If you are certain that you have not performed excessive page loads your IP may be in use by other people.

Or maybe you can use some other tools to see if that happens, like https://github.com/mikf/gallery-dl . And please note that you can download an gallery with E-Hentai's offical download ways, like using torrents to download if the gallery has torrents uploaded, or if you have enough GPs or credits, you can use the offical zip archive download or download with Hentai@Home.

maxcmonster commented 3 years ago

You are right. Since 1.5s waiting time was set for safety, not banned any more. Thanks a lot.