search

Bionus / imgbrd-grabber

Very customizable imageboard/booru downloader with powerful filenaming features.
https://www.bionus.org/imgbrd-grabber/
Apache License 2.0
2.51k stars 216 forks source link

Danbooru stopped working #3078

Open AndreiAdii opened 9 months ago

AndreiAdii commented 9 months ago

Steps to reproduce

  1. Search for any tag using Danbooru source
  2. Images won't load, neither the thumbnails
  3. Check Logs

Expected behavior Thumbnails and images should load

Context main.log settings.txt Renamed settings.ini to .txt for uploading

Screenshots image

System information

Bionus commented 9 months ago

Seems to be related to this: https://danbooru.donmai.us/forum_topics/25845

Some of you may have noticed that images have been slow to load or sometimes failing to load completely over the last few weeks. That's because over 30% of our bandwidth is taken up by bots and scrapers now and the image servers can no longer take it. So I'm blocking all bots to see how well the site performs with only real humans. It turns out that made things significantly better, reducing average response times from 2 to 4 seconds to 0.5 seconds.

I will gradually loosen the rules to try to block fewer real users and benign bots. But in general, traffic from real people takes priority over bots. There are too many AI bros trying to scrape the entirety of Danbooru these days and it degrades the site's performance too much for real people. I've tried blocking individual bots and IPs but it's a never-ending game of whack-a-mole. So at this point I'm left with wide-ranging bans to try to reduce the number of bots overall.

Some users have had luck fixing it by either of those two solutions: