search

mitchellkrogza / nginx-ultimate-bad-bot-blocker

Nginx Block Bad Bots, Spam Referrer Blocker, Vulnerability Scanners, User-Agents, Malware, Adware, Ransomware, Malicious Sites, with anti-DDOS, Wordpress Theme Detector Blocking and Fail2Ban Jail for Repeat Offenders
Other
3.81k stars 472 forks source link

bunny.net CDN is being blocked, partly #476

Closed andrewjs18 closed 2 years ago

andrewjs18 commented 2 years ago

hi there,

I'm using the bunny.net CDN. your nginx bad bot blocker was blocking access to my origin for some of the CDN edge servers, but not all of them.

I currently just whitelisted all of bunny.net's IPs in the whitelist-ips.conf, which took forever (like 300 IPs!)...is this the correct approach to it or is there an easier way to do it? should these IPs be whitelisted by default from your script?

here's their IP list: https://api.bunny.net/system/edgeserverlist/plain https://bunnycdn.com/api/system/edgeserverlist/Ipv6

skhaz commented 2 years ago

I am having the same issue.

mitchellkrogza commented 2 years ago

Whitelist all their IP's in whitelist-ips.conf

https://bunnycdn.com/api/system/edgeserverlist https://bunnycdn.com/api/system/edgeserverlist/Ipv6

mitchellkrogza commented 2 years ago

hi there,

I'm using the bunny.net CDN. your nginx bad bot blocker was blocking access to my origin for some of the CDN edge servers, but not all of them.

I currently just whitelisted all of bunny.net's IPs in the whitelist-ips.conf, which took forever (like 300 IPs!)...is this the correct approach to it or is there an easier way to do it? should these IPs be whitelisted by default from your script?

here's their IP list: https://api.bunny.net/system/edgeserverlist/plain https://bunnycdn.com/api/system/edgeserverlist/Ipv6

I will try add them into a default whitelist but you can just add these to your whitelist-ips.conf file

skhaz commented 2 years ago

@mitchellkrogza this list of IPs needs to be get updated frequently on the blocker. Adding it once will break at any moment in the future

mitchellkrogza commented 2 years ago

@mitchellkrogza this list of IPs needs to be get updated frequently on the blocker. Adding it once will break at any moment in the future

that's why it should be added to your own whitelist-ips.conf which never gets touched by my updates

andrewjs18 commented 2 years ago

hi there, I'm using the bunny.net CDN. your nginx bad bot blocker was blocking access to my origin for some of the CDN edge servers, but not all of them. I currently just whitelisted all of bunny.net's IPs in the whitelist-ips.conf, which took forever (like 300 IPs!)...is this the correct approach to it or is there an easier way to do it? should these IPs be whitelisted by default from your script? here's their IP list: https://api.bunny.net/system/edgeserverlist/plain https://bunnycdn.com/api/system/edgeserverlist/Ipv6

I will try add them into a default whitelist but you can just add these to your whitelist-ips.conf file

adding them to your default whitelist would be ideal. they're a large CDN with many POPs now. perhaps a script of some sort to keep your list of IPs synced up with their list of IPs would be most efficient?

is there anything we can ask bunny.net to do to make a more efficient solution? currently the have 345 IPs (ipv4 alone) for their CDN POPs.

mitchellkrogza commented 2 years ago

I will have to look into a default whitelist of these and other important ranges that can be grown over time

andrewjs18 commented 2 years ago

I will have to look into a default whitelist of these and other important ranges that can be grown over time

perfect, thank you.

I wonder if something could be coded where it sends an alert if a new IP is added to their IP list that would then get synced to your default whitelist.......or something along those lines.

mitchellkrogza commented 2 years ago

I will have to look into a default whitelist of these and other important ranges that can be grown over time

perfect, thank you.

I wonder if something could be coded where it sends an alert if a new IP is added to their IP list that would then get synced to your default whitelist.......or something along those lines.

Unfortunately we would need to keep tabs on their IP ranges. Any other important ranges you can think of as I already have Cloudflare

mitchellkrogza commented 2 years ago

@andrewjs18 @skhaz added in https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/commit/486416ff1147829da7df2286508b91503fa3e1e4

skhaz commented 2 years ago

@mitchellkrogza thank you

andrewjs18 commented 2 years ago

@andrewjs18 @skhaz added in 486416f

great, thank you!

I assume I should clear out the IPs I manually entered in the whitelist-ips.conf and just use yours?

mitchellkrogza commented 2 years ago

Just remember the warnings are only telling you something was blacklisted by the daily IP blacklists that get pushed out in daily updates and then some of those IPs are now whitelisted because of the whitelist ranges introduced which are mostly Bunny.net IPS which are erroneously blacklisted. The warnings are nothing more than warnings and do NOT affect Nginx performance or reloading time in any way whatsoever

AnTheMaker commented 1 year ago

Hi there! Might be a bit late, but I had the same issue, so I made a repo which contains the automatically updated list of IP addresses e.g. used by BunnyCDN: https://github.com/AnTheMaker/GoodBots/blob/main/iplists/bunnycdn.ips You could use that to automatically update your IP whitelist! Hope this helps!

GitHub
GoodBots/bunnycdn.ips at main · AnTheMaker/GoodBots
Updated lists of IP addresses/whitelists of good bots and crawlers. Includes GoogleBot, BingBot, DuckDuckBot, etc. - GoodBots/bunnycdn.ips at main · AnTheMaker/GoodBots
skhaz commented 1 year ago

Very nice @AnTheMaker!