mitchellkrogza
commented
8 months ago Thanks @robwent have updated this list - all the old ranges have been dropped entirely.
Closed robwent closed 8 months ago
mitchellkrogza
commented
8 months ago Thanks @robwent have updated this list - all the old ranges have been dropped entirely.
robwent
commented
8 months ago Quick question on this. I tried to whitelist the Bing ranges after the abuseipdb include like this: https://gist.github.com/robwent/d5b61fbe269216ff5b7e0dba93c9ff1b
But I still see blocked Bing IPs:
40.77.167.41 - - [24/Mar/2024:22:01:11 +0000] "GET /private-yacht-charter/page/74/ HTTP/2.0" 444 0 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) Chrome/116.0.1938.76 Safari/537.36"
Does a single IP marked with 1 trump an IP range marked as 0 after the initial 1 setting?
Gist
Integrating AbuseIPDB with Nginx bad bot blocker https://github.com/robwent/abuseipdb-bad-bot-blocker. Whitelisting Search Engines again. - blacklist-ips.conf
mitchellkrogza
commented
8 months ago 1 means current setting "blocked" 0 means "whitelisted" where have you placed the include for all those extra abuse-ipdb ranges you are adding because the order of where that is inserted is crucial to NOT break the whitelisting which follows AFTER All blacklists.
robwent
commented
8 months ago It's in blacklist-ips.conf. This is the full file: https://gist.github.com/robwent/30186093df14bd490ffc5f31f91d7f36
GitHub Gist: instantly share code, notes, and snippets.
I checked again this morning and it's a mixture of blocked and allowed. These are all verified:
Blocked 52.167.144.20
Ok 52.167.144.216 40.77.167.54 40.77.167.132
mitchellkrogza
commented
8 months ago It's in blacklist-ips.conf. This is the full file: https://gist.github.com/robwent/30186093df14bd490ffc5f31f91d7f36
Gistgist:30186093df14bd490ffc5f31f91d7f36GitHub Gist: instantly share code, notes, and snippets.
I checked again this morning and it's a mixture of blocked and allowed. These are all verified:
Blocked 52.167.144.20
Ok 52.167.144.216 40.77.167.54 40.77.167.132
You are creating duplication
https://gist.github.com/robwent/30186093df14bd490ffc5f31f91d7f36#file-gistfile1-txt-L213-L478
are already included in the globalblacklist.conf
GitHub Gist: instantly share code, notes, and snippets.
GitHub Gist: instantly share code, notes, and snippets.
robwent
commented
8 months ago Ok, it looks like the problem is that 52.167.144.20 is included in the global blacklist that gets added after the custom blacklist IP file. So should these two legit IPs be removed? https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/conf.d/globalblacklist.conf#L17314-L17315
I'm going to remove the duplicate whitelists from the blacklist-ips.conf file and whitelist those 2 in the whitelist-ips.conf instead.
Nginx Block Bad Bots, Spam Referrer Blocker, Vulnerability Scanners, User-Agents, Malware, Adware, Ransomware, Malicious Sites, with anti-DDOS, Wordpress Theme Detector Blocking and Fail2Ban Jail f...
robwent
commented
8 months ago I'm getting a bit lost here, but that IP is whitelisted again after being marked as 1 in the global blacklist. https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/conf.d/globalblacklist.conf#L18844
But it's still getting a 444 response.
Nginx Block Bad Bots, Spam Referrer Blocker, Vulnerability Scanners, User-Agents, Malware, Adware, Ransomware, Malicious Sites, with anti-DDOS, Wordpress Theme Detector Blocking and Fail2Ban Jail f...
When I was testing #556 I noticed a BingBot useragent getting blocked on the IP 40.77.167.254.
Bing confirms this is one of theirs:
I may be looking in the wrong place, but in this list there are 19 ranges: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/_generator_lists/bing-ip-ranges.list
I found an official list here which has 28: https://www.[bing.com/toolbox/bingbot.json](https://www.bing.com/toolbox/bingbot.json)
207.46.13.0/24 199.30.24.0/23 191.233.204.224/28 157.55.39.0/24 139.217.52.0/28 65.55.210.0/24 52.231.148.0/28 52.167.144.0/24 51.105.67.0/28 40.79.186.176/28 40.79.131.208/28 40.77.202.0/24 40.77.188.0/22 40.77.178.0/23 40.77.177.0/24 40.77.167.0/24 40.77.139.0/25 20.125.163.80/28 20.79.107.240/28 20.74.197.0/28 20.43.120.16/28 20.36.108.32/28 20.15.133.160/27 13.71.172.224/28 13.69.66.240/28 13.67.10.16/28 13.66.144.0/24 13.66.139.0/24
I think my blacklist would have overridden this anyway but the list looks out of date.