search

mitchellkrogza / apache-ultimate-bad-bot-blocker

Apache Block Bad Bots, (Referer) Spam Referrer Blocker, Vulnerability Scanners, Malware, Adware, Ransomware, Malicious Sites, Wordpress Theme Detectors and Fail2Ban Jail for Repeat Offenders
Other
807 stars 175 forks source link

[ADD/REMOVE] #146

Open ZerooCool opened 4 years ago

ZerooCool commented 4 years ago

In this file : https://github.com/mitchellkrogza/apache-ultimate-bad-bot-blocker/blob/master/Apache_2.4/custom.d/globalblacklist.conf

Can this line : BrowserMatchNoCase "(?:\b)AhrefsBot(?:\b)" bad_bot

Block this : AhrefsBot/6.1

54.36.148.99 - - [03/May/2020:17:52:21 +0200] "GET the-url HTTP/1.1" 200 13162 "-" "Mozilla/5.0 (compatible; AhrefsBot/6.1; +http://ahrefs.com/robot/)"

mitchellkrogza commented 4 years ago

It should block all variants

ZerooCool commented 4 years ago

It's strange, I have the impression that not: / I was able to block only by adding my variant.

I also blocked AspiegelBot and in this case, I have the impression that the robot continues to be able to read pages (HTTP code 200) but nevertheless, if I use cURL to test to consult a page as AspiegelBot, I'm stuck.

It's really strange. I don't have to understand certain things well. is beyond me.

If you want test, navigate with AhrefsBot or AspiegelBot to my website : https://www.visionduweb.fr

Vision du web - La Réponse Collaborative
Accueil
Objectifs de la plateforme Visionduweb L'objectif de Visionduweb est la mise en valeur d'outils informatiques gratuits, libres, opensource. Consulter le co...