Buck and SentiOne bot - Githubissues

mitchellkrogza / nginx-ultimate-bad-bot-blocker

Nginx Block Bad Bots, Spam Referrer Blocker, Vulnerability Scanners, User-Agents, Malware, Adware, Ransomware, Malicious Sites, with anti-DDOS, Wordpress Theme Detector Blocking and Fail2Ban Jail for Repeat Offenders

Other

3.82k stars 472 forks source link

Buck and SentiOne bot #416

Closed uded closed 3 years ago

uded commented 3 years ago

Buck bot: Buck/2.2; (+https://app.hypefactors.com/media-monitoring/about.html) This is yet another media/link/marketing scanner
SentiOne bot: SentiBot www.sentibot.eu (compatible with Googlebot) Social media and marketing scanner

Both scanning quite aggressively

uded commented 3 years ago

As for now I cannot find any information about those obeying robots.txt and also found no way to block them. Although with Buck I have added robots.txt two days ago and still waiting for it to stop crawling...