[User-Agent] (AI Search Crawlers) - Githubissues

mitchellkrogza / nginx-ultimate-bad-bot-blocker

Nginx Block Bad Bots, Spam Referrer Blocker, Vulnerability Scanners, User-Agents, Malware, Adware, Ransomware, Malicious Sites, with anti-DDOS, Wordpress Theme Detector Blocking and Fail2Ban Jail for Repeat Offenders

Other

3.96k stars 477 forks source link

[User-Agent] (AI Search Crawlers) #549

Open jlarmstrongiv opened 8 months ago

jlarmstrongiv commented 8 months ago

Paste the full User-Agent String here

Amazonbot
Applebot
PerplexityBot
YouBot
Amazonbot

Is this for Addition / Removal?

[ ] Addition
[ ] Removal
[x] Keep a watch on this one

Did the User-Agent request robots.txt first?

[ ] Yes
[ ] No

Post Log Excerpt to show User-Agent behavior (10-20 lines is enough)


Post log excerpt here (paste in between the ```     ``` markers)

Additional information

They should be watched, but they cite websites so may be okay for now

See also https://darkvisitors.com/agents

mitchellkrogza commented 6 months ago

Will scan some logs for these