Closed mkrvaric closed 1 year ago
Why ahrefs is not included by crawler detection?
54.36.149.249 - - [29/Jun/2023:08:08:52 +0000] "GET /hr/proizvod/kratke-hlace-nike-w-pro365-short-7in-hi-rise-34457 HTTP/2.0" 200 16893 "-" "Mozilla/5.0 (compatible; AhrefsSiteAudit/6.1; +http://ahrefs.com/robot/site-audit)"
see https://github.com/JayBizzle/Crawler-Detect/issues/495
Why ahrefs is not included by crawler detection?
54.36.149.249 - - [29/Jun/2023:08:08:52 +0000] "GET /hr/proizvod/kratke-hlace-nike-w-pro365-short-7in-hi-rise-34457 HTTP/2.0" 200 16893 "-" "Mozilla/5.0 (compatible; AhrefsSiteAudit/6.1; +http://ahrefs.com/robot/site-audit)"