The User-Agent "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)" is performing multiple GET and HEAD requests across various paths, possibly indicating scanning activity. The requests are not preceded by a robots.txt inquiry, suggesting non-compliance with web crawling standards.
Paste the full User-Agent String here
Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)
Is this for Addition / Removal?
Did the User-Agent request robots.txt first?
Post Log Excerpt to show User-Agent behavior (10-20 lines is enough)
172.208.68.180 - - [02/Jan/2024:20:54:53 -0500] "GET /www2/ HTTP/1.1" 400 272 "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)" "-" 172.208.68.180 - - [02/Jan/2024:20:54:53 -0500] "GET /www3/ HTTP/1.1" 400 272 "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)" "-" 172.208.68.180 - - [02/Jan/2024:20:54:53 -0500] "GET /www4/ HTTP/1.1" 400 272 "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)" "-" 172.208.68.180 - - [02/Jan/2024:20:54:53 -0500] "GET /www/ HTTP/1.1" 400 272 "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)" "-" 172.208.68.180 - - [02/Jan/2024:20:54:53 -0500] "GET /wwwjoin/ HTTP/1.1" 400 272 "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)" "-" 172.208.68.180 - - [02/Jan/2024:20:54:53 -0500] "GET /wwwrooot/ HTTP/1.1" 400 272 "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)" "-" 172.208.68.180 - - [02/Jan/2024:20:54:53 -0500] "GET /www-sql/ HTTP/1.1" 400 272 "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)" "-" 172.208.68.180 - - [02/Jan/2024:20:54:53 -0500] "GET /wwwstat/ HTTP/1.1" 400 272 "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)" "-" 172.208.68.180 - - [02/Jan/2024:20:54:53 -0500] "GET /wwwstats/ HTTP/1.1" 400 272 "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)" "-" 172.208.68.180 - - [02/Jan/2024:20:54:53 -0500] "GET /xGB/ HTTP/1.1" 400 272 "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)" "-" 172.208.68.180 - - [02/Jan/2024:20:54:53 -0500] "GET /xml/ HTTP/1.1" 400 272 "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)" "-" 172.208.68.180 - - [02/Jan/2024:20:54:53 -0500] "GET /XSL/ HTTP/1.1" 400 272 "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)" "-" 172.208.68.180 - - [02/Jan/2024:20:54:53 -0500] "GET /xtemp/ HTTP/1.1" 400 272 "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)" "-" 172.208.68.180 - - [02/Jan/2024:20:54:53 -0500] "GET /xymon/ HTTP/1.1" 400 272 "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)" "-" 172.208.68.180 - - [02/Jan/2024:20:54:53 -0500] "GET /zb41/ HTTP/1.1" 400 272 "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)" "-" 172.208.68.180 - - [02/Jan/2024:20:54:53 -0500] "GET /zipfiles/ HTTP/1.1" 400 272 "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)" "-" 172.208.68.180 - - [02/Jan /2024:20:54:53 -0500] "GET /zip/ HTTP/1.1" 400 272 "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)" "-" 172.208.68.180 - - [02/Jan/2024:20:54:53 -0500] "GET /_docs/ HTTP/1.1" 400 272 "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)" "-" 172.208.68.180 - - [02/Jan/2024:20:54:53 -0500] "GET /sitecore/shell/sitecore.version.xml HTTP/1.1" 400 272 "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)" "-" 172.208.68.180 - - [02/Jan/2024:20:54:53 -0500] "GET /sitecore/login/default.aspx HTTP/1.1" 400 272 "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)" "-" 172.208.68.180 - - [02/Jan/2024:20:54:53 -0500] "HEAD /sitecore/admin/stats.aspx HTTP/1.1" 400 0 "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)" "-" 172.208.68.180 - - [02/Jan/2024:20:54:53 -0500] "HEAD /sitecore/admin/unlock_admin.aspx HTTP/1.1" 400 0 "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)" "-" 172.208.68.180 - - [02/Jan/2024:20:54:53 -0500] "HEAD /sitecore/shell/Applications/shell.xml HTTP/1.1" 400 0 "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)" "-" 172.208.68.180 - - [02/Jan/2024:20:54:53 -0500] "HEAD /sitecore/admin/ShowConfig.aspx HTTP/1.1" 400 0 "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)" "-" 172.208.68.180 - - [02/Jan/2024:20:54:53 -0500] "HEAD /App_Config/Security/Domains.config.xml HTTP/1.1" 400 0 "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)" "-" 172.208.68.180 - - [02/Jan/2024:20:54:53 -0500] "HEAD /App_Config/Security/GlobalRoles.config.xml HTTP/1.1" 400 0 "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)" "-" 172.208.68.180 - - [02/Jan/2024:20:54:53 -0500] "HEAD /sitecore%20modules/staging/service/api.asmx HTTP/1.1" 400 0 "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)" "-" 172.208.68.180 - - [02/Jan/2024:20:54:53 -0500] "HEAD /sitecore%20modules/staging/workdir HTTP/1.1" 400 0 "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)" "-"
Additional information
The User-Agent "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)" is performing multiple GET and HEAD requests across various paths, possibly indicating scanning activity. The requests are not preceded by a robots.txt inquiry, suggesting non-compliance with web crawling standards.