Open extensionsapp opened 7 years ago
extensionsapp
commented
7 years ago 11 Jan 2012 - https://twitter.com/mubix/status/157115321155723264 3 Oct 2012 - https://twitter.com/mubix/status/253705438581903360 For more than 5 years we know about these requests.
extensionsapp
commented
7 years ago And, error in fail2ban:
jail.local
[nginxrepeatoffender]
enabled = true
logpath = %(/var/log/nginx/access.log)s
filter = nginxrepeatoffender
banaction = nginxrepeatoffender
bantime = 86400 ; 1 day
findtime = 604800 ; 1 week
maxretry = 20Sep 16 16:42:07 ziggo fail2ban[4992]: Starting authentication failure monitor: fail2banERROR Failed during configuration: Bad value substitution:
Sep 16 16:42:07 ziggo fail2ban[4992]: section: [nginxrepeatoffender]
Sep 16 16:42:07 ziggo fail2ban[4992]: option : logpath
Sep 16 16:42:07 ziggo fail2ban[4992]: key : /var/log/nginx/access.log
Sep 16 16:42:07 ziggo fail2ban[4992]: rawval :
Sep 16 16:42:07 ziggo fail2ban[4992]: failed!
mitchellkrogza
commented
7 years ago Those attack strings are very difficult to block, I started trying to work on a Regex for detecting that some time ago but it's so complex and those strings change so much I found it almost impossible. that's not to say it is impossible but it requires some very crafty regex patterns which I just don't have time to work out.
extensionsapp
commented
7 years ago @mitchellkrogza Fail2ban should have blocked these requests? But it did not work.
extensionsapp
commented
7 years ago If change logpath = /var/log/nginx/access.log
[nginxrepeatoffender]
enabled = true
logpath = /var/log/nginx/access.log
filter = nginxrepeatoffender
banaction = nginxrepeatoffender
bantime = 86400 ; 1 day
findtime = 604800 ; 1 week
maxretry = 20New error:
Sep 16 17:00:25 ziggo fail2ban[5378]: Starting authentication failure monitor: fail2banERROR Failed during configuration: Bad value substitution:
Sep 16 17:00:25 ziggo fail2ban[5378]: section: [nginxrepeatoffender]
Sep 16 17:00:25 ziggo fail2ban[5378]: option : action
Sep 16 17:00:25 ziggo fail2ban[5378]: key : port
Sep 16 17:00:25 ziggo fail2ban[5378]: rawval : ", protocol="%(protocol)s", chain="%(chain)s"]
Sep 16 17:00:25 ziggo fail2ban[5378]: failed!@extensionsapp I don't know of any Fail2Ban jail currently that will catch any of these. I've posted a question on Stack to see if some regex genius can figure out a regex pattern for these.
extensionsapp
commented
7 years ago @mitchellkrogza I fix problem in fail2ban, add port = http,https
[nginxrepeatoffender]
enabled = true
port = http,https
logpath = /var/log/nginx/access.log
filter = nginxrepeatoffender
banaction = nginxrepeatoffender
bantime = 86400 ; 1 day
findtime = 604800 ; 1 week
maxretry = 20Thanks @extensionsapp I'll try that out. Will let you know if I get any answer on Stack for actually detecting these complex strings like this.
\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00
\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00
\x16\x03\x01\x01\x22\x01\x00\x01\x1E\x03\x03\xB2\xF2\x5CF\x0C\xD8eb\x92m\x19\xBB\x81\xCE\x90\x9C\xC5\x90r+\x98@\xC00\x1AS4\xF3\xB9\x86\xF6\xC2\x00\x00\x88\xC00\xC0,\xC0(\xC0$\xC0\x14\xC0
Gh0st\xAD\x00\x00\x00\xE0\x00\x00\x00x\x9CKS``\x98\xC3\xC0\xC0\xC0\x06\xC4\x8C@\xBCQ\x96\x81\x81\x09H\x07\xA7\x16\x95e&\xA7*\x04$&g+\x182\x94\xF6\xB000\xAC\xA8rc\x00\x01\x11\xA0\x82\x1F\x5C`&\x83\xC7K7\x86\x19\xE5n\x0C9\x95n\x0C;\x84\x0F3\xAC\xE8sch\xA8^\xCF4'J\x97\xA9\x82\xE30\xC3\x91h]&\x90\xF8\xCE\x97S\xCBA4L?2=\xE1\xC4\x92\x86\x0B@\xF5`\x0CT\x1F\xAE\xAF]
\x16\x03\x01\x02\x00\x01\x00\x01\xFC\x03\x03\xCF#vw\x1Ew\x8A(7~R\x9F\xEF\xFFo\x1D\xDC\x97\x8A\xBC\xD4\x82\x1C\x81\x06\xC1\x93@mv\xF5\x15\x00\x00\xD8\x00\x05\x00\x04\x00\x02\x00\x01\x00\x16\x003\x009\x00:\x00\x18\x005\x00
\x16\x03\x01\x02\x00\x01\x00\x01\xFC\x03\x03\xE0^\xF6\x15\x9A\xA1\xD8\x02\x9A\xF2\x0B\x07\x89\xC7o\x83\xBE\xF4e\xC0\xC4\x0B\xA4\xA7_X\xFAItf\xA9\x00\x00\x00\xD8\x00\x05\x00\x04\x00\x02\x00\x01\x00\x16\x003\x009\x00:\x00\x18\x005\x00
\x16\x03\x01\x00\x8B\x01\x00\x00\x87\x03\x03\x22\xFCk\x07L\x07=\x22\xE9\x97\x82\xD9qu\x8C \x06\xE0\x10\x1E\x8A\xC5\xB1\xC7\xF2>6x
%D0%A2%D0%B8%D0%BC%D0%BE%D1%82%D0%B8%20%D0%A0%D0%B5%D0%B4%D1%84%D0%BE%D1%80%D0%B4
%D0%A7%D0%B0%D0%B4%20%D0%A1%D1%82%D0%B0%D1%85%D0%B5%D0%BB%D1%81%D0%BA%D0%B8```%D0%A2%D0%B8%D0%BC%D0%BE%D1%82%D0%B8%20%D0%A0%D0%B5%D0%B4%D1%84%D0%BE%D1%80%D0%B4
%D0%A7%D0%B0%D0%B4%20%D0%A1%D1%82%D0%B0%D1%85%D0%B5%D0%BB%D1%81%D0%BA%D0%B8These are the cyrillic names of the actors.
Тимоти Редфорд
Чад Стахелски@extensionsapp well done, I would never have figured that out 🥇
extensionsapp
commented
7 years ago @mitchellkrogza I block these requests:
jail.local
[nginx-x00]
enabled = true
port = http,https
filter = nginx-x00
logpath = /var/log/nginx/access.log
bantime = 86400
findtime = 86400
maxretry = 2filter.d/nginx-x00.conf
[Definition]
failregex = ^<HOST> .* ".*\\x.*" .*$
ignoreregex =\x03 - not normal in URL %5Cx03 - notmal in URL
Therefore, you can safely block all requests.
mitchellkrogza
commented
7 years ago Nice one @extensionsapp I will try it out. Someone on Stack Exchange posted this Regex for me, do you want to try it out too?? ^(?:(?:\w+)?\\x[^\\\n]+)+|(?:%[A-F0-9]{2})+$
mitchellkrogza
commented
7 years ago Another option with Nginx is
add_header Allow "GET, POST, HEAD" always;
if ( $request_method !~ ^(GET|POST|HEAD)$ )
{ return 405; }.... in all my logs these requests don't use GET,HEAD or POST.
extensionsapp
commented
7 years ago Yes, this is also a good option. It would be nice to check that it's better to block through a fail2ban/iptables or block through nginx.
Nginx sends a 405 error to the bot when it is blocked. This page has about 30Kb. If the bot will DDoS 3333 requests per second - 3333 * 30 KB = 100 MB channel will be fully loaded. Iptables probably does not send anything to the bot.
mitchellkrogza
commented
7 years ago @extensionsapp very true but changing that to
add_header Allow "GET, POST, HEAD" always;
if ( $request_method !~ ^(GET|POST|HEAD)$ )
{ return 444; }would just drop the connection immediately
mitchellkrogza
commented
7 years ago I do agree that catching these with Fail2Ban and blocking them at IPTables level is first prize. Then things like my Fail2Ban Perma-Ban filter can also ban them for extended periods.
RayOei
commented
6 years ago The nginx-x00.conf seems to work for me. Thanks @extensionsapp !
As a note: improving the title to add something like "\x03\x00" for "these requests" might make this issue slightly better visible... I think?
mitchellkrogza
commented
6 years ago Thanks for the feedback @RayOei I have updated the title. Can you share your final solution?
RayOei
commented
6 years ago Nothing new to add: I used what @extensionsapp proposed ;-)
thakur-isha
commented
5 years ago @mitchellkrogza I fix problem in fail2ban, add
port = http,https[nginxrepeatoffender] enabled = true port = http,https logpath = /var/log/nginx/access.log filter = nginxrepeatoffender banaction = nginxrepeatoffender bantime = 86400 ; 1 day findtime = 604800 ; 1 week maxretry = 20
Hi @mitchellkrogza.I am newbie in nginx. Can you please tell me in which file I have to set these about parameters?
Regards Isha
thakur-isha
commented
5 years ago @mitchellkrogza I fix problem in fail2ban, add
port = http,https[nginxrepeatoffender] enabled = true port = http,https logpath = /var/log/nginx/access.log filter = nginxrepeatoffender banaction = nginxrepeatoffender bantime = 86400 ; 1 day findtime = 604800 ; 1 week maxretry = 20
One question: If we will block both http,https then which request will work on our server. Thanks in advance.
mitchellkrogza
commented
4 years ago Both will be blocked
thakur-isha
commented
4 years ago Thanks for you response but if we will restrict both http and https request then if we have some API in our website , will be able to access those APIs?
ansell
commented
2 years ago These requests are generally queries for other protocols such as RDP, so you could also focus on specifically allowing HTTP.
A fail2ban-level approach to reducing noise from non-HTTP request lines that I have used myself is the following, failing everything by default, and then ignoring valid structure HTTP queries:
[Definition]
failregex = ^<HOST> .+ ".*" .+$
ignoreregex = ^<HOST> .+ "[A-Z]+ /.* HTTP/[0-9.]+" .+$
beatquantum
commented
2 years ago Oh, the solution from @ansell works like this. Although the access.log file still contains the following entry when the offending IP visits you for the first time...
20.243.nn.nn - - [23/Jul/2022:06:53:45 +0100] "" 400 0 "-" "-"
...my fail2ban setting would ban that IP from further probes, thus leading to a cleaner and more actionable log.
This will not work for everyone (those who use empty probes); but it works for me. Thanks @ansell
mixpc
commented
1 year ago @thakur-isha or anyone informed, please, any feedback on why [nginx-x00] is not working in my setting?
nginx saves log in non default location but with standard log structure (not custom)
-rw-r--r-- 1 root root 5.3K Apr 19 19:57 /etc/nginx/access.80.log
server {
access_log /etc/nginx/access.80.log;
the log shows
172.105.128.11 - - [19/Apr/2023:16:33:58 +0200] "\x16\x03\x01\x00\x85\x01\x00\x00\x81\x03\x039W\xE7VO\xF0\x9C\x08k\x16\xA7T\x06\xC0\xEE\x83\xDCM\xAE\xFC\xF5\x14\xC6\xD6\xEE\xDEc\x8CPMv\xD2\x00\x00 \xC0/\xC00\xC0+\xC0,\xCC\xA8\xCC\xA9\xC0\x13\xC0\x09\xC0\x14\xC0" 400 150 "-" "-"
fail2ban is active and running ok however it does not seem to block such attempts:
Status for the jail: nginx-x00
|- Filter
| |- Currently failed: 0
| |- Total failed: 0
| `- Journal matches:
`- Actions
|- Currently banned: 0
|- Total banned: 0
`- Banned IP list:
Fail2ban and nginx config:
/etc/fail2ban/jail.local
[DEFAULT]
enabled = true
ignoreip = 127.0.0.1/8
ignorecommand =
backend = systemd
mode = normal
#filter = %(__name__)s[mode=%(mode)s]
findtime = 600
maxretry = 3
bantime = 600
banaction = route
action = %(banaction)s[blocktype=blackhole]
[nginx-x00]
enabled = true
port = http,https
filter = nginx-x00
logpath = /etc/nginx/access.80.fail2ban.log
# /etc/nginx/access.443.log
bantime = 86400
findtime = 86400
maxretry = 1
/etc/fail2ban/filter.d/nginx-x00.conf
[Definition]
failregex = ^<HOST> .* ".*\\x.*" .*$
ignoreregex =
Thank you in advance for your valuable feedback.
beatquantum
commented
1 year ago The logpath in jail and location of access_log in nginx are different in your configuration and hence fail2ban does not pick up.
mixpc
commented
1 year ago [nginx-x00]
backend = autosolved it!
Hello.
How to block these bots?
Server load reached 78%
"Java/1.6.0_24"- This is not a browser, so you need to block it.