Closed aledbf closed 4 years ago
Hi @aledbf,
Can you give us further detail on the configuration? Whats is your server/http/location configuration?
@zimmerle apologies for the delay:
nginx.conf
load_module /etc/nginx/modules/ngx_http_modsecurity_module.so;
daemon off;
worker_processes 8;
worker_rlimit_nofile 130048;
worker_shutdown_timeout 240s ;
events {
multi_accept on;
worker_connections 16384;
use epoll;
}
.....
server {
server_name foo.bar ;
listen 80 ;
location / {
modsecurity on;
# https://gist.github.com/aledbf/6773613529292bdbbe6454741c21b0c9
modsecurity_rules_file /etc/nginx/modsecurity/modsecurity.conf;
# https://gist.github.com/aledbf/913f9997e3386f92ba2550eb62e21a48
modsecurity_rules_file /etc/nginx/owasp-modsecurity-crs/nginx-modsecurity.conf;
modsecurity_rules '
SecRuleEngine On
SecRequestBodyAccess On
SecAuditEngine RelevantOnly
SecAuditLogParts ABIJDEFHZ
SecAuditLog /dev/stdout
SecRule REQUEST_HEADERS:User-Agent \"block-ua\" \"log,deny,id:107,status:403,msg:\'UA blocked\'\"
';
proxy_pass http://upstream_balancer;
proxy_redirect off;
}
}
## end server foo.bar
Test configuration:
$ curl http://localhost -H 'Host: foo.bar' -H "User-Agent: block-ua"
Hostname: http-svc-5d699dc6cd-ghg9p
Pod Information:
node name: kind-control-plane
pod name: http-svc-5d699dc6cd-ghg9p
pod namespace: default
pod IP: 10.244.0.6
Server values:
server_version=nginx: 1.12.2 - lua: 10010
Request Information:
client_address=10.244.0.8
method=GET
real path=/
query=
request_version=1.1
request_scheme=http
request_uri=http://foo.bar:8080/
Request Headers:
accept=*/*
host=foo.bar
user-agent=block-ua
x-forwarded-for=172.17.0.1
x-forwarded-host=foo.bar
x-forwarded-port=80
x-forwarded-proto=http
x-real-ip=172.17.0.1
x-request-id=da01d515022fd1878e7fdced6efe67d2
x-scheme=http
Request Body:
-no body in request-
The test should return 403.
NGINX log:
172.17.0.1 - - [30/Mar/2020:23:53:16 +0000] "GET / HTTP/1.1" 200 683 "-" "block-ua" 68 0.019 [default-http-svc-8080] [] 10.244.0.6:8080 683 0.020 200 da01d515022fd1878e7fdced6efe67d2
foo.bar 172.17.0.1 - [30/Mar/2020:23:53:16 +0000] "GET / HTTP/1.1" 200 671 - "block-ua" 158561239615.836189 - /var/log/audit//20200330/20200330-2353/20200330-235316-158561239615.836189 0 1679.000000 md5:89c9f10e9935e3c90c372d07006b6a55
cat /var/log/audit/20200330/20200330-2350/20200330-235040-158561224041.121580
---Yx7ouq9l---A--
[30/Mar/2020:23:50:40 +0000] 158561224041.121580 172.17.0.1 46518 10.244.0.8 80
---Yx7ouq9l---B--
GET / HTTP/1.1
Host: foo.bar
Accept: */*
User-Agent: block-ua
---Yx7ouq9l---D--
---Yx7ouq9l---E--
\x0a\x0aHostname: http-svc-5d699dc6cd-ghg9p\x0a\x0aPod Information:\x0a\x09node name:\x09kind-control-plane\x0a\x09pod name:\x09http-svc-5d699dc6cd-ghg9p\x0a\x09pod namespace:\x09default\x0a\x09pod IP:\x0910.244.0.6\x0a\x0aServer values:\x0a\x09server_version=nginx: 1.12.2 - lua: 10010\x0a\x0aRequest Information:\x0a\x09client_address=10.244.0.8\x0a\x09method=GET\x0a\x09real path=/\x0a\x09query=\x0a\x09request_version=1.1\x0a\x09request_scheme=http\x0a\x09request_uri=http://foo.bar:8080/\x0a\x0aRequest Headers:\x0a\x09accept=*/*\x0a\x09host=foo.bar\x0a\x09user-agent=block-ua\x0a\x09x-forwarded-for=172.17.0.1\x0a\x09x-forwarded-host=foo.bar\x0a\x09x-forwarded-port=80\x0a\x09x-forwarded-proto=http\x0a\x09x-real-ip=172.17.0.1\x0a\x09x-request-id=39b526deb9248ef6af746a40fa3f0e18\x0a\x09x-scheme=http\x0a\x0aRequest Body:\x0a\x09-no body in request-\x0a\x0a
---Yx7ouq9l---F--
HTTP/1.1 200
Server: nginx/1.17.9
Date: Mon, 30 Mar 2020 23:50:40 GMT
Content-Type: text/plain
Connection: keep-alive
---Yx7ouq9l---H--
ModSecurity: Warning. Matched "Operator `Rx' with parameter `block-ua' against variable `REQUEST_HEADERS:User-Agent' (Value: `block-ua' ) [file "<<reference missing or not informed>>"] [line "7"] [id "107"] [rev ""] [msg "UA blocked"] [data ""] [severity "0"] [ver ""] [maturity "0"] [accuracy "0"] [hostname "10.244.0.8"] [uri "/"] [unique_id "158561224041.121580"] [ref "o0,8v53,8"]
---Yx7ouq9l---I--
---Yx7ouq9l---J--
---Yx7ouq9l---Z--
NGINX version:
nginx -V
nginx version: nginx/1.17.9
built by gcc 9.2.0 (Alpine 9.2.0)
built with OpenSSL 1.1.1d 10 Sep 2019
TLS SNI support enabled
configure arguments: --prefix=/usr/local/nginx --conf-path=/etc/nginx/nginx.conf --modules-path=/etc/nginx/modules --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-debug --with-compat --with-pcre-jit --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_addition_module --with-http_dav_module --with-http_geoip_module --with-http_gzip_static_module --with-http_sub_module --with-http_v2_module --with-stream --with-stream_ssl_module --with-stream_ssl_preread_module --with-threads --with-http_secure_link_module --with-http_gunzip_module --with-file-aio --without-mail_pop3_module --without-mail_smtp_module --without-mail_imap_module --without-http_uwsgi_module --without-http_scgi_module --with-cc-opt='-g -Og -fPIE -fstack-protector-strong -Wformat -Werror=format-security -Wno-deprecated-declarations -fno-strict-aliasing -D_FORTIFY_SOURCE=2 --param=ssp-buffer-size=4 -DTCP_FASTOPEN=23 -fPIC -Wno-cast-function-type -I/root/.hunter/_Base/2c5c6fc/d64af22/92161a9/Install/include -m64 -mtune=native' --with-ld-opt='-fPIE -fPIC -pie -Wl,-z,relro -Wl,-z,now -L/root/.hunter/_Base/2c5c6fc/d64af22/92161a9/Install/lib' --user=www-data --group=www-data --add-module=/tmp/build/ngx_devel_kit-0.3.1rc1 --add-module=/tmp/build/set-misc-nginx-module-0.32 --add-module=/tmp/build/headers-more-nginx-module-0.33 --add-module=/tmp/build/nginx-http-auth-digest-cd8641886c873cf543255aeda20d23e4cd603d05 --add-module=/tmp/build/ngx_http_substitutions_filter_module-bc58cb11844bc42735bbaef7085ea86ace46d05b --add-module=/tmp/build/lua-nginx-module-0.10.15 --add-module=/tmp/build/stream-lua-nginx-module-0.0.7 --add-module=/tmp/build/lua-upstream-nginx-module-0.07 --add-module=/tmp/build/nginx-influxdb-module-5b09391cb7b9a889687c0aa67964c06a2d933e8b --add-dynamic-module=/tmp/build/nginx-opentracing-0.9.0/opentracing --add-dynamic-module=/tmp/build/ModSecurity-nginx-1.0.1 --add-dynamic-module=/tmp/build/ngx_http_geoip2_module-3.3 --add-module=/tmp/build/nginx_ajp_module-bf6cd93f2098b59260de8d494f0f4b1f11a84627 --add-module=/tmp/build/ngx_brotli
From the behavior I see:
Not sure if this works as expected, i.e., if I include modsecurity_rules_file
directive/s they cannot be override modsecurity_rules
in the same block.
@zimmerle here is a test that reproduces the issue
#!/usr/bin/perl
use warnings;
use strict;
use Test::More;
use Socket qw/ CRLF /;
BEGIN { use FindBin; chdir($FindBin::Bin); }
use lib 'lib';
use Test::Nginx;
###############################################################################
select STDERR; $| = 1;
select STDOUT; $| = 1;
my $t = Test::Nginx->new()->has(qw/http/);
$t->write_file_expand('nginx.conf', <<'EOF');
%%TEST_GLOBALS%%
daemon off;
events {
}
http {
%%TEST_GLOBALS_HTTP%%
log_format testmodsec '$status "$http_user_agent"';
access_log %%TESTDIR%%/access_test.log testmodsec;
server {
listen 127.0.0.1:8080;
server_name localhost;
# if I uncomment any modsecurity_rules_file it does not work as expected
#modsecurity_rules_file %%TESTDIR%%/modsecurity.conf;
location /blocked {
modsecurity on;
# if I uncomment any modsecurity_rules_file it does not work as expected
#modsecurity_rules_file %%TESTDIR%%/modsecurity.conf;
modsecurity_rules '
SecRuleEngine On
SecRequestBodyAccess On
SecAuditEngine RelevantOnly
SecAuditLogParts ABIJDEFHZ
SecAuditLog %%TESTDIR%%/modsec.log
SecRule REQUEST_HEADERS:User-Agent \"block-ua\" \"log,deny,id:107,status:403,msg:\'UA blocked\'\"
';
return 200;
}
location /not-blocked {
return 200;
}
}
}
EOF
$t->write_file('modsecurity.conf', <<EOF);
# similar to https://github.com/SpiderLabs/ModSecurity/blob/v3/master/modsecurity.conf-recommended
SecRuleEngine DetectionOnly
SecRequestBodyAccess On
SecRule REQUEST_HEADERS:Content-Type "(?:application(?:/soap\+|/)|text/)xml" \
"id:'200000',phase:1,t:none,t:lowercase,pass,nolog,ctl:requestBodyProcessor=XML"
SecRule REQUEST_HEADERS:Content-Type "application/json" \
"id:'200001',phase:1,t:none,t:lowercase,pass,nolog,ctl:requestBodyProcessor=JSON"
SecRequestBodyLimit 13107200
SecRequestBodyNoFilesLimit 131072
SecRequestBodyLimitAction Reject
SecPcreMatchLimit 1000
SecPcreMatchLimitRecursion 1000
SecResponseBodyAccess On
SecResponseBodyMimeType text/plain text/html text/xml
SecResponseBodyLimit 524288
SecResponseBodyLimitAction ProcessPartial
SecAuditEngine RelevantOnly
SecAuditLogRelevantStatus "^(?:5|4(?!04))"
SecAuditLogParts ABIJDEFHZ
SecAuditLogType Concurrent
SecArgumentSeparator &
SecCookieFormat 0
SecStatusEngine On
EOF
$t->write_file("/ua", "should be moved/blocked before this.");
$t->run();
$t->todo_alerts();
$t->plan(4);
###############################################################################
like(http('GET /blocked HTTP/1.0' . CRLF
. 'Host: localhost' . CRLF
. 'User-Agent: block-ua' . CRLF . CRLF), qr/403 Forbidden/, 'block user agent');
like($t->read_file('access_test.log'), qr/403 "block-ua"/, 'nginx logs');
like($t->read_file('modsec.log'), qr/403 Forbidden/, 'modsec logs');
like(http('GET /not-blocked HTTP/1.0' . CRLF
. 'Host: localhost' . CRLF . CRLF), qr/HTTP\/1.1 200 OK/, 'return 200 on /not-blocked');
If I uncomment any modsecurity_rules_file
it does not work as expected.
@zimmerle friendly ping
@zimmerle friendly ping
In my queue. Sorry for the delay.
I've run into the same issue.
@zimmerle friendly ping :)
@zimmerle friendly ping :)
@zimmerle friendly ping :)
There are some changes in the rules load approach for 3.1. As of now, we are dealing with an issue that is likely to be a consequence of rules merging (see SpiderLabs/ModSecurity#2374 and SpiderLabs/ModSecurity#2376 for further info). At the commit e0dc84cba5074509275820d79ce3d333658f20c7 we have changed the behavior of SecDefaultAction to overwrite a configuration inside a child RuleSet (subdomain or folder). Those upcoming changes will hit the release status eventually before that, it will hit the v3/master.
Respecting the configuration hierarchy the changes in a child configuration should just reflect in the child itself, it is not propagated to the father. The child of a child (or grandchild) should contemplate the configuration of the grandfather and father.
What could be happing in your use case scenario is the fact that modsecurity_rules_file and modsecurity_rules are two different Nginx configurations, as such, one has precedence to another; The configurations are not being applied in the order that you read but in the precedence order. Having said that: By using only one option: modsecurity_rules_file or modsecurity_rules do you happen to notice a difference?
@zimmerle thank you for the feedback
By using only one option: modsecurity_rules_file or modsecurity_rules do you happen to notice a difference?
I am going to test this :)
What could be happing in your use case scenario is the fact that modsecurity_rules_file and modsecurity_rules are two different Nginx configurations,
This means we cannot merge this configurations? (modsecurity_rules_file in the server block and modsecurity_rules in a location)
Is possible to define a "global" behavior and change something in a location? (not sure that is possible)
It is possible. What I am suggesting - for testing - is to stick to one option: modsecurity_rules_file or modsecurity_rules. You can have multiple entries of modsecurity_rules_file, in that case, the appearance order in the file will be respected. I am suspecting that mixing modsecurity_rules_file and modsecurity_rules may have caused you the issue.
Instead of:
modsecurity_rules '
SecRuleEngine On
SecRequestBodyAccess On
SecAuditEngine RelevantOnly
SecAuditLogParts ABIJDEFHZ
SecAuditLog %%TESTDIR%%/modsec.log
SecRule REQUEST_HEADERS:User-Agent \"block-ua\" \"log,deny,id:107,status:403,msg:\'UA blocked\'\"
';
Use modsecurity_rules_file pointing to a file with that same content and let me know if that works.
Closing. Avoiding the mix of modsecurity_rules
and modsecurity_rules_file
avoids the issue.
Edit: This is not fixed in ingress-nginx (yet).
This PR should fix the issue https://github.com/kubernetes/ingress-nginx/pull/8021
I am using this configuration:
http section:
a particular location:
The issue here is related to
modsecurity_rules
configuration. There is no enforced mode in the location Is this the right way to configure the module?How is it possible to debug this misconfiguration? Thanks