search

owasp-modsecurity / ModSecurity-nginx

ModSecurity v3 Nginx Connector
Apache License 2.0
1.49k stars 277 forks source link

File upload finishes with MULTIPART_UNMATCHED_BOUNDARY error #244

Closed airween closed 3 years ago

airween commented 3 years ago

I always run into a MULTIPART_UNMATCHED_BOUNDARY error when I upload any kind of file.

The test environment is @defanator's Vagrant image.

Components:

test@vagrant:~/ModSecurity$ git describe 
v3.0.4-118-g4127c1bf

test@vagrant:~/ModSecurity-nginx$ git describe 
v1.0.1-23-g21bc821

/usr/sbin/nginx -V
nginx version: nginx/1.19.6
built by gcc 8.3.0 (Debian 8.3.0-6) 
built with OpenSSL 1.1.1d  10 Sep 2019
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules \
 --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log \
 --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock \
 --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp \
 --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp \
 --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio \
 --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module \
 --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module \
 --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module \
 --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail \
 --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module \
 --with-stream_ssl_preread_module --add-dynamic-module=../ModSecurity-nginx

This is what I modified:

diff --git a/states/files/etc/nginx/fastcgi_params b/states/files/etc/nginx/fastcgi_params
new file mode 100644
index 0000000..28decb9
--- /dev/null
+++ b/states/files/etc/nginx/fastcgi_params
@@ -0,0 +1,25 @@
+
+fastcgi_param  QUERY_STRING       $query_string;
+fastcgi_param  REQUEST_METHOD     $request_method;
+fastcgi_param  CONTENT_TYPE       $content_type;
+fastcgi_param  CONTENT_LENGTH     $content_length;
+
+fastcgi_param  SCRIPT_NAME        $fastcgi_script_name;
+fastcgi_param  REQUEST_URI        $request_uri;
+fastcgi_param  DOCUMENT_URI       $document_uri;
+fastcgi_param  DOCUMENT_ROOT      $document_root;
+fastcgi_param  SERVER_PROTOCOL    $server_protocol;
+fastcgi_param  REQUEST_SCHEME     $scheme;
+fastcgi_param  HTTPS              $https if_not_empty;
+
+fastcgi_param  GATEWAY_INTERFACE  CGI/1.1;
+fastcgi_param  SERVER_SOFTWARE    nginx/$nginx_version;
+
+fastcgi_param  REMOTE_ADDR        $remote_addr;
+fastcgi_param  REMOTE_PORT        $remote_port;
+fastcgi_param  SERVER_ADDR        $server_addr;
+fastcgi_param  SERVER_PORT        $server_port;
+fastcgi_param  SERVER_NAME        $server_name;
+
+# PHP only, required if PHP was built with --enable-force-cgi-redirect
+fastcgi_param  REDIRECT_STATUS    200;
diff --git a/states/files/etc/nginx/modsec/main.conf b/states/files/etc/nginx/modsec/main.conf
index 16af6e5..fc181a0 100644
--- a/states/files/etc/nginx/modsec/main.conf
+++ b/states/files/etc/nginx/modsec/main.conf
@@ -1,5 +1,5 @@
 include /etc/nginx/modsec/modsecurity.conf

 # OWASP CRS rules
-include /etc/nginx/modsec/owasp-crs/crs-setup.conf
-include /etc/nginx/modsec/owasp-crs/rules/*.conf
+#include /etc/nginx/modsec/owasp-crs/crs-setup.conf
+#include /etc/nginx/modsec/owasp-crs/rules/*.conf
diff --git a/states/files/etc/nginx/modsec/modsecurity.conf b/states/files/etc/nginx/modsec/modsecurity.conf
new file mode 100644
index 0000000..284e543
--- /dev/null
+++ b/states/files/etc/nginx/modsec/modsecurity.conf
@@ -0,0 +1,49 @@
+SecRuleEngine On
+SecRequestBodyAccess On
+SecRule REQUEST_HEADERS:Content-Type "(?:application(?:/soap\+|/)|text/)xml" \
+     "id:'200000',phase:1,t:none,t:lowercase,pass,nolog,ctl:requestBodyProcessor=XML"
+SecRule REQUEST_HEADERS:Content-Type "application/json" \
+     "id:'200001',phase:1,t:none,t:lowercase,pass,nolog,ctl:requestBodyProcessor=JSON,ctl:ruleRemoveById=930120"
+SecRequestBodyLimit 13107200
+SecRequestBodyNoFilesLimit 131072
+SecRequestBodyLimitAction Reject
+SecRule REQBODY_ERROR "!@eq 0" \
+"id:'200002', phase:2,t:none,log,deny,status:400,msg:'Failed to parse request body.',logdata:'%{reqbody_error_msg}',severity:2"
+SecRule MULTIPART_STRICT_ERROR "!@eq 0" \
+"id:'200003',phase:2,t:none,log,deny,status:400, \
+msg:'Multipart request body failed strict validation: \
+PE %{REQBODY_PROCESSOR_ERROR}, \
+BQ %{MULTIPART_BOUNDARY_QUOTED}, \
+BW %{MULTIPART_BOUNDARY_WHITESPACE}, \
+DB %{MULTIPART_DATA_BEFORE}, \
+DA %{MULTIPART_DATA_AFTER}, \
+HF %{MULTIPART_HEADER_FOLDING}, \
+LF %{MULTIPART_LF_LINE}, \
+SM %{MULTIPART_MISSING_SEMICOLON}, \
+IQ %{MULTIPART_INVALID_QUOTING}, \
+IP %{MULTIPART_INVALID_PART}, \
+IH %{MULTIPART_INVALID_HEADER_FOLDING}, \
+FL %{MULTIPART_FILE_LIMIT_EXCEEDED}'"
+SecRule MULTIPART_UNMATCHED_BOUNDARY "@gt 1" \
+    "id:'200004',phase:2,t:none,log,deny,msg:'Multipart parser detected a possible unmatched boundary.'"
+SecPcreMatchLimit 1000
+SecPcreMatchLimitRecursion 1000
+SecRule TX:/^MSC_/ "!@streq 0" \
+        "id:'200005',phase:2,t:none,deny,msg:'ModSecurity internal error flagged: %{MATCHED_VAR_NAME}'"
+SecResponseBodyAccess On
+SecResponseBodyMimeType text/plain text/html text/xml
+SecResponseBodyLimit 524288
+SecResponseBodyLimitAction ProcessPartial
+SecTmpDir /tmp/
+SecDataDir /tmp/
+SecDebugLog /var/log/nginx/modsec_debug.log
+SecDebugLogLevel 9
+SecAuditEngine On
+SecAuditLogRelevantStatus "^(?:5|4(?!04))"
+SecAuditLogParts ABCDEFHIJZ
+SecAuditLogType Serial
+SecAuditLog /var/log/modsec_audit.log
+SecArgumentSeparator &
+SecCookieFormat 0
+SecUnicodeMapFile unicode.mapping 20127
+SecStatusEngine On
diff --git a/states/files/etc/nginx/nginx.conf b/states/files/etc/nginx/nginx.conf
index 15cd224..0cb2772 100644
--- a/states/files/etc/nginx/nginx.conf
+++ b/states/files/etc/nginx/nginx.conf
@@ -33,6 +33,10 @@ http {
         server 127.0.0.1:3131;
     }

+    upstream php-fpm {
+        server unix:/run/php/php7.3-fpm.sock;
+    }
+
     server {
         listen 8080 default_server backlog=32000;
         location / {
@@ -67,6 +71,8 @@ http {
             include uwsgi_params;
             uwsgi_pass upload-app;
             client_max_body_size 256m;
+            modsecurity on;
+            modsecurity_rules_file /etc/nginx/modsec/main.conf;
         }

         location /modsec-light/ {
@@ -94,6 +100,15 @@ http {
             include uwsgi_params;
             uwsgi_pass upload-app;
         }
+        location ~ \.php$ {
+            modsecurity on;
+            modsecurity_rules_file /etc/nginx/modsec/main.conf;
+            fastcgi_pass  php-fpm;
+            fastcgi_index index.php;
+            fastcgi_param SCRIPT_FILENAME /var/www/html/$fastcgi_script_name;
+            fastcgi_intercept_errors on;
+            include fastcgi_params;
+        }
     }

     include /etc/nginx/conf.d/*.conf;

Short summary of modifications:

Here is the PHP file what I used:

# cat /var/www/html/index.php 
<?php

var_dump($_POST);

The file what I try to upload:

$ cat text.txt 
test text

curl commands:

$ curl -vvv "http://localhost/upload/" -F "data=@text.txt"
* Expire in 0 ms for 6 (transfer 0x56389b68df50)
...
* Expire in 1 ms for 1 (transfer 0x56389b68df50)
*   Trying ::1...
* TCP_NODELAY set
* Expire in 149998 ms for 3 (transfer 0x56389b68df50)
* Expire in 200 ms for 4 (transfer 0x56389b68df50)
* connect to ::1 port 80 failed: Connection refused
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Expire in 149998 ms for 3 (transfer 0x56389b68df50)
* Connected to localhost (127.0.0.1) port 80 (#0)
> POST /upload/ HTTP/1.1
> Host: localhost
> User-Agent: curl/7.64.0
> Accept: */*
> Content-Length: 196
> Content-Type: multipart/form-data; boundary=------------------------0e87336e7e623e42
> 
< HTTP/1.1 403 Forbidden
< Server: nginx/1.19.6
< Date: Thu, 01 Apr 2021 19:06:25 GMT
< Content-Type: text/html
< Content-Length: 153
< Connection: keep-alive
* HTTP error before end of send, stop sending
< 
<html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx/1.19.6</center>
</body>
</html>
* Closing connection 0

in the error.log:

2021/04/01 19:06:25 [error] 3329#3329: *2 [client 127.0.0.1] ModSecurity: Access denied with code 403 (phase 2). Matched "Operator `Gt' with parameter `1' against variable `MULTIPART_UNMATCHED_BOUNDARY' (Value: `2' ) [file "/etc/nginx/modsec/modsecurity.conf"] [line "15"] [id "200004"] [rev ""] [msg "Multipart parser detected a possible unmatched boundary."] [data ""] [severity "0"] [ver ""] [maturity "0"] [accuracy "0"] [hostname "127.0.0.1"] [uri "/upload/"] [unique_id "1617303985"] [ref "v180,1"], client: 127.0.0.1, server: , request: "POST /upload/ HTTP/1.1", host: "localhost"

Note: this request has sent to /upload/ which was already configured.

Another curl command:

$ curl -vvv "http://localhost/index.php" -F "data=@text.txt"
* Expire in 0 ms for 6 (transfer 0x558bc0435f50)
...
* Expire in 0 ms for 1 (transfer 0x558bc0435f50)
*   Trying ::1...
* TCP_NODELAY set
* Expire in 149999 ms for 3 (transfer 0x558bc0435f50)
* Expire in 200 ms for 4 (transfer 0x558bc0435f50)
* connect to ::1 port 80 failed: Connection refused
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Expire in 149998 ms for 3 (transfer 0x558bc0435f50)
* Connected to localhost (127.0.0.1) port 80 (#0)
> POST /index.php HTTP/1.1
> Host: localhost
> User-Agent: curl/7.64.0
> Accept: */*
> Content-Length: 196
> Content-Type: multipart/form-data; boundary=------------------------9dd881243a0bfb97
> 
< HTTP/1.1 403 Forbidden
< Server: nginx/1.19.6
< Date: Thu, 01 Apr 2021 19:08:37 GMT
< Content-Type: text/html
< Content-Length: 153
< Connection: keep-alive
* HTTP error before end of send, stop sending
< 
<html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx/1.19.6</center>
</body>
</html>
* Closing connection 0

in the error log I got:

2021/04/01 19:08:37 [error] 3329#3329: *3 [client 127.0.0.1] ModSecurity: Access denied with code 403 (phase 2). Matched "Operator `Gt' with parameter `1' against variable `MULTIPART_UNMATCHED_BOUNDARY' (Value: `2' ) [file "/etc/nginx/modsec/modsecurity.conf"] [line "15"] [id "200004"] [rev ""] [msg "Multipart parser detected a possible unmatched boundary."] [data ""] [severity "0"] [ver ""] [maturity "0"] [accuracy "0"] [hostname "127.0.0.1"] [uri "/index.php"] [unique_id "1617304117"] [ref "v182,1"], client: 127.0.0.1, server: , request: "POST /index.php HTTP/1.1", host: "localhost"

When I run tcpdump, I see that there IS the final boundary in the request. I also tried it from browser, final boundary also at there.

Am I spoiling something? Or is this a really unexpected behavior?

martinhsv commented 3 years ago

Hi @airween ,

I have seen the 'UNMATCHED_BOUNDARY' in unexpected cases (i.e. where the final boundary is indeed present). Typically this is because the request body stopped being processed before the final boundary was reached (hence the parser does not signal that the final boundary was found). For example:

I tried your example in my test setup and did not get that error.

Suggestions to proceed:

airween commented 3 years ago

Hi @martinhsv,

thanks for quick feedback.

  • the request body stopped being processed due to a configured limit (for example using SecRequestBodyLimit with the associated action being ProcessPartial)

as you can see in my modsecurity.conf, I have an explicit value for that:

SecRequestBodyLimit 13107200

So I'm sure the problem is not this (the size of sent file is 10 bytes). Also you can see the whole config in @defanator's repository - the value is 256M.

  • the request body stopped being processed due to a serious parsing issue. For example one of the multipart parts did not have a "name"

this is the relevant output of tcpdump it runs during the request:

POST /upload/ HTTP/1.1
Host: localhost
User-Agent: curl/7.64.0
Accept: */*
Content-Length: 196
Content-Type: multipart/form-data; boundary=------------------------426732a72a1236b1

--------------------------426732a72a1236b1
Content-Disposition: form-data; name="data"; filename="text.txt"
Content-Type: text/plain

test text

--------------------------426732a72a1236b1--

There are both name argument and final boundary.

I tried your example in my test setup and did not get that error.

Can you share your setup? Or can you try with the mentioned Vagrant config?

  • the debug log (at level 9) may give away what other error was encountered during parsing

you mean about this?

[1617312599] [] [4] Initializing transaction
[1617312599] [] [4] Transaction context created.
[1617312599] [] [4] Starting phase CONNECTION. (SecRules 0)
[1617312599] [] [9] This phase consists of 0 rule(s).
[1617312599] [] [4] Starting phase URI. (SecRules 0 + 1/2)
[1617312599] [/upload/] [4] Starting phase REQUEST_HEADERS.  (SecRules 1)
[1617312599] [/upload/] [9] This phase consists of 2 rule(s).
[1617312599] [/upload/] [4] (Rule: 200000) Executing operator "Rx" with param "(?:application(?:/soap\+|/)|text/)xml" against REQUEST_HEADERS:Content-Type.
[1617312599] [/upload/] [9]  T (0) t:lowercase: "multipart/form-data; boundary=------------------------426732a72a1236b1"
[1617312599] [/upload/] [9] Target value: "multipart/form-data; boundary=------------------------426732a72a1236b1" (Variable: REQUEST_HEADERS:Content-Type)
[1617312599] [/upload/] [4] Rule returned 0.
[1617312599] [/upload/] [9] Matched vars cleaned.
[1617312599] [/upload/] [4] (Rule: 200001) Executing operator "Rx" with param "application/json" against REQUEST_HEADERS:Content-Type.
[1617312599] [/upload/] [9]  T (0) t:lowercase: "multipart/form-data; boundary=------------------------426732a72a1236b1"
[1617312599] [/upload/] [9] Target value: "multipart/form-data; boundary=------------------------426732a72a1236b1" (Variable: REQUEST_HEADERS:Content-Type)
[1617312599] [/upload/] [4] Rule returned 0.
[1617312599] [/upload/] [9] Matched vars cleaned.
[1617312599] [/upload/] [9] Appending request body: 196 bytes. Limit set to: 13107200.000000
[1617312599] [/upload/] [4] Starting phase REQUEST_BODY. (SecRules 2)
[1617312599] [/upload/] [9] Multipart: Boundary: ------------------------426732a72a1236b1
[1617312599] [/upload/] [9] Multipart: Added part header "Content-Disposition" "form-data; name="data"; filename="text.txt"".
[1617312599] [/upload/] [9] Multipart: Added part header "Content-Type" "text/plain".
[1617312599] [/upload/] [9] Multipart: Content-Disposition name: data.
[1617312599] [/upload/] [9] Multipart: Content-Disposition filename: text.txt.
[1617312599] [/upload/] [9] Multipart: Added file part to the list: name "data" file name "text.txt" (offset 138, length 10)
[1617312599] [/upload/] [4] Multipart: Cleanup started (keep files set to Not set)
[1617312599] [/upload/] [9] This phase consists of 4 rule(s).
[1617312599] [/upload/] [4] (Rule: 200002) Executing operator "Eq" with param "0" against REQBODY_ERROR.
[1617312599] [/upload/] [9] Target value: "0" (Variable: REQBODY_ERROR)
[1617312599] [/upload/] [4] Rule returned 0.
[1617312599] [/upload/] [9] Matched vars cleaned.
[1617312599] [/upload/] [4] (Rule: 200003) Executing operator "Eq" with param "0" against MULTIPART_STRICT_ERROR.
[1617312599] [/upload/] [9] Target value: "0" (Variable: MULTIPART_STRICT_ERROR)
[1617312599] [/upload/] [4] Rule returned 0.
[1617312599] [/upload/] [9] Matched vars cleaned.
[1617312599] [/upload/] [4] (Rule: 200004) Executing operator "Gt" with param "1" against MULTIPART_UNMATCHED_BOUNDARY.
[1617312599] [/upload/] [9] Target value: "2" (Variable: MULTIPART_UNMATCHED_BOUNDARY)
[1617312599] [/upload/] [9] Matched vars updated.
[1617312599] [/upload/] [4] Rule returned 1.
[1617312599] [/upload/] [9] Saving msg: Multipart parser detected a possible unmatched boundary.
[1617312599] [/upload/] [9] Running action: log
  • is there a chance that the file content is triggering a parsing error? One thought is to try to try the same test but with extremely simple file content -- for example the 6-char string abcdef
$ cat abcdef.txt 
abcdef
$ curl -vvv "http://localhost/upload/" -F "data=@abcdef.txt"
* Expire in 0 ms for 6 (transfer 0x55ffa8f4df50)
...
* Expire in 1 ms for 1 (transfer 0x55ffa8f4df50)
*   Trying ::1...
* TCP_NODELAY set
* Expire in 149998 ms for 3 (transfer 0x55ffa8f4df50)
* Expire in 200 ms for 4 (transfer 0x55ffa8f4df50)
* connect to ::1 port 80 failed: Connection refused
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Expire in 149998 ms for 3 (transfer 0x55ffa8f4df50)
* Connected to localhost (127.0.0.1) port 80 (#0)
> POST /upload/ HTTP/1.1
> Host: localhost
> User-Agent: curl/7.64.0
> Accept: */*
> Content-Length: 195
> Content-Type: multipart/form-data; boundary=------------------------f67b76cec353b936
> 
< HTTP/1.1 403 Forbidden
< Server: nginx/1.19.6
< Date: Thu, 01 Apr 2021 21:32:21 GMT
< Content-Type: text/html
< Content-Length: 153
< Connection: keep-alive
* HTTP error before end of send, stop sending
< 
<html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx/1.19.6</center>
</body>
</html>
* Closing connection 0

error.log:

2021/04/01 21:32:21 [error] 7378#7378: *3 [client 127.0.0.1] ModSecurity: Access denied with code 403 (phase 2). Matched "Operator `Gt' with parameter `1' against variable `MULTIPART_UNMATCHED_BOUNDARY' (Value: `2' ) [file "/etc/nginx/modsec/modsecurity.conf"] [line "15"] [id "200004"] [rev ""] [msg "Multipart parser detected a possible unmatched boundary."] [data ""] [severity "0"] [ver ""] [maturity "0"] [accuracy "0"] [hostname "127.0.0.1"] [uri "/upload/"] [unique_id "1617312741"] [ref "v180,1"], client: 127.0.0.1, server: , request: "POST /upload/ HTTP/1.1", host: "localhost"

debug.log

[1617312741] [] [4] Initializing transaction
[1617312741] [] [4] Transaction context created.
[1617312741] [] [4] Starting phase CONNECTION. (SecRules 0)
[1617312741] [] [9] This phase consists of 0 rule(s).
[1617312741] [] [4] Starting phase URI. (SecRules 0 + 1/2)
[1617312741] [/upload/] [4] Starting phase REQUEST_HEADERS.  (SecRules 1)
[1617312741] [/upload/] [9] This phase consists of 2 rule(s).
[1617312741] [/upload/] [4] (Rule: 200000) Executing operator "Rx" with param "(?:application(?:/soap\+|/)|text/)xml" against REQUEST_HEADERS:Content-Type.
[1617312741] [/upload/] [9]  T (0) t:lowercase: "multipart/form-data; boundary=------------------------f67b76cec353b936"
[1617312741] [/upload/] [9] Target value: "multipart/form-data; boundary=------------------------f67b76cec353b936" (Variable: REQUEST_HEADERS:Content-Type)
[1617312741] [/upload/] [4] Rule returned 0.
[1617312741] [/upload/] [9] Matched vars cleaned.
[1617312741] [/upload/] [4] (Rule: 200001) Executing operator "Rx" with param "application/json" against REQUEST_HEADERS:Content-Type.
[1617312741] [/upload/] [9]  T (0) t:lowercase: "multipart/form-data; boundary=------------------------f67b76cec353b936"
[1617312741] [/upload/] [9] Target value: "multipart/form-data; boundary=------------------------f67b76cec353b936" (Variable: REQUEST_HEADERS:Content-Type)
[1617312741] [/upload/] [4] Rule returned 0.
[1617312741] [/upload/] [9] Matched vars cleaned.
[1617312741] [/upload/] [9] Appending request body: 195 bytes. Limit set to: 13107200.000000
[1617312741] [/upload/] [4] Starting phase REQUEST_BODY. (SecRules 2)
[1617312741] [/upload/] [9] Multipart: Boundary: ------------------------f67b76cec353b936
[1617312741] [/upload/] [9] Multipart: Added part header "Content-Disposition" "form-data; name="data"; filename="abcdef.txt"".
[1617312741] [/upload/] [9] Multipart: Added part header "Content-Type" "text/plain".
[1617312741] [/upload/] [9] Multipart: Content-Disposition name: data.
[1617312741] [/upload/] [9] Multipart: Content-Disposition filename: abcdef.txt.
[1617312741] [/upload/] [9] Multipart: Added file part to the list: name "data" file name "abcdef.txt" (offset 140, length 7)
[1617312741] [/upload/] [4] Multipart: Cleanup started (keep files set to Not set)
[1617312741] [/upload/] [9] This phase consists of 4 rule(s).
[1617312741] [/upload/] [4] (Rule: 200002) Executing operator "Eq" with param "0" against REQBODY_ERROR.
[1617312741] [/upload/] [9] Target value: "0" (Variable: REQBODY_ERROR)
[1617312741] [/upload/] [4] Rule returned 0.
[1617312741] [/upload/] [9] Matched vars cleaned.
[1617312741] [/upload/] [4] (Rule: 200003) Executing operator "Eq" with param "0" against MULTIPART_STRICT_ERROR.
[1617312741] [/upload/] [9] Target value: "0" (Variable: MULTIPART_STRICT_ERROR)
[1617312741] [/upload/] [4] Rule returned 0.
[1617312741] [/upload/] [9] Matched vars cleaned.
[1617312741] [/upload/] [4] (Rule: 200004) Executing operator "Gt" with param "1" against MULTIPART_UNMATCHED_BOUNDARY.
[1617312741] [/upload/] [9] Target value: "2" (Variable: MULTIPART_UNMATCHED_BOUNDARY)
[1617312741] [/upload/] [9] Matched vars updated.
[1617312741] [/upload/] [4] Rule returned 1.
[1617312741] [/upload/] [9] Saving msg: Multipart parser detected a possible unmatched boundary.
[1617312741] [/upload/] [9] Running action: log

Same result.

martinhsv commented 3 years ago

Thanks for the extra information. I'll try to have a closer look on Monday.

airween commented 3 years ago

I think I found the root cause of the problem - check this rule:

+SecRule MULTIPART_UNMATCHED_BOUNDARY "@gt 1" \
+    "id:'200004',phase:2,t:none,log,deny,msg:'Multipart parser detected a possible unmatched boundary.'"

I have no idea where did I got this rule, but this is definitely wrong. According to recommended modsecurity.conf, the correct operator and its argument are @eq 1 not @gt 1.

So that was my faul - thanks for your time and sorry for the noise.

martinhsv commented 3 years ago

Hi @airween ,

Given your update, I didn't wind up debugging this.

However, I strongly suspect that this is an unintended effect of the way the PEM solution was implemented (#1747 and #1924) .

In my 'B' comments at https://github.com/SpiderLabs/ModSecurity/pull/2417#issuecomment-712296544 , note the use case when the PEM-like content is in the final part of the multipart body.

Just from code inspection, I suspect what you've recently seen is a similarly impacted case. I.e. that in your case, the flag=2 result is being triggered because there is only one part within the whole multipart body.

I think this lends weight that we this PEM-inspired modification should really be properly revisited.

airween commented 3 years ago

Hi @martinhsv,

thanks for your time.

Just from code inspection, I suspect what you've recently seen is a similarly impacted case. I.e. that in your case, the flag=2 result is being triggered because there is only one part within the whole multipart body.

yes, I think this is true - but the problem was in my config, namely the operator and its argument was wrong in that rule. I have no idea where did I get, but luckily I found the root case.

I think this lends weight that we this PEM-inspired modification should really be properly revisited.

yes, absolutely - as I see the sent PR is nearly half year old. Do you plan to approve it only to 3.1?

Thanks again.