File upload : files over 120 kB returns 500 error

[Saperlu]

Hi, I found out files over 120 kB cant be uploaded when coraza is enabled, and it is not about a CRS rule file because I disabled them.

create the file with : dd if=/dev/zero of=file.pdf bs=10K count=13

caddy configuration

    coraza_waf {
            directives `
                SecAction \
                    "id:900000,\
                    phase:1,\
                    nolog,\
                    pass,\
                    t:none,\
                    setvar:tx.blocking_paranoia_level=1"
                SecAction \
                    "id:900001,\
                    phase:1,\
                    nolog,\
                    pass,\
                    t:none,\
                    setvar:tx.detection_paranoia_level=1"
            `
        include coraza.conf
        include coreruleset/crs-setup.conf
    }

coraza.conf

I tried uping SecRequestBodyLimit and setting SecRequestBodyAccess but this has no effect

SecRuleEngine On

SecRequestBodyAccess Off

SecRule REQUEST_HEADERS:Content-Type "(?:application(?:/soap\+|/)|text/)xml" \
     "id:'200000',phase:1,t:none,t:lowercase,pass,nolog,ctl:requestBodyProcessor=XML"

SecRule REQUEST_HEADERS:Content-Type "application/json" \
     "id:'200001',phase:1,t:none,t:lowercase,pass,nolog,ctl:requestBodyProcessor=JSON"

SecRequestBodyLimit 131072000000

SecRule REQBODY_ERROR "!@eq 0" \
"id:'200002', phase:2,t:none,log,deny,status:400,msg:'Failed to parse request body.',logdata:'%{reqbody_error_msg}',severity:2"

SecRule MULTIPART_STRICT_ERROR "!@eq 0" \
"id:'200003',phase:2,t:none,log,deny,status:400, \
msg:'Multipart request body failed strict validation: \
PE %{REQBODY_PROCESSOR_ERROR}, \
BQ %{MULTIPART_BOUNDARY_QUOTED}, \
BW %{MULTIPART_BOUNDARY_WHITESPACE}, \
DB %{MULTIPART_DATA_BEFORE}, \
DA %{MULTIPART_DATA_AFTER}, \
HF %{MULTIPART_HEADER_FOLDING}, \
LF %{MULTIPART_LF_LINE}, \
SM %{MULTIPART_MISSING_SEMICOLON}, \
IQ %{MULTIPART_INVALID_QUOTING}, \
IP %{MULTIPART_INVALID_PART}, \
IH %{MULTIPART_INVALID_HEADER_FOLDING}, \
FL %{MULTIPART_FILE_LIMIT_EXCEEDED}'"

SecRule MULTIPART_UNMATCHED_BOUNDARY "@eq 1" \
    "id:'200004',phase:2,t:none,log,deny,msg:'Multipart parser detected a possible unmatched boundary.'"

SecRule TX:/^COR_/ "!@streq 0" \
        "id:'200005',phase:2,t:none,deny,msg:'Coraza internal error flagged: %{MATCHED_VAR_NAME}'"

SecResponseBodyAccess On
SecResponseBodyMimeType text/plain text/html text/xml
SecResponseBodyLimit 524288
SecResponseBodyLimitAction ProcessPartial
SecTmpDir /tmp/coraza/tmp
SecDataDir /tmp/coraza/data
SecAuditEngine On
SecAuditLogType concurrent 
SecAuditLogStorageDir /var/log/caddy/
SecAuditLogRelevantStatus "^(?:(5|4)(0|1)[0-9])$"
SecArgumentSeparator &
SecCookieFormat 0
SecDefaultAction "phase:1,log,auditlog,pass"
SecDefaultAction "phase:2,log,auditlog,pass"

SecAction \
"id:900220,\
 phase:1,\
 nolog,\
 pass,\
 t:none,\
 setvar:'tx.allowed_request_content_type=|application/x-www-form-urlencoded| |multipart/form-data| |multipart/related| |text/xml| |application/xml| |application/soap+xml| |application/x-amf| |application/json| |application/cloudevents+json| |application/cloudevents-batch+json| |application/octet-stream| |application/csp-report| |application/xss-auditor-report| |text/plain| |application/x-www-form-urlencoded;| |multipart/form-data;| |multipart/related;| |text/xml;| |application/xml;| |application/soap+xml;| |application/x-amf;| |application/json;| |application/cloudevents+json;| |application/cloudevents-batch+json;| |application/octet-stream;| |application/csp-report;| |application/xss-auditor-report;| |text/plain|'"

SecCollectionTimeout 600

SecAction \
    "id:900990,\
    phase:1,\
    nolog,\
    pass,\
    t:none,\
    setvar:tx.crs_setup_version=400"

caddy logs

I get this error when triggering the 500 error (/tmp/coraza/tmp is SecTmpDir) :

http.log.error  open /tmp/coraza/tmp/body2809431227: no such file or directory  {"request": {"remote_ip": "::1", "remote_port": "46222", "proto": "HTTP/1.1", "method": "POST", "host": "[redacted]", "uri": "/fr/imce", "headers": {"Te": ["trailers"], "X-Forwarded-For": ["[redacted]"], "Content-Type": ["multipart/form-data; boundary=---------------------------755505775042860173259666447"], "Origin": ["https://www.bookbeo.com"], "Referer": ["[redacted]/imce"], "X-Forwarded-Proto": ["https"], "X-Requested-With": ["XMLHttpRequest"], "User-Agent": ["Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:106.0) Gecko/20100101 Firefox/106.0"], "Cookie": [], "Sec-Fetch-Site": ["same-origin"], "X-Forwarded-Host": ["[redacted]"], "X-Request-Id": ["g5hzRLV8qFTs3jo9ZoxVB"], "Accept": ["application/json, text/javascript, */*; q=0.01"], "Sec-Fetch-Mode": ["cors"], "Accept-Language": ["fr,fr-FR;q=0.8,en-US;q=0.5,en;q=0.3"], "Sec-Fetch-Dest": ["empty"], "Content-Length": ["133737"], "Accept-Encoding": ["gzip, deflate"]}}, "duration": 0.323760727}

[jptosso]

Hey @Saperlu thank you for your report. Before debugging, can you validate the directory /tmp/coraza/tmp exists and that the user running caddy has access to write to it?

[Saperlu]

Well played, this dir was not on the container where I was trying to replicate the bug, sorry for that.

So I tried on the prod container (the dir exists on that container) and it seems that setting SecResponseBodyAccess to Off works. Is there a way to disable this for a custom URI only ?

Also, this shouldn't have this behaviour, I guess, so here are caddy logs :

error   http.log.error  EOF {"request": {"remote_ip": "[redacted]", "remote_port": "65182", "proto": "HTTP/2.0", "method": "POST", "host": "[redacted]", "uri": "/fr/imce", "headers": {"X-Requested-With": ["XMLHttpRequest"], "Sec-Ch-Ua-Platform": ["\"macOS\""], "Sec-Fetch-Site": ["same-origin"], "Sec-Fetch-Dest": ["empty"], "Content-Length": ["16389"], "Sec-Ch-Ua": ["\"Google Chrome\";v=\"107\", \"Chromium\";v=\"107\", \"Not=A?Brand\";v=\"24\""], "Accept-Encoding": ["gzip, deflate, br"], "Accept-Language": ["fr-FR,fr;q=0.9,en-US;q=0.8,en;q=0.7"], "User-Agent": ["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36"], "Referer": ["https://[redacted]/imce"], "Sec-Ch-Ua-Mobile": ["?0"], "Origin": ["https://[redacted]"], "Cookie": [], "Accept": ["application/json, text/javascript, */*; q=0.01"], "Content-Type": ["multipart/form-data; boundary=----WebKitFormBoundaryoiUZBSJxElcPL8SO"], "Sec-Fetch-Mode": ["cors"]}, "tls": {"resumed": false, "version": 772, "cipher_suite": 4865, "proto": "h2", "server_name": "www.bookbeo.com"}}, "duration": 0.241701613, "status": 502, "err_id": "3mucdx5hf", "err_trace": "reverseproxy.statusError (reverseproxy.go:1271)"}

[jcchavezs]

Interesting finding. Worth to check cc @anuraaga @m4tteoP

On Thu, 10 Nov 2022, 14:23 Lucien Charleux, @.***> wrote:

Well played, this dir was not on the container where I was trying to replicate the bug.

So I tried on the prod container (the dir exists on that container) and it seems that setting SecResponseBodyAccess to Off works. Is there a way to disable this for a custom URI only ?

Also, this shouldn't have this behaviour, I guess, so here are caddy logs :

error http.log.error EOF {"request": {"remote_ip": "172.16.40.100", "remote_port": "65182", "proto": "HTTP/2.0", "method": "POST", "host": "www.bookbeo.com", "uri": "/fr/imce", "headers": {"X-Requested-With": ["XMLHttpRequest"], "Sec-Ch-Ua-Platform": ["\"macOS\""], "Sec-Fetch-Site": ["same-origin"], "Sec-Fetch-Dest": ["empty"], "Content-Length": ["16389"], "Sec-Ch-Ua": ["\"Google Chrome\";v=\"107\", \"Chromium\";v=\"107\", \"Not=A?Brand\";v=\"24\""], "Accept-Encoding": ["gzip, deflate, br"], "Accept-Language": ["fr-FR,fr;q=0.9,en-US;q=0.8,en;q=0.7"], "User-Agent": ["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36"], "Referer": ["https://www.bookbeo.com/fr/imce"], "Sec-Ch-Ua-Mobile": ["?0"], "Origin": ["https://www.bookbeo.com"], "Cookie": [], "Accept": ["application/json, text/javascript, /; q=0.01"], "Content-Type": ["multipart/form-data; boundary=----WebKitFormBoundaryoiUZBSJxElcPL8SO"], "Sec-Fetch-Mode": ["cors"]}, "tls": {"resumed": false, "version": 772, "cipher_suite": 4865, "proto": "h2", "server_name": "www.bookbeo.com"}}, "duration": 0.241701613, "status": 502, "err_id": "3mucdx5hf", "err_trace": "reverseproxy.statusError (reverseproxy.go:1271)"}

— Reply to this email directly, view it on GitHub https://github.com/corazawaf/coraza-caddy/issues/29#issuecomment-1310276642, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAXOYAR7QL2PB7BYZPVGU2LWHTZMZANCNFSM6AAAAAAR4M3BIM . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[jptosso]

Caddy connector is using an old version of coraza. There is a draft PR to upgrade to the new immutability pattern.

[jptosso]

Fully Upgraded in https://github.com/corazawaf/coraza-caddy/releases/tag/v1.2.2

[jcchavezs]

Could you please check again @Saperlu ?

[jcchavezs]

We completely rewrite the connector. Could you please try it again?

[github-actions[bot]]

This issue has been open 30 days waiting for feedback. Remove the stale label or comment, or this will be closed in 14 days.

[github-actions[bot]]

This issue was closed because it has been inactive for 14 days since being marked as stale.