search

apache / apisix

The Cloud-Native API Gateway
https://apisix.apache.org/blog/
Apache License 2.0
14.55k stars 2.52k forks source link

bug: request-validation plugin is buggy when header 'Content-Type: application/x-www-form-urlencoded' has parameters #11687

Open bradib0y opened 4 weeks ago

bradib0y commented 4 weeks ago

Current Behavior

The Content-Type header value has this required format, according to RFC 7231:

Valid Examples: Content-Type: application/x-www-form-urlencoded Content-Type: application/x-www-form-urlencoded; charset=utf-8

The second example is parameterized. In that case, APISIX request-validation plugin will fail with the following error: parse error: Invalid numeric literal at line 1, column 8

Removing the plugin is the only solution, if you have a parameterized Content-Type header.

Expected Behavior

It should accept the parameterized content type header.

I found the issue in the source code. The body is treated as JSON by default, then the form data type is check with an exact header value match, so if it is extended by params, the body will be treated as JSON, and failing with the validation on the first character, because it is not { or [ as expected with JSON.

Source from the repo: apisix\plugins\request-validation.lua

local body_is_json = true
if headers["content-type"] == "application/x-www-form-urlencoded" then -- đŸ”´ the issue is here. It should be a Lua equivalent of "startsWith" not "=="
    -- use 0 to avoid truncated result and keep the behavior as the
    -- same as other platforms
    req_body, err = ngx.decode_args(body, 0)
    body_is_json = false
else -- JSON as default
    req_body, err = core.json.decode(body)
end

Error Logs

Response

parse error: Invalid numeric literal at line 1, column 8

APISIX logs

2024/10/28 17:04:41 [warn] 24#24: *2233722 [lua] utils.lua:418: find_and_log(): Using openid-connect introspection_endpoint with no TLS is a security risk, context: ngx.timer
2024/10/28 17:04:41 [warn] 24#24: *2233722 [lua] utils.lua:458: check_tls_bool(): Keeping ssl_verify disabled in openid-connect configuration is a security risk, context: ngx.timer
2024/10/28 17:04:49 [error] 10#10: *2383148 [lua] request-validation.lua:101: phasefunc(): failed to decode the req body: Expected value but found invalid token at character 1, client: 10.111.4.1, server: , request: "POST /auth/login HTTP/2.0", host: "apisixmyteam.myteam.mycompany.dev"
2024/10/28 17:04:49 [warn] 10#10: *2383148 [lua] plugin.lua:1171: runplugin(): request-validation exits with http status code 400, client: 10.111.4.1, server: , request: "POST /auth/login HTTP/2.0", host: "apisixmyteam.myteam.mycompany.dev"
10.111.4.1 - apisix-myproject [28/Oct/2024:17:04:47 +0000] apisixmyteam.myteam.mycompany.dev "POST /auth/login HTTP/2.0" 200 2345 0.076 "-" "curl/7.81.0" 10.99.211.21:80 200 0.076 "http://apisixmyteam.myteam.mycompany.dev/auth/realms/myproject-dev/protocol/openid-connect/token"
10.111.4.1 - - [28/Oct/2024:17:04:49 +0000] apisixmyteam.myteam.mycompany.dev "POST /auth/login HTTP/2.0" 400 35 0.000 "-" "curl/7.81.0" - - - "http://apisixmyteam.myteam.mycompany.dev"
10.111.4.1 - apisix-myproject [28/Oct/2024:17:05:45 +0000] apisixmyteam.myteam.mycompany.dev "POST /auth/login HTTP/2.0" 200 2345 0.066 "-" "curl/7.81.0" 10.99.211.21:80 200 0.066 "http://apisixmyteam.myteam.mycompany.dev/auth/realms/myproject-dev/protocol/openid-connect/token"
2024/10/28 17:06:01 [error] 15#15: *2385302 [lua] request-validation.lua:101: phasefunc(): failed to decode the req body: Expected value but found invalid token at character 1, client: 10.111.4.1, server: , request: "POST /auth/login HTTP/2.0", host: "apisixmyteam.myteam.mycompany.dev"
2024/10/28 17:06:01 [warn] 15#15: *2385302 [lua] plugin.lua:1171: runplugin(): request-validation exits with http status code 400, client: 10.111.4.1, server: , request: "POST /auth/login HTTP/2.0", host: "apisixmyteam.myteam.mycompany.dev"
10.111.4.1 - - [28/Oct/2024:17:06:01 +0000] apisixmyteam.myteam.mycompany.dev "POST /auth/login HTTP/2.0" 400 35 0.000 "-" "curl/7.81.0" - - - "http://apisixmyteam.myteam.mycompany.dev"
2024/10/28 17:06:28 [error] 9#9: *2386078 [lua] request-validation.lua:101: phasefunc(): failed to decode the req body: Expected value but found invalid token at character 1, client: 10.111.4.1, server: , request: "POST /auth/login HTTP/2.0", host: "apisixmyteam.myteam.mycompany.dev"
2024/10/28 17:06:28 [warn] 9#9: *2386078 [lua] plugin.lua:1171: runplugin(): request-validation exits with http status code 400, client: 10.111.4.1, server: , request: "POST /auth/login HTTP/2.0", host: "apisixmyteam.myteam.mycompany.dev"
10.111.4.1 - - [28/Oct/2024:17:06:28 +0000] apisixmyteam.myteam.mycompany.dev "POST /auth/login HTTP/2.0" 400 35 0.000 "-" "curl/7.81.0" - - - "http://apisixmyteam.myteam.mycompany.dev"

Steps to Reproduce

  1. create an apisix route
  2. configure request-validation plugin with any settings
  3. send a request with the following header, it will work: Content-Type: application/x-www-form-urlencoded
  4. send a request with the following header, it won't work: Content-Type: application/x-www-form-urlencoded; charset=utf-8

`

Environment

bradib0y commented 4 weeks ago

I fixed the bug and wrote a test case, but I was not able to run it

fix (request-validation.lua)

        local body_is_json = true
        local FORM_URLENCODED_MEDIA_TYPE="application/x-www-form-urlencoded"
        if string.sub(headers["content-type"], 1,
            string.len(FORM_URLENCODED_MEDIA_TYPE)) == FORM_URLENCODED_MEDIA_TYPE then
            -- use 0 to avoid truncated result and keep the behavior as the
            -- same as other platforms
            req_body, err = ngx.decode_args(body, 0)
            body_is_json = false
        else -- JSON as default
            req_body, err = core.json.decode(body)
        end

test case (request-validation.t)

=== TEST 53: test urlencoded post data with charset parameter
--- more_headers
Content-Type: application/x-www-form-urlencoded; charset=utf-8
--- request eval
"POST /echo
" . "a=b&" x 101 . "required_payload=101-hello"
--- response_body eval
qr/101-hello/