Failure accessing web UI and API thru reverse proxy on firmware 1.20 - HTTP error 413

[jho1670]

This has been working for me, previously, and broke upon upgrade to 1.20.

I have caddyserver with a SSO portal auth, reverse proxying opengarage. When I load it now, I get The request was too large.

Sample Caddyfile

og.fqdn.domain.com {
        route * {
                reverse_proxy * {
                        to 192.168.1.10
                }
        }
        log {
                output file /var/log/caddy/og.log
        }
}

The caddyserver log seems to show opengarage is returning HTTP 413, when I'm hitting /jc (for example)

{"level":"error","ts":1669087222.8393056,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_ip":"<removed>","remote_port":"10811","proto":"HTTP/2.0","method":"GET","host":"<removed>","uri":"/jc","headers":{"Accept-Language":["en-US,en;q=0.9"],"Sec-Ch-Ua":["\"Chromium\";v=\"107\", \"Not=A?Brand\";v=\"24\""],"Sec-Ch-Ua-Platform":["\"Windows\""],"Upgrade-Insecure-Requests":["1"],"Sec-Fetch-Site":["none"],"Sec-Fetch-Dest":["document"],"Accept-Encoding":["gzip, deflate, br"],"Cache-Control":["max-age=0"],"Sec-Ch-Ua-Mobile":["?0"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"],"Sec-Fetch-Mode":["navigate"],"Cookie":[],"Sec-Fetch-User":["?1"],"Dnt":["1"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36"]},"tls":{"resumed":false,"version":772,"cipher_suite":4867,"proto":"h2","server_name":"<removed>"}},"user_id":"<removed>","duration":0.020516459,"size":25,"status":413,"resp_headers":{"Server":["Caddy"]}} {"level":"error","ts":1669087222.9489264,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_ip":"<removed>","remote_port":"10811","proto":"HTTP/2.0","method":"GET","host":"<removed>","uri":"/favicon.ico","headers":{"Cookie":[],"Dnt":["1"],"Sec-Ch-Ua":["\"Chromium\";v=\"107\", \"Not=A?Brand\";v=\"24\""],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36"],"Accept":["image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8"],"Sec-Fetch-Site":["same-origin"],"Referer":["<removed>/jc"],"Accept-Encoding":["gzip, deflate, br"],"Accept-Language":["en-US,en;q=0.9"],"Sec-Ch-Ua-Mobile":["?0"],"Sec-Ch-Ua-Platform":["\"Windows\""],"Sec-Fetch-Mode":["no-cors"],"Sec-Fetch-Dest":["image"]},"tls":{"resumed":false,"version":772,"cipher_suite":4867,"proto":"h2","server_name":"<removed>"}},"user_id":"<removed>","duration":0.019879474,"size":25,"status":413,"resp_headers":{"Server":["Caddy"]}}

[rayshobby]

The major change in firmware 1.2.0 is adopting OTC for remote access. The local server loop is also handled by the OTC library. I see two places that can return error 413: https://github.com/OpenThingsIO/OpenThings-Framework-Firmware-Library/blob/master/OTF-Controller-Library/OpenThingsFramework.cpp#L137 https://github.com/OpenThingsIO/OpenThings-Framework-Firmware-Library/blob/master/OTF-Controller-Library/OpenThingsFramework.cpp#L172 For the moment perhaps the best work-around for you is to downgrade to firmware 1.1.2.

[rayshobby]

I am wondering if your caddy server is sending a request that is not GET request (i.e. it might be a POST or PUT request etc.) The reason I am asking is that after some debugging, I found the OTF library has a bug that results in any non-GET request returning a 413 error. The normal use case of OpenGarage shouldn't trigger this because all requests are indeed GET requests. But since you have another server that's sending request to it, I am not sure what it's sending to the controller, and it can certainly trigger this bug.

[rayshobby]

I just checked in a new version of firmware, 1.2.1, which is firmware 1.2.0 but compiled with the OTF local server issue fixed (i.e. the OTF local server loop had a bug when handling POST or non-GET requests). You can give it a try: https://github.com/OpenGarage/OpenGarage-Firmware/tree/master/Compiled If the problem still exists, then the only remaining possibility is your caddy server is sending a request to OpenGarage that has a long header (larger than 1KB in size). That would be the other place that triggers the 413 error.