search

gbip / sentry_tunnel

Proxy sentry request to a sentry server using a tunnel/proxy endpoint
BSD 2-Clause "Simplified" License
28 stars 11 forks source link

Big request body yields HTTP 400 Bad Request #9

Open dominikks opened 12 months ago

dominikks commented 12 months ago

In our Sentry deployment, we routinely get envelopes upwards of 8 kB in size. An example envelope might look as follows:

{"event_id":"394892e5d08540629f04030f086fd7a1","sent_at":"2023-07-28T12:35:10.454Z","sdk":{"name":"sentry.javascript.angular-ivy","version":"7.57.0"},"dsn":"https://abcdefgh@anonymized-sentry-instance.com/1"}
{"type":"event"}
{"exception":{"values":[{"type":"TypeError","value":"null has no properties","stacktrace":{"frames":[{"filename":"https://our-app-url.com/de/main.0580ea5ff873bfe6.js","function":"Me","in_app":true,"lineno":3,"colno":1823190},{"filename":"https://our-app-url.com/de/polyfills.e2fee55d4d4e7362.js","function":"Ae/t[s]","in_app":true,"lineno":1,"colno":34874},{"filename":"https://our-app-url.com/de/polyfills.e2fee55d4d4e7362.js","function":"16518/</</</</<","in_app":true,"lineno":1,"colno":546},{"filename":"https://our-app-url.com/de/main.0580ea5ff873bfe6.js","function":"ngDoCheck","in_app":true,"lineno":3,"colno":139492},{"filename":"https://our-app-url.com/de/main.0580ea5ff873bfe6.js","function":"detectChanges","in_app":true,"lineno":3,"colno":1866126},{"filename":"https://our-app-url.com/de/main.0580ea5ff873bfe6.js","function":"wc","in_app":true,"lineno":3,"colno":1864024},{"filename":"https://our-app-url.com/de/main.0580ea5ff873bfe6.js","function":"ic","in_app":true,"lineno":3,"colno":1855751},{"filename":"https://our-app-url.com/de/main.0580ea5ff873bfe6.js","function":"v_","in_app":true,"lineno":3,"colno":1857009},{"filename":"https://our-app-url.com/de/main.0580ea5ff873bfe6.js","function":"template","in_app":true,"lineno":3,"colno":142743},{"filename":"https://our-app-url.com/de/main.0580ea5ff873bfe6.js","function":"Ku","in_app":true,"lineno":3,"colno":1872812},{"filename":"https://our-app-url.com/de/main.0580ea5ff873bfe6.js","function":"Ro","in_app":true,"lineno":3,"colno":1859256},{"filename":"https://our-app-url.com/de/main.0580ea5ff873bfe6.js","function":"yd","in_app":true,"lineno":3,"colno":1864573},{"filename":"https://our-app-url.com/de/main.0580ea5ff873bfe6.js","function":"set ngIf","in_app":true,"lineno":3,"colno":1773982},{"filename":"https://our-app-url.com/de/main.0580ea5ff873bfe6.js","function":"_updateView","in_app":true,"lineno":3,"colno":1774371},{"filename":"https://our-app-url.com/de/main.0580ea5ff873bfe6.js","function":"createEmbeddedView","in_app":true,"lineno":3,"colno":1906839},{"filename":"https://our-app-url.com/de/main.0580ea5ff873bfe6.js","function":"createEmbeddedView","in_app":true,"lineno":3,"colno":1905948},{"filename":"https://our-app-url.com/de/main.0580ea5ff873bfe6.js","function":"Il","in_app":true,"lineno":3,"colno":1855275},{"filename":"https://our-app-url.com/de/main.0580ea5ff873bfe6.js","function":"v_","in_app":true,"lineno":3,"colno":1857009},{"filename":"https://our-app-url.com/de/main.0580ea5ff873bfe6.js","function":"Yo","in_app":true,"lineno":3,"colno":135807},{"filename":"https://our-app-url.com/de/main.0580ea5ff873bfe6.js","function":"eh","in_app":true,"lineno":3,"colno":1874809},{"filename":"https://our-app-url.com/de/main.0580ea5ff873bfe6.js","function":"O","in_app":true,"lineno":3,"colno":1875507},{"filename":"https://our-app-url.com/de/main.0580ea5ff873bfe6.js","function":"listen","in_app":true,"lineno":3,"colno":109488},{"filename":"https://our-app-url.com/de/main.0580ea5ff873bfe6.js","function":"listen","in_app":true,"lineno":3,"colno":2540963},{"filename":"https://our-app-url.com/de/main.0580ea5ff873bfe6.js","function":"addEventListener","in_app":true,"lineno":3,"colno":2535567},{"filename":"https://our-app-url.com/de/main.0580ea5ff873bfe6.js","function":"addEventListener","in_app":true,"lineno":3,"colno":2542769},{"filename":"https://our-app-url.com/de/main.0580ea5ff873bfe6.js","function":"k/</</<","in_app":true,"lineno":3,"colno":1063456},{"filename":"https://our-app-url.com/de/polyfills.e2fee55d4d4e7362.js","function":"Ae/t[s]","in_app":true,"lineno":1,"colno":34874},{"filename":"https://our-app-url.com/de/polyfills.e2fee55d4d4e7362.js","function":"16518/</</</</<","in_app":true,"lineno":1,"colno":546},{"filename":"https://our-app-url.com/de/polyfills.e2fee55d4d4e7362.js","function":"p/<","in_app":true,"lineno":1,"colno":45268},{"filename":"https://our-app-url.com/de/polyfills.e2fee55d4d4e7362.js","function":"scheduleEventTask","in_app":true,"lineno":1,"colno":24848},{"filename":"https://our-app-url.com/de/polyfills.e2fee55d4d4e7362.js","function":"scheduleTask","in_app":true,"lineno":1,"colno":24457},{"filename":"https://our-app-url.com/de/polyfills.e2fee55d4d4e7362.js","function":"scheduleTask","in_app":true,"lineno":1,"colno":28225},{"filename":"https://our-app-url.com/de/polyfills.e2fee55d4d4e7362.js","function":"onScheduleTask","in_app":true,"lineno":1,"colno":25548},{"filename":"https://our-app-url.com/de/polyfills.e2fee55d4d4e7362.js","function":"scheduleTask","in_app":true,"lineno":1,"colno":28327},{"filename":"https://our-app-url.com/de/polyfills.e2fee55d4d4e7362.js","function":"De/o<","in_app":true,"lineno":1,"colno":43351},{"filename":"https://our-app-url.com/de/polyfills.e2fee55d4d4e7362.js","function":"Ae/t[s]","in_app":true,"lineno":1,"colno":34874},{"filename":"https://our-app-url.com/de/polyfills.e2fee55d4d4e7362.js","function":"EventListener.handleEvent*16518/</</</</<","in_app":true,"lineno":1,"colno":546},{"filename":"https://our-app-url.com/de/polyfills.e2fee55d4d4e7362.js","function":"le","in_app":true,"lineno":1,"colno":42844},{"filename":"https://our-app-url.com/de/polyfills.e2fee55d4d4e7362.js","function":"B","in_app":true,"lineno":1,"colno":42671},{"filename":"https://our-app-url.com/de/polyfills.e2fee55d4d4e7362.js","function":"v","in_app":true,"lineno":1,"colno":42253},{"filename":"https://our-app-url.com/de/polyfills.e2fee55d4d4e7362.js","function":"invokeTask","in_app":true,"lineno":1,"colno":29618},{"filename":"https://our-app-url.com/de/polyfills.e2fee55d4d4e7362.js","function":"runTask","in_app":true,"lineno":1,"colno":23892},{"filename":"https://our-app-url.com/de/polyfills.e2fee55d4d4e7362.js","function":"invokeTask","in_app":true,"lineno":1,"colno":28485},{"filename":"https://our-app-url.com/de/main.0580ea5ff873bfe6.js","function":"onInvokeTask","in_app":true,"lineno":3,"colno":1917496},{"filename":"https://our-app-url.com/de/polyfills.e2fee55d4d4e7362.js","function":"invokeTask","in_app":true,"lineno":1,"colno":28564},{"filename":"https://our-app-url.com/de/main.0580ea5ff873bfe6.js","function":"k/<","in_app":true,"lineno":3,"colno":2538415},{"filename":"https://our-app-url.com/de/main.0580ea5ff873bfe6.js","function":"G","in_app":true,"lineno":3,"colno":1876015},{"filename":"https://our-app-url.com/de/main.0580ea5ff873bfe6.js","function":"N","in_app":true,"lineno":3,"colno":1875844},{"filename":"https://our-app-url.com/de/main.0580ea5ff873bfe6.js","function":"Yo/<","in_app":true,"lineno":3,"colno":135872},{"filename":"https://our-app-url.com/de/main.0580ea5ff873bfe6.js","function":"crash","in_app":true,"lineno":3,"colno":141064}]},"mechanism":{"type":"angular","handled":false}}]},"level":"error","event_id":"394892e5d08540629f04030f086fd7a1","platform":"javascript","timestamp":1690547710.452,"environment":"nightly","release":"7.5.2","sdk":{"integrations":["InboundFilters","FunctionToString","Breadcrumbs","GlobalHandlers","LinkedErrors","Dedupe","HttpContext","BrowserTracing"],"name":"sentry.javascript.angular-ivy","version":"7.57.0","packages":[{"name":"npm:@sentry/angular-ivy","version":"7.57.0"}]},"contexts":{"angular":{"version":15}},"breadcrumbs":[{"timestamp":1690546950.965,"category":"console","data":{"arguments":["Matomo not found"],"logger":"console"},"level":"warning","message":"Matomo not found"},{"timestamp":1690546950.994,"category":"navigation","data":{"from":"/de/start","to":"/de/start"}},{"timestamp":1690546951.327,"category":"xhr","data":{"method":"GET","url":"/api/user","status_code":401},"type":"http"},{"timestamp":1690546951.333,"category":"xhr","data":{"method":"GET","url":"/api/quickstart/projects?showInvisible=false","status_code":200},"type":"http"},{"timestamp":1690546952.337,"category":"sentry.transaction","event_id":"08e72500c54f4a33882c3d75fc14d84f","message":"08e72500c54f4a33882c3d75fc14d84f"},{"timestamp":1690547707.477,"category":"ui.click","message":"mat-icon.mat-icon.notranslate.material-symbols-outlined.mat-icon-no-color"},{"timestamp":1690547709.084,"category":"ui.click","message":"div#mat-tab-label-0-3.mat-ripple.mat-tab-label.mat-focus-indicator.ng-star-inserted.cdk-focused.cdk-mouse-focused"},{"timestamp":1690547710.449,"category":"ui.click","message":"span.mat-button-wrapper"}],"request":{"url":"https://our-app-url.com/de/start","headers":{"User-Agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/115.0"}}}

For these large requests, the Sentry tunnel Docker container always returns an HTTP 400 error with an empty response body. Smaller requests work just fine.

I have tried sending this request directly to the Sentry tunnel container via curl and get the same error. When I send the exact same request and simply shorten the third line of the request body to {"exception":{"values":[]}}, the request goes through just fine (HTTP 200 OK). Therefore, I hightly suspect that the size of the request is the problem somehow.

Any idea what the issue might be? Thanks in advance!

creckord commented 11 months ago

Got the same problem. I tried with varying body lengths, and for me, requests up to Content-Length: 1000 go through, and everything from Content-Length: 1001 upwards gets rejected with a 400, with absolutely no indication in the tunnel logs.

creckord commented 11 months ago

Hm, this is interesting... I just rebuilt the sentry_tunnel docker image from source, following https://github.com/gbip/sentry_tunnel/blob/main/DEV.md, and see different behavior than from the docker hub image.

Requests of any (reasonable) length above 1000 bytes go through without a problem. Starting at a certain size (1024?) I see HTTP 100s before the final 200, but everything works as expected.

dominikks commented 10 months ago

This worked for me as well! After rebuilding the image from source as you described all my requests go through normally. I don't even get any HTTP 100 responses.

It seems like there is something going on with the current image in Docker Hub. I'll stick to using my custom built image for now. Thanks for the hint!

JoeyEamigh commented 10 months ago

For anyone who wants to use an image without this issue without building their own I threw one up at ghcr.io/joeyeamigh/sentry_tunnel/sentry_tunnel:latest. It is working for me!

gbip commented 6 months ago

I really should setup proper tooling for this project, however I did not have time to maintain this project recently...