Request entity too large - config setting doesn't help

[danielcrisp]

I am getting the following error in the terminal when trying to run Glimpse on my project.

Error: request entity too large
at readStream ([path]/node_modules/raw-body/index.js:196:17)

This matches a 413 Payload too large error in a request to the message-ingress endpoint that I can see in the DevTools network panel.

I can see the original POST request in the Glimpse Client and the response body is indeed huge: 396989 bytes! This is a mock response and the size is correct - it's fake data.

I've tried setting the server.max.json.body.size configuration value to 500000 but it doesn't seem to affect the issue.

import * as glimpse from '@glimpse/glimpse';
// start glimpse
glimpse.init({
    server: {
        max: {
            json: {
                body: {
                    size: 135000 * 10
                }
            }
        }
    }
});

I also tried increasing the inspector.http.body.capture.maxsize value to 490000 but I was getting TypeScript errors because the inspector object is not a valid configuration option.

I have also tried changing these settings using the glimpse.config.json approach:

{
    "server.max.json.body.size": 500000,
    "inspector.http.body.capture.maxsize": 490000
}

Neither approach seems to help sadly.

All other (smaller) requests are working fine and I can see Glimpse working its magic nicely :-)

Other info

Node version: v6.7.0 OS: OSX 10.11.6 Browser: Chrome v59

This is running on a single page application so I have manually injected the HUD.

[mike-kaufman]

Thanks for opening the issue @danielcrisp. Are you running mongodb? With release 0.22.11 we started capturing data passed to mongo DB (documents on insert commands, queries objects and update objects). We have a bug where we're not truncating captured data & this can result in the errors you describe. We should have a fix out in a few weeks time.

[danielcrisp]

Hi @mike-kaufman, nope I'm not using mongodb - it's just a simple Express app that mocks our API endpoints with fake JSON data.

I'm familiar with the huge amounts of data that mongodb can expose for a single document though. Is the size the problem? Does your fix address the size for all requests or is it only specific to mongod?

[mike-kaufman]

interesting. What type of request is generating a body with size of 396989 bytes? Is this an XHR or fetch request from browser? Or from a node's HttpClient?

Does your fix address the size for all requests or is it only specific to mongod?

We don't have the fix yet, we just know this is a problem with the mongodb messages we're generating, and we're planning to fix it.

[mike-kaufman]

btw - if you have a simple repro case that we can run, that would be most helpful. Understand if your code isn't public or your environment is too hard to replicate.

[danielcrisp]

It's an XHR request from the browser, but it's the response body that is huge - not the request body.

Let me see if I can knock up a quick repo...

[danielcrisp]

Ok, try this: https://github.com/danielcrisp/glimpse-example

[mike-kaufman]

Thanks @danielcrisp - the repro you put up was perfect. The issue here is those setting values were never getting communicated to our "Browser Agent" - the piece of code that runs in your browser and gathers diagnostic data for what's happening client side. I've got a fix up to address this. We should have this release next week.

Once the fix is available, you'll need to update your repro since you're injecting the browser agent by hand. You'd need some code like the following, note the config-settings attribute and the values there. They would need to correspond to the values you're using for "server.max.json.body.size" and "inspector.http.body.capture.maxsize". Of course, if you're not hand-injecting the browser agent script, then this should "just work".

    <script src="http://localhost:3000/glimpse/agent/agent.js"
        id="__glimpse_browser_agent"
        data-message-ingress-template="http://localhost:3000/glimpse/message-ingress/"
        config-settings="{&quot;server.max.json.body.size&quot;:20000,&quot;inspector.http.body.capture.maxsize&quot;:10000}">
    </script>

Let us know if you have any ?s or feedback.

Thanks,

Mike

[danielcrisp]

Great! Sounds good - when the fix is released I'll update the repo to give it a go.

Thanks Mike,

Dan

[mike-kaufman]

ok, version 0.22.15 is now up on npm, and it has the fix for this. Please let us know if you have any other problems or feedback.