search

18F / team-api-server

Imports and publishes updates to Team API data
https://www.npmjs.com/package/team-api-server
Other
1 stars 4 forks source link

.about.yml imports appear to be failing #12

Open ertzeid opened 8 years ago

ertzeid commented 8 years ago

Seeing this error a lot in the logs:

Error: request entity too large
    at makeError (/usr/local/18f/nvm/versions/node/v5.7.0/lib/node_modules/team-api-server/node_modules/raw-body/index.js:184:15)
    at module.exports (/usr/local/18f/nvm/versions/node/v5.7.0/lib/node_modules/team-api-server/node_modules/raw-body/index.js:40:15)
    at read (/usr/local/18f/nvm/versions/node/v5.7.0/lib/node_modules/team-api-server/node_modules/body-parser/lib/read.js:62:3)
    at jsonParser (/usr/local/18f/nvm/versions/node/v5.7.0/lib/node_modules/team-api-server/node_modules/body-parser/lib/types/json.js:87:5)
    at Layer.handle [as handle_request] (/usr/local/18f/nvm/versions/node/v5.7.0/lib/node_modules/team-api-server/node_modules/express/lib/router/layer.js:82:5)
    at trim_prefix (/usr/local/18f/nvm/versions/node/v5.7.0/lib/node_modules/team-api-server/node_modules/express/lib/router/index.js:270:13)
    at /usr/local/18f/nvm/versions/node/v5.7.0/lib/node_modules/team-api-server/node_modules/express/lib/router/index.js:237:9
    at Function.proto.process_params (/usr/local/18f/nvm/versions/node/v5.7.0/lib/node_modules/team-api-server/node_modules/express/lib/router/index.js:312:12)
    at /usr/local/18f/nvm/versions/node/v5.7.0/lib/node_modules/team-api-server/node_modules/express/lib/router/index.js:228:12
    at Function.match_layer (/usr/local/18f/nvm/versions/node/v5.7.0/lib/node_modules/team-api-server/node_modules/express/lib/router/index.js:295:3)

And, probably relatedly, there is evidence that changes to project .about.yml files have not been getting imported.

mbland commented 8 years ago

This is (almost certainly) because the default payload size for the JSON body-parser middleware is 100kb, and some webhook payloads easily exceed that. I had to hack around this in 18F/pages-server by creating the 18F/hookshot fork (back when it was still part of 18F/pages). This is where the payloadLimit config parameter for 18F/pages-server comes from.

It may be worth investigating githooked, forked from the original hookshot library, as an alternative, as it allows for passing through configuration to increase the limit and is actively maintained.

afeld commented 8 years ago

I'm not seeing any errors in the logs or anything...any ideas of where I could track down the problem?

jseppi commented 8 years ago

PR #14 solves this by replacing hookshot with githooked, which allows specifying the max json body size. I set it to 5mb (https://github.com/18F/team-api-server/pull/14/files#diff-168726dbe96b3ce427e7fedce31bb0bcR54), the maximum size payload that GitHub will send via a webhook (ref https://developer.github.com/webhooks/#payloads)