Multipart form data file upload runs into size limit issue

[girishbr]

I have this file upload api (multipart/form-data) and I run into a http 413 when I upload a 3.2 mb json file.

{
  "expected": 3207276,
  "length": 3207276,
  "limit": 102400,
  "status": 413,
  "statusCode": 413,
  "message": "request entity too large",
  "type": "entity.too.large"
}

Swagger-node uses multer internally. And multer doesn't really impose any restrictions. https://github.com/expressjs/multer#limits - it's infinity by default.

and neither do you, in swagger tools, https://github.com/apigee-127/swagger-tools/blob/master/middleware/swagger-metadata.js#L41, impose any limits.

I run express 4, so body parser is ruled out.

I am running on localhost, no nginx or any other proxy in between.

[ramden]

+1

[girishbr]

@ramden downgrade to express 3. Everything works like a charm! Weird though.

[NickBlow]

+1 any solutions so far?

[NickBlow]

I got it working by injecting my own version of multer, in case anyone has the same issue and wants to stick with express 4. Just have a look at the multer docs for whatever settings you want to use, this is the code I used that stores files up to 52mb in memory and has a single file being uploaded called 'file'.

import SwaggerExpress from 'swagger-express-mw';
import express from 'express';
import multer from 'multer';
...
let app = express();

SwaggerExpress.create(config, (err, swaggerExpress) => {
  if (err) {
    throw err;
  }
  let storage = multer.memoryStorage(); //you might need to change this, check multer docs

  let mult = multer({ //you might need to change this, check multer docs
    storage: storage,
    limits: {
      fileSize: 52428800
    }
  }).fields([{name: "file"}]);

  app.use(mult);

  // install middleware
  swaggerExpress.register(app);

  var port = process.env.PORT || 10010;
  app.listen(port);
});

[Boelensman1]

+1 The above solution does for some reason not work for me.

Am trying to find out why the default setup does not work in express 4

[johnmurphy01]

+1 Any updates on this? I'm running into the same problem. @NickBlow's solution isn't working at the moment but am still trying to figure it out.

[wompeter]

+1

[dionysius]

I also stumbled also on this issue, since there are a lot of different reasons when you google about this, it's difficult to find out if it really affects you. Thus I'll post here my versions and my usage behavior:

$ npm list swagger-tools
└─┬ swagger-express-mw@0.1.0
  └─┬ swagger-node-runner@0.5.13
    └── swagger-tools@0.9.16 

I use json in the request body, and one of the fields can get very large. First I found out, that the limit happens when the content-length is about ~100000 (since 106k is too large and 80k is fine)

The mult fix above also didn't help, so I scratched my head around the source and found the location in node_modules/swagger-tools/middleware/swagger-metadata.js:48

var jsonBodyParser = bp.json(); // original
var jsonBodyParser = bp.json({limit: '10mb'}); // this fixes it

The big question is now, how can I overwrite that? since this is just a hacky local fix...

Edit: just found the documentation in body-parser about the 100kb default limit

Edit 2: I tried to just call body-parser first in my app, and it worked! add this to app.js

var bp = require('body-parser'); // add this somewhere in your imports

var app = express(); // search for this (should be existing)
app.use(bodyParser.json({limit: '10mb'})) // and add this

That probably means, for different bodyParser stuff (text, urlencode) you may be able to set the limit on the same way

[johnmurphy01]

I had to add the following yaml file to my project:

# swagger configuration file

# values in the swagger hash are system configuration for swagger-node
swagger:

  fittingsDirs: [ api/fittings ]
  defaultPipe: null
  swaggerControllerPipe: swagger_controllers  # defines the standard processing pipe for controllers

  # values defined in the bagpipes key are the bagpipes pipes and fittings definitions
  # (see https://github.com/apigee-127/bagpipes)
  bagpipes:

    _router:
      name: swagger_router
      mockMode: false
      mockControllersDirs: [ api/mocks ]
      controllersDirs: [ api/controllers ]

    _swagger_validate:
      name: swagger_validator
      validateResponse: true

    swagger_params_parser:
      name: swagger_params_parser
      jsonOptions:
        limit: '5mb'

    # pipe for all swagger-node controllers
    swagger_controllers:
      - cors
      - swagger_security
      - swagger_params_parser
      - _swagger_validate
      - express_compatibility
      - _router

    # pipe to serve swagger (endpoint is in swagger.yaml)
    swagger_raw:
      name: swagger_raw

# any other values in this file are just loaded into the config for application access...

Pay attention particularly to the jsonOptions setting.

I created a swagger folder in my project and underneath I had my default.yaml file with the above and my swagger.json file with my definitions. I then removed bodyParser because I was still using it but it was not needed since swagger handles it for you.

[ais-one]

This one seems to work for me

https://github.com/swagger-api/swagger-node/issues/338#issuecomment-241029655

It throws 500 error and spits out HTML though, need to find a way to make it more elegant...

[chriszrc]

@dionysius that was totally it, even though swagger has it's own config for body-parser, by default my app (and I assume most others), have something like this in the default express config:

constructor() {
    const root = path.normalize(__dirname + '/../..');
    app.set('appPath', root + 'client');
    app.use(bodyParser.json());
    app.use(bodyParser.urlencoded({ extended: true }));
    app.use(cookieParser(process.env.SESSION_SECRET));
    app.use(express.static(`${root}/public`));
  }

adding a larger limit for body-parser totally fixed this problem:

app.use(bodyParser.json({limit: '6mb'}));

Thanks!