Can't deploy - error "Your code size must be less than 200MB."

[tmilar]

I'm trying to deploy my app using sls deploy, but recently, I've started getting this error.

$ sls deploy --stage prod --debug
Initializing...
(node:41224) ExperimentalWarning: The fs.promises API is experimental
Action: "deploy" - Stage: "prod" - Org: "___" - App: "____" - Name: "______"

 Error: Your code size must be less than 200MB.  Try using Webpack, Parcel, or AWS Lambda layers to reduce your code size.
    at run (C:\Users\Tomas\AppData\Roaming\nvm\v10.16.0\node_modules\serverless\node_modules\@serverless\platform-client\src\instance.js:295:13)
    at process._tickCallback (internal/process/next_tick.js:68:7)

2s » Serverless » Your code size must be less than 200MB.  Try using Webpack, Parcel, or AWS Lambda layers to reduce your code size.

  Documentation: https://github.com/serverless/components
  Support: https://app.serverless.com/support
  Slack: https://www.serverless.com/slack/

My serverless.yml config is pretty simple:

app: ____
component: express@1.6.0
name: ______

inputs:
  src:
    src: ./ # this is the root of the app, ie. the folder containing node_modules
    exclude: # exclude the src files - not super important though
      - src

Serverless Framework version:

$ sls --version

serverless ⚡framework

components version: 2.34.9

Node version:

$ node --version
v10.16.0
$ npm --version
6.9.0

The weird part is that a week ago everything was working fine, and I didn't really have too much dependencies that would exceed the 200MB mark. For perspective, I've working in a different project with puppeteer which embeds a whole chrome instance that weighs about 60MB by itself.

This is my package.json:

{
  "name": "____",
  "version": "0.0.1",
  "description": "",
  "scripts": {
    "start": "cross-env NODE_ENV=development ts-node scripts/server.ts",
    "start:dev": "sls dev",
    "start:prod": "cross-env NODE_ENV=production npm run build && ts-node scripts/server.ts",
....
    "build": "npm run clean && tsc",
    "clean": "rimraf dist",
    "test": "cross-env DOTENV_CONFIG_PATH=.env.test mocha",
    "lint": "tsc --noEmit && eslint . --ext \"ts\"",
    "lint:fix": "npm run lint -- --fix",
    "deploy": "npm run build && npm run lint && sls deploy",
    "deploy:prod": "cross-env NODE_ENV=production npm run build && npm run lint && npm test && sls deploy --stage prod"
  },
  "engines": {
    "node": "^10.17.0"
  },
  "license": "ISC",
  "dependencies": {
    "body-parser": "1.19.0",
    "class-transformer": "0.3.1",
    "class-validator": "0.12.2",
    "cors": "2.8.5",
    "envalid": "6.0.2",
    "express": "4.17.1",
    "express-jwt": "6.0.0",
    "express-jwt-authz": "2.4.1",
    "jsonwebtoken": "8.5.1",
    "jwks-rsa": "1.9.0",
    "module-alias": "2.2.2",
    "morgan": "1.10.0",
    "node-fetch": "2.6.1",
    "passport-jwt": "4.0.0",
    "pg": "8.3.3",
    "pg-hstore": "2.3.3",
    "reflect-metadata": "0.1.13",
    "sequelize": "6.1.1",
    "sequelize-typescript": "2.0.0-beta.0",
    "swagger-ui-express": "4.1.4",
    "typescript-ioc": "3.2.2",
    "typescript-rest": "3.0.2",
    "typescript-rest-ioc": "1.0.0",
    "typescript-rest-swagger": "1.1.4"
  },
  "devDependencies": {
    "@types/chai": "4.2.12",
    "@types/cors": "2.8.7",
    "@types/express": "4.17.8",
    "@types/jsonwebtoken": "8.5.0",
    "@types/mocha": "8.0.3",
    "@types/module-alias": "2.0.0",
    "@types/morgan": "1.9.1",
    "@types/node": "14.6.4",
    "@types/node-fetch": "2.5.7",
    "@types/passport-jwt": "3.0.3",
    "@types/pg": "7.14.5",
    "@types/umzug": "2.2.3",
    "@types/validator": "13.1.0",
    "@typescript-eslint/eslint-plugin": "4.1.0",
    "@typescript-eslint/parser": "4.1.0",
    "chai": "4.2.0",
    "cross-env": "7.0.2",
    "dotenv": "8.2.0",
    "eslint": "7.8.1",
    "eslint-config-prettier": "6.11.0",
    "eslint-import-resolver-typescript": "2.3.0",
    "eslint-plugin-import": "2.22.0",
    "eslint-plugin-node": "11.1.0",
    "eslint-plugin-prettier": "3.1.4",
    "eslint-plugin-unicorn": "21.0.0",
    "mocha": "8.1.3",
    "nock": "13.0.4",
    "prettier": "2.1.1",
    "rimraf": "3.0.2",
    "ts-node": "9.0.0",
    "typescript": "4.0.2",
    "umzug": "2.3.0"
  }
}

Also, if I've tried building the app by preinstalling deps using npm ci but it still didn't work. The only workaround that did work was wiping out ALL my node_modules folder and then installing only production deps just like this:

npm i && npm run build && rimraf node_modules && npm i --production && sls deploy

However it has the downside that it's a pretty CPU and disk (and time) resource-intensive step.

Also, I remember that the previous setup of Serverless Framework for example for AWS Lambda functions, was smart enough to only package the actually used dependencies, not all the node_modules folder, unless you opted out by specifying excludeDevDependencies: false

So, what options do I have? Is this a bug that needs to be addressed on Serverless Framework side? Do I manually need to deplete and reinstall production-only dependencies every time? Or should I setup a webpack config just for this?

Thanks

[ariel-frischer]

I'm running into the same issue, except my node_modules folder is actually large at about 2.3GB due to ffmpeg use, and other media editing packages. I would love a feature that splits node_modules into layers automatically and adds them to the lambda function. I can't seem to figure this out even with more custom deployments, I have yet to find an article showing how to split the node_modules folder into multiple lambda layers. If anyone has insight on this please refer me!

[ariel-frischer]

Another idea, if this can be deployed inside a docker container that would solve the memory issues.

[pss4281]

Doesn't excludeDevDependencies: false still work with the latest serverless cli? I'd make a guess it might be only supported by non-component based services.

One possible solution to this might be installing dev dependencies globally or outside the lambda folder somewhere above the file tree in the same project folder