search

agutoli / serverless-layers

Serverless.js plugin implementing AWS Lambda Layers, reducing lambda size, warm-up, and deployment time.
MIT License
229 stars 53 forks source link

Unzipped size must be smaller than 262144000 bytes #138

Closed ralcorta closed 9 months ago

ralcorta commented 1 year ago

When I use the package, I got this error:

Unzipped size must be smaller than 262144000 bytes

Script: sls deploy --verbose -s production

Complete Error

Deploying service-name to stage production (us-east-1)

Packaging
Using deployment bucket 'service-name.us-east-1.production.deploys'

[ LayersPlugin ]: => default
... ○ Downloading package.json from bucket...
... ○ package.json The specified key does not exist..
... ○  Changes identified ! Re-installing...
... ∅ [warning] ".npmrc" file does not exists!
... ∅ [warning] "yarn.lock" file does not exists!
... ○ npm install --production --only=prod 

... ○ Created layer package /project/path/.serverless/service-name-production-nodejs-default.zip (160.9 MB)
... ○ Uploading layer package...
... ○ OK...
ServerlessLayers error:
    Action: Lambda:publishLayerVersion
    Params: {"Content":{"S3Bucket":"service-name.us-east-1.production.deploys","S3Key":"serverless/service-name/production/layers/service-name-production-nodejs-default.zip"},"LayerName":"service-name-production-nodejs-default","Description":"created by serverless-layers plugin","CompatibleRuntimes":["nodejs"],"CompatibleArchitectures":["x86_64","arm64"]}
AWS SDK error:
    Unzipped size must be smaller than 262144000 bytes

⠦ Packaging (525s)

Serverles.yml config (I'm using .ts but for better understanding)

service: service-name
frameworkVersion: '3'
useDotenv: true
provider:
  name: aws
  runtime: nodejs16.x
  memorySize: 128
  deploymentBucket: service-name.us-east-1.dev.deploys
  profile: client-name
  iamRoleStatements:
    - Effect: Allow
      Action:
        - cognito-idp:AdminInitiateAuth
        - cognito-idp:AdminCreateUser
        - cognito-idp:AdminSetUserPassword
      Resource: '*'
    - Effect: Allow
      Action:
        - ssm:GetParameter
        - ssm:GetParameters
        - ssm:GetParametersByPath
      Resource: arn:aws:ssm:us-east-1:AWS-ID:parameter/client-name/dev/*
    - Effect: Allow
      Action:
        - ssm:DescribeParameters
      Resource: arn:aws:ssm:us-east-1:AWS-ID:*
    - Effect: Allow
      Action:
        - kms:Decrypt
      Resource: arn:aws:kms:us-east-1:AWS-ID:key/alias/aws/ssm
  apiGateway:
    minimumCompressionSize: 1024
    shouldStartNameWithService: true
  environment:
    AWS_NODEJS_CONNECTION_REUSE_ENABLED: '1'
    NODE_OPTIONS: '--enable-source-maps --stack-trace-limit=1000'
    DATABASE_URL: DATABASE_URL
    AWS_USER_POOL_ID: AWS_USER_POOL_ID
    AWS_CLIENT_ID: AWS_CLIENT_ID
  stage: dev
  region: us-east-1
  versionFunctions: true
  deploymentBucketObject:
    name: service-name.us-east-1.dev.deploys
plugins:
  - serverless-esbuild
  - serverless-prune-plugin
  - serverless-offline
  - serverless-deployment-bucket
  - serverless-layers
custom:
  env:
    production:
      DATABASE_URL: DATABASE_URL
      AWS_USER_POOL_ID: AWS_USER_POOL_ID
      AWS_CLIENT_ID: AWS_CLIENT_ID
    dev:
      DATABASE_URL: DATABASE_URL
      AWS_USER_POOL_ID: AWS_USER_POOL_ID
      AWS_CLIENT_ID: AWS_CLIENT_ID
    test:
      DATABASE_URL: DATABASE_URL
      AWS_USER_POOL_ID: AWS_USER_POOL_ID
      AWS_CLIENT_ID: AWS_CLIENT_ID
  esbuild:
    bundle: true
    minify: false
    sourcemap: true
    exclude:
      - aws-sdk
    target: node16
    define: {}
    platform: node
    concurrency: 10
    watch:
      pattern:
        - src/**/*.ts
      ignore:
        - temp/**/*
  autoswagger:
    title: Swagger title
    apiType: httpApi
    generateSwaggerOnDeploy: false
    typefiles:
      - ./src/core/types-dto.ts
    swaggerPath: swagger
    apiKeyHeaders:
      - Authorization
    useStage: false
    basePath: /dev
    schemes:
      - http
      - https
    excludeStages:
      - test
      - production
    useRedirectUI: true
  customDomain:
    domainName: api.client-name.com.ar
    basePath: ''
    createRoute53Record: true
  prune:
    automatic: true
    number: 0
  serverless-layers:
    retainVersions: 0
    common:
      dependenciesPath: ./package.json
package:
  individually: true
  patterns:
    - node_modules/.prisma/client/schema.prisma
    - node_modules/.prisma/client/libquery_engine-*
    - node_modules/.prisma/client/libquery_engine-rhel-*
    - node_modules/.prisma/client/*.node
    - '!node_modules/prisma/libquery_engine-*'
    - '!node_modules/@prisma/engines/**'
    - '!node_modules/**'
    - '!.git/**'
    - '!test/**'
    - '!e2e/**'
    - '!src/**'
  artifactsS3KeyDirname: serverless/service-name/dev/code-artifacts
functions:
  # ... More functions
 resources:
  Resources:
    CognitoUserPool:
      Type: AWS::Cognito::UserPool
      Properties:
        UserPoolName: dev-service-name-user-pool
        UsernameAttributes:
          - email
        AutoVerifiedAttributes:
          - email
    CognitoUserPoolClient:
      Type: AWS::Cognito::UserPoolClient
      Properties:
        ClientName: dev-service-name-user-pool-client
        UserPoolId:
          Ref: CognitoUserPool

package.json

{
    "name": "service-name",
    "version": "1.0.0",
    "description": "",
    "main": "serverless.ts",
    "engines": {
      "node": ">=16.15.0"
    },
    "scripts": {
      "local": "sls offline start",
      "package": "sls package",
      "login": "sls login",
      "deploy": "sls deploy --verbose -s production",
      "deploy:remove": "serverless remove -s production",
      "test": "cross-env NODE_ENV=dev mocha -r ts-node/register tests/*.test.ts --exit",
      "doctor": "sls doctor"
    },
    "keywords": [],
    "author": "",
    "license": "ISC",
    "devDependencies": {
      "@serverless/typescript": "^3.27.0",
      "@types/aws-lambda": "^8.10.110",
      "@types/http-errors": "^2.0.1",
      "@types/node": "^18.11.11",
      "esbuild": "^0.16.17",
      "nyc": "^15.1.0",
      "prettier": "2.8.3",
      "prettier-plugin-prisma": "^4.10.0",
      "prisma": "^4.9.0",
      "prisma-dbml-generator": "^0.10.0",
      "serverless": "^3.25.1",
      "serverless-auto-swagger": "^2.12.0",
      "serverless-deployment-bucket": "^1.6.0",
      "serverless-domain-manager": "^6.3.2",
      "serverless-esbuild": "^1.37.3",
      "serverless-layers": "^2.6.1",
      "serverless-offline": "^12.0.1",
      "serverless-webpack": "^5.11.0",
      "serverless-webpack-prisma": "^1.1.2",
      "ts-node": "^10.9.1",
      "tsconfig-paths": "^4.1.2",
      "typescript": "^4.9.5"
    },
    "dependencies": {
      "@aws-amplify/auth": "^5.1.8",
      "@aws-amplify/core": "^5.0.14",
      "@middy/core": "^4.0.2",
      "@middy/error-logger": "^4.0.2",
      "@middy/http-cors": "^4.0.2",
      "@middy/http-error-handler": "^4.0.2",
      "@middy/http-event-normalizer": "^4.0.2",
      "@middy/http-header-normalizer": "^4.0.2",
      "@middy/http-json-body-parser": "^4.0.2",
      "@middy/http-security-headers": "^4.0.2",
      "@middy/http-urlencode-body-parser": "^4.0.2",
      "@middy/http-urlencode-path-parser": "^4.0.2",
      "@prisma/client": "^4.9.0",
      "amazon-cognito-identity-js": "^6.1.2",
      "axios": "^1.3.2",
      "dotenv": "^16.0.3",
      "http-errors": "^2.0.0",
      "jsonwebtoken": "^9.0.0",
      "jwk-to-pem": "^2.0.5",
      "middy-middleware-class-validator": "^2.0.2",
      "middy-middleware-json-error-handler": "^3.0.0",
      "reflect-metadata": "^0.1.13",
      "serverless-dotenv-plugin": "^4.0.2",
      "serverless-prune-plugin": "^2.0.1",
      "tsyringe": "^4.7.0"
    }
  }

From what I can see the layer formed by node_modules does not weigh more than 250 MB. According to the zip this is the size of 160.9 MB

thordickinson commented 1 year ago

I've facing the same error, do anyone know how to solve this? I'm just starting my project, so its size is pretty small.

thordickinson commented 1 year ago

Somehow I was able to make it work by removing my current stack sls remove and deploying it again, hope it could give a clue on the root cause.

jonathankeebler commented 1 year ago

Here's another solution that worked for me:

1) before adding the serverless-layer plugin to your serverless.yml, add the following to deploy without the node_modules, and deploy it (the lambdas will not function until you complete step 2):

package:
  patterns:
    - "!node_modules/**"

2) Add the serverless-layer plugin back in and deploy again:

plugins:
  - serverless-layers
mayank-ssense commented 11 months ago

If we try : https://www.serverless.com/framework/docs/providers/aws/guide/deploying 

provider:
  name: aws
  deploymentMethod: direct
  deploymentPrefix: test

Is that possible solution ?? Like change default artifacts location ?? Is that workaround solution ??