search

floydspace / serverless-esbuild

💨 A Serverless framework plugin to bundle JavaScript and TypeScript with extremely fast esbuild
MIT License
445 stars 138 forks source link

slow performance when creating multiple zip files. #495

Open JamesKyburz opened 12 months ago

JamesKyburz commented 12 months ago

Slow zip performance

When creating multiple zip files there are performance issues, some blocking the event loop.

I have a private repository with a serverless stack containing 8 lambdas which takes 70s to package (and alot more in GitHub actions...)

To Reproduce

Create a serverless.yml with multiple functions using similar dependencies.

Using the config

'use strict'

module.exports = function esmConfig() {
  const nodeMajorVersion = process.version.match(/v([\d]+)/)[1]
  return {
    bundle: true,
    minify: true,
    format: 'esm',
    outputFileExtension: '.mjs',
    sourcemap: true,
    target: `node${nodeMajorVersion}`,
    exclude: ['@aws-sdk/*'],
    define: {
      'require.resolve': 'undefined'
    },
    platform: 'node',
    banner: {
      js: `
import { createRequire } from 'node:module'
import { dirname } from 'node:path'
import { fileURLToPath } from 'node:url'

const require = createRequire(import.meta.url)
const __filename = fileURLToPath(import.meta.url)
const __dirname = dirname(__filename)
`,
    },
    watch: {
      pattern: './functions/**',
      ignore: ['.serverless/**/*', '.build']
    }
  }
}

Expected behavior A clear and concise description of what you expected to happen.

A large list of exclude files (9K+), blocking fs operations including the zip function causes slowdown.

Screenshots or Logs Adding lots of console logs, and a timer interval with 3s delay to show event loop blockages.

After a few tweaks of making sync fs async the time was 70s zipping 8 functions.

When loop is output is shows the delay since the last output (this should be around 3000ms depending on of the event loop was blocked or not)

Packaging something for stage local (us-east-1)
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
-bundle
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
+bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/create-project/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/create-project"}
+bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/initial-bom/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/initial-bom"}
+bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/project/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/project"}
+bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/projects/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/projects"}
+bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/update-row/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/update-row"}
+bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/select-product/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/select-product"}
+bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/update-project/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/update-project"}
+bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/add-material/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/add-material"}
-bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/add-material/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/add-material"}
+bundle# pkg.rebuild
-bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/update-row/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/update-row"}
+bundle# pkg.rebuild
-bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/create-project/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/create-project"}
+bundle# pkg.rebuild
-bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/project/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/project"}
+bundle# pkg.rebuild
-bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/select-product/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/select-product"}
+bundle# pkg.rebuild
-bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/initial-bom/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/initial-bom"}
+bundle# pkg.rebuild
-bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/projects/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/projects"}
+bundle# pkg.rebuild
-bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/update-project/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/update-project"}
+bundle# pkg.rebuild
-bundle# pkg.rebuild
-bundle# pkg.rebuild
-bundle# pkg.rebuild
-bundle# pkg.rebuild
-bundle# pkg.rebuild
-bundle# pkg.rebuild
-bundle# pkg.rebuild
-bundle# pkg.rebuild
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
+bundle
-packExternalModules
pack external [9be10826-915b-4379-94fb-d155330fddf4]
findUpIO {"name":"package.json","directory":"/Users/x/Documents/src/private-stack/packages/project"}
findUpIO {"name":"package.json","directory":"/Users/x/Documents/src/private-stack/packages"}
findUpIO {"name":"package.json","directory":"/Users/x/Documents/src/private-stack"}
external options {
  "exclude": [],
  "external": [],
  "upperPackageJson": "/Users/x/Documents/src/private-stack"
}
pack external [9be10826-915b-4379-94fb-d155330fddf4] took 0 ms
+packExternalModules
-copyExtras
loop! [3000]ms {"handles":4,"requests":10}
loop! [3000]ms {"handles":4,"requests":1}
+copyExtras
-pack
pack[2fde070a-92f8-4e0a-874e-686efab601d3]
pack[2fde070a-92f8-4e0a-874e-686efab601d3] step 1 took 64 ms
pack[2fde070a-92f8-4e0a-874e-686efab601d3] build options {
  "concurrency": null,
  "zipConcurrency": null,
  "bundle": true,
  "target": "node18",
  "external": [],
  "exclude": [
    "@aws-sdk/*"
  ],
  "nativeZip": false,
  "packager": "npm",
  "packagerOptions": {
    "noInstall": false,
    "ignoreLockfile": false
  },
  "installExtraArgs": [],
  "watch": {
    "pattern": "./functions/**",
    "ignore": [
      ".serverless/**/*",
      ".build"
    ],
    "chokidar": {
      "ignoreInitial": true
    }
  },
  "keepOutputDirectory": false,
  "platform": "node",
  "outputFileExtension": ".mjs",
  "skipBuild": false,
  "skipBuildExcludeFns": [],
  "minify": true,
  "format": "esm",
  "sourcemap": true,
  "define": {
    "require.resolve": "undefined"
  },
  "banner": {
    "js": "\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"
  }
}
pack[2fde070a-92f8-4e0a-874e-686efab601d3] step 2 took 65 ms
pack[2fde070a-92f8-4e0a-874e-686efab601d3] step 3 took 65 ms
zip mapper[d942aec8-c067-4ea7-98eb-524c5c079db0]
zip mapper[df95d021-7c20-4a3d-a818-17ccf7d71ac0]
zip mapper[80bd9f0e-9765-484b-a3d4-37db70f96d24]
zip mapper[d2272b7d-3d98-4583-be29-80cae63c8707]
zip mapper[049b790e-1c5a-49e2-8b8c-e36e5e1a753a]
zip mapper[dce2c865-bcd0-414f-9693-6c9a39f6d9bf]
zip mapper[a90cbd59-796e-4871-9e29-e01aab0a968b]
zip mapper[eeaaa25d-4194-44eb-b264-76db7d6cd9d7]
zipMapper[80bd9f0e-9765-484b-a3d4-37db70f96d24] 164 so far took 252 ms
zipMapper[df95d021-7c20-4a3d-a818-17ccf7d71ac0] 164 so far took 255 ms
zipMapper[049b790e-1c5a-49e2-8b8c-e36e5e1a753a] 164 so far took 257 ms
zipMapper[d2272b7d-3d98-4583-be29-80cae63c8707] 164 so far took 259 ms
zipMapper[d942aec8-c067-4ea7-98eb-524c5c079db0] 164 so far took 262 ms
zipMapper[dce2c865-bcd0-414f-9693-6c9a39f6d9bf] 164 so far took 263 ms
zipMapper[a90cbd59-796e-4871-9e29-e01aab0a968b] 164 so far took 265 ms
zipMapper[eeaaa25d-4194-44eb-b264-76db7d6cd9d7] 164 so far took 272 ms
loop! [3006]ms {"handles":4,"requests":9280}
loop! [3123]ms {"handles":4,"requests":9280}
loop! [3029]ms {"handles":4,"requests":9280}
loop! [3015]ms {"handles":4,"requests":9280}
loop! [3078]ms {"handles":4,"requests":9280}
loop! [3050]ms {"handles":4,"requests":9280}
loop! [3031]ms {"handles":4,"requests":9280}
loop! [3053]ms {"handles":4,"requests":9280}
exclude calculation [80bd9f0e-9765-484b-a3d4-37db70f96d24] took 25 seconds
exclude file count [80bd9f0e-9765-484b-a3d4-37db70f96d24] 9576
zipMapper[80bd9f0e-9765-484b-a3d4-37db70f96d24] 175 so far took 25 seconds
zipMapper[80bd9f0e-9765-484b-a3d4-37db70f96d24] 182 so far took 25 seconds
zipMapper[80bd9f0e-9765-484b-a3d4-37db70f96d24] 185 so far took 25 seconds
zipMapper[80bd9f0e-9765-484b-a3d4-37db70f96d24] 203 so far took 30 seconds
zipping[80bd9f0e-9765-484b-a3d4-37db70f96d24]
about to zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/project.zip
exclude calculation [049b790e-1c5a-49e2-8b8c-e36e5e1a753a] took 30 seconds
exclude file count [049b790e-1c5a-49e2-8b8c-e36e5e1a753a] 9576
zipMapper[049b790e-1c5a-49e2-8b8c-e36e5e1a753a] 175 so far took 30 seconds
zipMapper[049b790e-1c5a-49e2-8b8c-e36e5e1a753a] 182 so far took 30 seconds
zipMapper[049b790e-1c5a-49e2-8b8c-e36e5e1a753a] 185 so far took 30 seconds
zipMapper[049b790e-1c5a-49e2-8b8c-e36e5e1a753a] 203 so far took 35 seconds
zipping[049b790e-1c5a-49e2-8b8c-e36e5e1a753a]
about to zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/update-row.zip
exclude calculation [d2272b7d-3d98-4583-be29-80cae63c8707] took 35 seconds
exclude file count [d2272b7d-3d98-4583-be29-80cae63c8707] 9576
zipMapper[d2272b7d-3d98-4583-be29-80cae63c8707] 175 so far took 35 seconds
zipMapper[d2272b7d-3d98-4583-be29-80cae63c8707] 182 so far took 35 seconds
zipMapper[d2272b7d-3d98-4583-be29-80cae63c8707] 185 so far took 35 seconds
zipMapper[d2272b7d-3d98-4583-be29-80cae63c8707] 203 so far took 40 seconds
zipping[d2272b7d-3d98-4583-be29-80cae63c8707]
about to zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/projects.zip
exclude calculation [d942aec8-c067-4ea7-98eb-524c5c079db0] took 40 seconds
exclude file count [d942aec8-c067-4ea7-98eb-524c5c079db0] 9576
zipMapper[d942aec8-c067-4ea7-98eb-524c5c079db0] 175 so far took 40 seconds
zipMapper[d942aec8-c067-4ea7-98eb-524c5c079db0] 182 so far took 40 seconds
zipMapper[d942aec8-c067-4ea7-98eb-524c5c079db0] 185 so far took 40 seconds
zipMapper[d942aec8-c067-4ea7-98eb-524c5c079db0] 203 so far took 45 seconds
zipping[d942aec8-c067-4ea7-98eb-524c5c079db0]
about to zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/create-project.zip
exclude calculation [dce2c865-bcd0-414f-9693-6c9a39f6d9bf] took 44 seconds
exclude file count [dce2c865-bcd0-414f-9693-6c9a39f6d9bf] 9576
zipMapper[dce2c865-bcd0-414f-9693-6c9a39f6d9bf] 175 so far took 45 seconds
zipMapper[dce2c865-bcd0-414f-9693-6c9a39f6d9bf] 182 so far took 45 seconds
zipMapper[dce2c865-bcd0-414f-9693-6c9a39f6d9bf] 185 so far took 45 seconds
zipMapper[dce2c865-bcd0-414f-9693-6c9a39f6d9bf] 203 so far took 49 seconds
zipping[dce2c865-bcd0-414f-9693-6c9a39f6d9bf]
about to zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/select-product.zip
loop! [25451]ms {"handles":4,"requests":1030}
exclude calculation [df95d021-7c20-4a3d-a818-17ccf7d71ac0] took 49 seconds
exclude file count [df95d021-7c20-4a3d-a818-17ccf7d71ac0] 9576
zipMapper[df95d021-7c20-4a3d-a818-17ccf7d71ac0] 175 so far took 49 seconds
zipMapper[df95d021-7c20-4a3d-a818-17ccf7d71ac0] 182 so far took 49 seconds
zipMapper[df95d021-7c20-4a3d-a818-17ccf7d71ac0] 185 so far took 49 seconds
zipMapper[df95d021-7c20-4a3d-a818-17ccf7d71ac0] 203 so far took 54 seconds
zipping[df95d021-7c20-4a3d-a818-17ccf7d71ac0]
about to zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/initial-bom.zip
exclude calculation [eeaaa25d-4194-44eb-b264-76db7d6cd9d7] took 54 seconds
exclude file count [eeaaa25d-4194-44eb-b264-76db7d6cd9d7] 9576
zipMapper[eeaaa25d-4194-44eb-b264-76db7d6cd9d7] 175 so far took 54 seconds
zipMapper[eeaaa25d-4194-44eb-b264-76db7d6cd9d7] 182 so far took 54 seconds
zipMapper[eeaaa25d-4194-44eb-b264-76db7d6cd9d7] 185 so far took 54 seconds
zipMapper[eeaaa25d-4194-44eb-b264-76db7d6cd9d7] 203 so far took 59 seconds
zipping[eeaaa25d-4194-44eb-b264-76db7d6cd9d7]
about to zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/add-material.zip
loop! [9521]ms {"handles":4,"requests":14}
exclude calculation [a90cbd59-796e-4871-9e29-e01aab0a968b] took 58 seconds
exclude file count [a90cbd59-796e-4871-9e29-e01aab0a968b] 9576
zipMapper[a90cbd59-796e-4871-9e29-e01aab0a968b] 175 so far took 59 seconds
zipMapper[a90cbd59-796e-4871-9e29-e01aab0a968b] 182 so far took 59 seconds
zipMapper[a90cbd59-796e-4871-9e29-e01aab0a968b] 185 so far took 59 seconds
zipMapper[a90cbd59-796e-4871-9e29-e01aab0a968b] 203 so far took 1 minute
zipping[a90cbd59-796e-4871-9e29-e01aab0a968b]
about to zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/update-project.zip
loop! [4697]ms {"handles":4,"requests":10}
zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/update-row.zip: 28.427s
zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/project.zip: 33.363s
zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/create-project.zip: 18.937s
zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/select-product.zip: 14.258s
zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/projects.zip: 23.702s
zipped[049b790e-1c5a-49e2-8b8c-e36e5e1a753a] took 28 seconds
zip mapper[049b790e-1c5a-49e2-8b8c-e36e5e1a753a] took 1 minute
zipped[80bd9f0e-9765-484b-a3d4-37db70f96d24] took 33 seconds
zip mapper[80bd9f0e-9765-484b-a3d4-37db70f96d24] took 1 minute
zipped[d942aec8-c067-4ea7-98eb-524c5c079db0] took 19 seconds
zip mapper[d942aec8-c067-4ea7-98eb-524c5c079db0] took 1 minute
zipped[dce2c865-bcd0-414f-9693-6c9a39f6d9bf] took 14 seconds
zip mapper[dce2c865-bcd0-414f-9693-6c9a39f6d9bf] took 1 minute
zipped[d2272b7d-3d98-4583-be29-80cae63c8707] took 24 seconds
zip mapper[d2272b7d-3d98-4583-be29-80cae63c8707] took 1 minute
zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/update-project.zip: 33.987ms
zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/add-material.zip: 4.731s
zipped[a90cbd59-796e-4871-9e29-e01aab0a968b] took 34 ms
zip mapper[a90cbd59-796e-4871-9e29-e01aab0a968b] took 1 minute
zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/initial-bom.zip: 9.425s
zipped[eeaaa25d-4194-44eb-b264-76db7d6cd9d7] took 5 seconds
zip mapper[eeaaa25d-4194-44eb-b264-76db7d6cd9d7] took 1 minute
zipped[df95d021-7c20-4a3d-a818-17ccf7d71ac0] took 9 seconds
zip mapper[df95d021-7c20-4a3d-a818-17ccf7d71ac0] took 1 minute
+pack

✔ Service packaged (70s)
loop! [3001]ms {"handles":3,"requests":0}

Changing the exclude to be an empty array instead of 9k+ files made a massive difference, this time sls package took under 10s.

The changed line was this line

Packaging something for stage local (us-east-1)
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
-bundle
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
+bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/create-project/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/create-project"}
+bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/initial-bom/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/initial-bom"}
+bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/project/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/project"}
+bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/projects/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/projects"}
+bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/update-row/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/update-row"}
+bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/select-product/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/select-product"}
+bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/update-project/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/update-project"}
+bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/add-material/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/add-material"}
-bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/add-material/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/add-material"}
+bundle# pkg.rebuild
-bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/create-project/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/create-project"}
+bundle# pkg.rebuild
-bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/update-row/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/update-row"}
+bundle# pkg.rebuild
-bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/initial-bom/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/initial-bom"}
+bundle# pkg.rebuild
-bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/projects/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/projects"}
+bundle# pkg.rebuild
-bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/project/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/project"}
+bundle# pkg.rebuild
-bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/select-product/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/select-product"}
+bundle# pkg.rebuild
-bundle# pkg.context {"bundle":true,"target":"node18","external":["@aws-sdk/*"],"platform":"node","minify":true,"format":"esm","sourcemap":true,"define":{"require.resolve":"undefined"},"banner":{"js":"\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"},"plugins":[],"outExtension":{".js":".mjs"},"entryPoints":["functions/update-project/index.ts"],"outdir":"/Users/x/Documents/src/private-stack/packages/project/.esbuild/.build/functions/update-project"}
+bundle# pkg.rebuild
-bundle# pkg.rebuild
-bundle# pkg.rebuild
-bundle# pkg.rebuild
-bundle# pkg.rebuild
-bundle# pkg.rebuild
-bundle# pkg.rebuild
-bundle# pkg.rebuild
-bundle# pkg.rebuild
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
about to perform blocking existsSync
+bundle
-packExternalModules
pack external [7edb1fe2-d90b-4989-b5e3-8402c7b7ebde]
findUpIO {"name":"package.json","directory":"/Users/x/Documents/src/private-stack/packages/project"}
findUpIO {"name":"package.json","directory":"/Users/x/Documents/src/private-stack/packages"}
findUpIO {"name":"package.json","directory":"/Users/x/Documents/src/private-stack"}
external options {
  "exclude": [],
  "external": [],
  "upperPackageJson": "/Users/x/Documents/src/private-stack"
}
pack external [7edb1fe2-d90b-4989-b5e3-8402c7b7ebde] took 1 ms
+packExternalModules
-copyExtras
loop! [3000]ms {"handles":4,"requests":10}
loop! [3000]ms {"handles":4,"requests":1}
+copyExtras
-pack
pack[25373c50-1cb7-4914-adf5-a8ab8ff77065]
pack[25373c50-1cb7-4914-adf5-a8ab8ff77065] step 1 took 66 ms
pack[25373c50-1cb7-4914-adf5-a8ab8ff77065] build options {
  "concurrency": null,
  "zipConcurrency": null,
  "bundle": true,
  "target": "node18",
  "external": [],
  "exclude": [
    "@aws-sdk/*"
  ],
  "nativeZip": false,
  "packager": "npm",
  "packagerOptions": {
    "noInstall": false,
    "ignoreLockfile": false
  },
  "installExtraArgs": [],
  "watch": {
    "pattern": "./functions/**",
    "ignore": [
      ".serverless/**/*",
      ".build"
    ],
    "chokidar": {
      "ignoreInitial": true
    }
  },
  "keepOutputDirectory": false,
  "platform": "node",
  "outputFileExtension": ".mjs",
  "skipBuild": false,
  "skipBuildExcludeFns": [],
  "minify": true,
  "format": "esm",
  "sourcemap": true,
  "define": {
    "require.resolve": "undefined"
  },
  "banner": {
    "js": "\nimport { createRequire } from 'node:module'\nimport { dirname } from 'node:path'\nimport { fileURLToPath } from 'node:url'\n\nconst require = createRequire(import.meta.url)\nconst __filename = fileURLToPath(import.meta.url)\nconst __dirname = dirname(__filename)\n"
  }
}
pack[25373c50-1cb7-4914-adf5-a8ab8ff77065] step 2 took 67 ms
pack[25373c50-1cb7-4914-adf5-a8ab8ff77065] step 3 took 67 ms
zip mapper[2fea2fa1-1e0d-4096-b242-bf4bc925397f]
zip mapper[4750e4dd-584f-4fcb-8896-8b0bf0d8aa5c]
zip mapper[4909a123-683e-4c6c-ac32-e479cd80aea4]
zip mapper[5e64fb4c-7668-4e7b-9ef9-06d17591ee30]
zip mapper[8f27abb4-4b27-4a8c-a286-dd71d6a1b423]
zip mapper[0082936c-58d4-4d4d-809c-d7a289ad91d4]
zip mapper[8f8f9792-f8f1-4aa1-99d7-960bcd397f9b]
zip mapper[1620f390-5a22-483b-8454-de6a4cccda46]
zipMapper[4750e4dd-584f-4fcb-8896-8b0bf0d8aa5c] 164 so far took 251 ms
exclude calculation [4750e4dd-584f-4fcb-8896-8b0bf0d8aa5c] took 0 ms
exclude file count [4750e4dd-584f-4fcb-8896-8b0bf0d8aa5c] 0
zipMapper[4750e4dd-584f-4fcb-8896-8b0bf0d8aa5c] 175 so far took 251 ms
zipMapper[4750e4dd-584f-4fcb-8896-8b0bf0d8aa5c] 182 so far took 251 ms
zipMapper[4750e4dd-584f-4fcb-8896-8b0bf0d8aa5c] 185 so far took 251 ms
zipMapper[4750e4dd-584f-4fcb-8896-8b0bf0d8aa5c] 203 so far took 256 ms
zipping[4750e4dd-584f-4fcb-8896-8b0bf0d8aa5c]
about to zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/initial-bom.zip
zipMapper[5e64fb4c-7668-4e7b-9ef9-06d17591ee30] 164 so far took 256 ms
exclude calculation [5e64fb4c-7668-4e7b-9ef9-06d17591ee30] took 0 ms
exclude file count [5e64fb4c-7668-4e7b-9ef9-06d17591ee30] 0
zipMapper[5e64fb4c-7668-4e7b-9ef9-06d17591ee30] 175 so far took 257 ms
zipMapper[5e64fb4c-7668-4e7b-9ef9-06d17591ee30] 182 so far took 257 ms
zipMapper[5e64fb4c-7668-4e7b-9ef9-06d17591ee30] 185 so far took 257 ms
zipMapper[5e64fb4c-7668-4e7b-9ef9-06d17591ee30] 203 so far took 261 ms
zipping[5e64fb4c-7668-4e7b-9ef9-06d17591ee30]
about to zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/projects.zip
zipMapper[4909a123-683e-4c6c-ac32-e479cd80aea4] 164 so far took 261 ms
exclude calculation [4909a123-683e-4c6c-ac32-e479cd80aea4] took 0 ms
exclude file count [4909a123-683e-4c6c-ac32-e479cd80aea4] 0
zipMapper[4909a123-683e-4c6c-ac32-e479cd80aea4] 175 so far took 261 ms
zipMapper[4909a123-683e-4c6c-ac32-e479cd80aea4] 182 so far took 261 ms
zipMapper[4909a123-683e-4c6c-ac32-e479cd80aea4] 185 so far took 261 ms
zipMapper[4909a123-683e-4c6c-ac32-e479cd80aea4] 203 so far took 265 ms
zipping[4909a123-683e-4c6c-ac32-e479cd80aea4]
about to zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/project.zip
zipMapper[0082936c-58d4-4d4d-809c-d7a289ad91d4] 164 so far took 266 ms
exclude calculation [0082936c-58d4-4d4d-809c-d7a289ad91d4] took 0 ms
exclude file count [0082936c-58d4-4d4d-809c-d7a289ad91d4] 0
zipMapper[0082936c-58d4-4d4d-809c-d7a289ad91d4] 175 so far took 266 ms
zipMapper[0082936c-58d4-4d4d-809c-d7a289ad91d4] 182 so far took 266 ms
zipMapper[0082936c-58d4-4d4d-809c-d7a289ad91d4] 185 so far took 266 ms
zipMapper[0082936c-58d4-4d4d-809c-d7a289ad91d4] 203 so far took 269 ms
zipping[0082936c-58d4-4d4d-809c-d7a289ad91d4]
about to zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/select-product.zip
zipMapper[8f27abb4-4b27-4a8c-a286-dd71d6a1b423] 164 so far took 270 ms
exclude calculation [8f27abb4-4b27-4a8c-a286-dd71d6a1b423] took 0 ms
exclude file count [8f27abb4-4b27-4a8c-a286-dd71d6a1b423] 0
zipMapper[8f27abb4-4b27-4a8c-a286-dd71d6a1b423] 175 so far took 270 ms
zipMapper[8f27abb4-4b27-4a8c-a286-dd71d6a1b423] 182 so far took 270 ms
zipMapper[8f27abb4-4b27-4a8c-a286-dd71d6a1b423] 185 so far took 270 ms
zipMapper[8f27abb4-4b27-4a8c-a286-dd71d6a1b423] 203 so far took 273 ms
zipping[8f27abb4-4b27-4a8c-a286-dd71d6a1b423]
about to zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/update-row.zip
zipMapper[2fea2fa1-1e0d-4096-b242-bf4bc925397f] 164 so far took 275 ms
exclude calculation [2fea2fa1-1e0d-4096-b242-bf4bc925397f] took 0 ms
exclude file count [2fea2fa1-1e0d-4096-b242-bf4bc925397f] 0
zipMapper[2fea2fa1-1e0d-4096-b242-bf4bc925397f] 175 so far took 275 ms
zipMapper[2fea2fa1-1e0d-4096-b242-bf4bc925397f] 182 so far took 275 ms
zipMapper[2fea2fa1-1e0d-4096-b242-bf4bc925397f] 185 so far took 275 ms
zipMapper[2fea2fa1-1e0d-4096-b242-bf4bc925397f] 203 so far took 278 ms
zipping[2fea2fa1-1e0d-4096-b242-bf4bc925397f]
about to zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/create-project.zip
zipMapper[1620f390-5a22-483b-8454-de6a4cccda46] 164 so far took 278 ms
exclude calculation [1620f390-5a22-483b-8454-de6a4cccda46] took 0 ms
exclude file count [1620f390-5a22-483b-8454-de6a4cccda46] 0
zipMapper[1620f390-5a22-483b-8454-de6a4cccda46] 175 so far took 278 ms
zipMapper[1620f390-5a22-483b-8454-de6a4cccda46] 182 so far took 279 ms
zipMapper[1620f390-5a22-483b-8454-de6a4cccda46] 185 so far took 279 ms
zipMapper[1620f390-5a22-483b-8454-de6a4cccda46] 203 so far took 282 ms
zipping[1620f390-5a22-483b-8454-de6a4cccda46]
about to zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/add-material.zip
zipMapper[8f8f9792-f8f1-4aa1-99d7-960bcd397f9b] 164 so far took 282 ms
exclude calculation [8f8f9792-f8f1-4aa1-99d7-960bcd397f9b] took 0 ms
exclude file count [8f8f9792-f8f1-4aa1-99d7-960bcd397f9b] 0
zipMapper[8f8f9792-f8f1-4aa1-99d7-960bcd397f9b] 175 so far took 282 ms
zipMapper[8f8f9792-f8f1-4aa1-99d7-960bcd397f9b] 182 so far took 282 ms
zipMapper[8f8f9792-f8f1-4aa1-99d7-960bcd397f9b] 185 so far took 282 ms
zipMapper[8f8f9792-f8f1-4aa1-99d7-960bcd397f9b] 203 so far took 286 ms
zipping[8f8f9792-f8f1-4aa1-99d7-960bcd397f9b]
about to zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/update-project.zip
zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/projects.zip: 2.062s
zipped[5e64fb4c-7668-4e7b-9ef9-06d17591ee30] took 2 seconds
zip mapper[5e64fb4c-7668-4e7b-9ef9-06d17591ee30] took 2 seconds
zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/update-row.zip: 2.051s
zipped[8f27abb4-4b27-4a8c-a286-dd71d6a1b423] took 2 seconds
zip mapper[8f27abb4-4b27-4a8c-a286-dd71d6a1b423] took 2 seconds
loop! [3000]ms {"handles":4,"requests":8}
zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/project.zip: 2.293s
zipped[4909a123-683e-4c6c-ac32-e479cd80aea4] took 2 seconds
zip mapper[4909a123-683e-4c6c-ac32-e479cd80aea4] took 3 seconds
zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/select-product.zip: 2.738s
zipped[0082936c-58d4-4d4d-809c-d7a289ad91d4] took 3 seconds
zip mapper[0082936c-58d4-4d4d-809c-d7a289ad91d4] took 3 seconds
zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/add-material.zip: 2.727s
zipped[1620f390-5a22-483b-8454-de6a4cccda46] took 3 seconds
zip mapper[1620f390-5a22-483b-8454-de6a4cccda46] took 3 seconds
zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/initial-bom.zip: 2.754s
zipped[4750e4dd-584f-4fcb-8896-8b0bf0d8aa5c] took 3 seconds
zip mapper[4750e4dd-584f-4fcb-8896-8b0bf0d8aa5c] took 3 seconds
zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/create-project.zip: 2.733s
zipped[2fea2fa1-1e0d-4096-b242-bf4bc925397f] took 3 seconds
zip mapper[2fea2fa1-1e0d-4096-b242-bf4bc925397f] took 3 seconds
zip /Users/x/Documents/src/private-stack/packages/project/.esbuild/.serverless/update-project.zip: 2.726s
zipped[8f8f9792-f8f1-4aa1-99d7-960bcd397f9b] took 3 seconds
zip mapper[8f8f9792-f8f1-4aa1-99d7-960bcd397f9b] took 3 seconds
+pack

✔ Service packaged (9s)
loop! [3001]ms {"handles":3,"requests":0}

Versions (please complete the following information):

Additional context

I can send a pull request changing the fs operations, and speeding up zip too. However I have some thoughts on why and how exclude is used, if the config has bundle true then only a handful of files should be added to the zip (maybe just one).

So here we could handle exclude differently?

fredrik commented 11 months ago

@JamesKyburz Not sure if it's useful for you, but the slowdowns and lockups I experienced when zipping all the files at the same time were relieved by setting zipConcurrency.