Serverless V4 not packaging external modules as part of zip when using Serverless Compose

[joe-price-jt]

Issue description

Bug

We're currently using Serverless Compose to deploy a number of Node services defined in a single repository in sequence. All services use a common build file:

# build.yml

esbuild:
  treeShaking: true
  sourcemap: true
  minifyWhitespace: true
  minifySyntax: true
  external:
    - zod

# serverless.yml

service: some-service

frameworkVersion: '4'

build: ${file(../build.yml)}

provider:
  name: aws
  region: eu-west-2

functions:
  hello:
    handler: handler.hello
    runtime: nodejs20.x

In this case, zod is labelled as a dependency in our top-level package.json.

As zod is being labelled as an external dependency in the build configuration; when I package this service via Serverless Compose (e.g. npx sls some-service package --stage=local), I expect zod to be included as part of the node_modules directory for each lambda function zip. However, this is not the case:

This is behaving as expected in a different repository with the same build configuration where Serverless Compose is not being utilised:

When packaging via Serverless Compose with debug logs enabled, I can see that Serverless V4 is registering zod as an external dependency:

s:esbuild: Setting default external for node version  20 nodejs20.x
s:esbuild: Initial External  [ '@aws-sdk/*', 'zod' ]
s:esbuild: Initial Exclude  [ '@aws-sdk/*' ]
s:esbuild: Setting default external for node version  20 nodejs20.x
s:esbuild: Initial External  [ '@aws-sdk/*', 'zod' ]
s:esbuild: Initial Exclude  [ '@aws-sdk/*' ]

Summary

When packaging services via Serverless Compose, Serverless V4 does not seem to include packages labelled as external in the build configuration as part of the node_modules directory for each lambda function zip. This is preventing us from migrating our repository using Serverless Compose to V4. Is there something that I've missed here?

Context

Serverless Framework version: 4.4.16

[czubocha]

Possibly a duplicate of https://github.com/serverless/serverless/issues/12840

[eahefnawy]

Ok, I think I see what's going on here. I opened a PR with a fix:

https://github.com/serverless/serverless/pull/12965

[Mmarzex]

@joe-price-jt Is there a reason you cannot define your dependencies in the service's package.json? The best practice is for it to be defined as a dependency in every service.

[joe-price-jt]

Hey @Mmarzex, apologies for my very slow reply!

The repository isn't structured such that each service has its own package.json with its own dependencies. The top-level package.json manages all of the dependencies that are used across the services.

[jurkowski]

Encountering the same issue – yes, the goal is also to move away from a top-level package.json but this is a wider refactor. @Mmarzex are you planning to enforce this and is @eahefnawy's change likely to make it to a release anytime soon? This is currently blocking our upgrade to V4.