document function build issues with esbuild in monorepos, with pnpm, yarn

josefaidt commented 6 months ago

Describe the content issue: A clear and concise description of what the content issue is. Is there content missing or is there incorrect content?

add a (temporary) troubleshooting doc to callout the need for installing esbuild at the monorepo root when using yarn, pnpm, or other package manager install strategies that symlink the esbuild bin in monorepo packages

https://github.com/aws-amplify/amplify-backend/issues/1512

when executing pnpm ampx sandbox from within the monorepo package CDK fails to execute esbuild for Function builds in the target directory

➜  p ampx sandbox

  Amplify Sandbox

  Identifier:   josef
  Stack:        amplify-identityprovidersbackend-josef-sandbox-04818156ae

  To specify a different sandbox identifier, use --identifier

Unable to build the Amplify backend definition.
Caused By: Error: Failed to bundle asset amplify-identityprovidersbackend-josef-sandbox-04818156ae/auth/SecretFetcherResourceProviderLambda/Code/Stage, bundle output is located at /Users/josef/Documents/projects/aws-amplify/amplify-gen2-identity-providers/packages/identity-providers-backend/.amplify/artifacts/cdk.out/bundling-temp-bb3b79a8a866090c1bd808cc6e9822677dbcd6fd39430da733e289729027d116-error: Error: bash -c pnpm exec -- esbuild --bundle "/Users/josef/Documents/projects/aws-amplify/amplify-gen2-identity-providers/node_modules/.pnpm/@aws-amplify+backend@1.0.3_@aws-sdk+client-cloudformation@3.598.0_@aws-sdk+client-s3@3.598.0__tn7afzbpzntiestvo3f3w5qwye/node_modules/@aws-amplify/backend/lib/engine/backend-secret/lambda/backend_secret_fetcher.js" --target=node18 --platform=node --outfile="/Users/josef/Documents/projects/aws-amplify/amplify-gen2-identity-providers/packages/identity-providers-backend/.amplify/artifacts/cdk.out/bundling-temp-bb3b79a8a866090c1bd808cc6e9822677dbcd6fd39430da733e289729027d116/index.js" --external:@aws-sdk/* run in directory /Users/josef/Documents/projects/aws-amplify/amplify-gen2-identity-providers exited with status 254
    at AssetStaging.bundle (/Users/josef/Documents/projects/aws-amplify/amplify-gen2-identity-providers/node_modules/.pnpm/aws-cdk-lib@2.146.0_constructs@10.3.0/node_modules/aws-cdk-lib/core/lib/asset-staging.js:2:619)

Resolution: Check your backend definition in the `amplify` folder for syntax and type errors.

However if you wrap ampx sandbox in an npm script and run with npm run dev, for example, it works fine. Otherwise you will need to install esbuild at the project root where CDK is executing from. Since pnpm will only link the bin in the directory in which it is specified as a dependency, we receive a vague error

URL page where content issue is:

euan-cowie commented 5 months ago

Is there any update on this? I have tried to achieve the fix in the mentioned issue but have had no success.

{
  "name": "test",
  "dependencies": {
    "esbuild": "^0.21.4"
  },
  "devDependencies": {
    "eslint": "^8.48.0",
    "prettier": "^3.0.3",
    "syncpack": "^11.2.1",
    "turbo": "latest"
  },
  "private": true,
  "scripts": {
    "build": "turbo run build",
    "dev": "turbo run dev ios "
  }
}

This is my root package.json but still I get

2024-06-06T18:15:22.720Z [INFO]: BackendSynthError: Unable to build the Amplify backend definition.

103

Resolution: Check your backend definition in the `amplify` folder for syntax and type errors.

104

Cause: Error: Failed to bundle asset amplify-d3hkzx89yodirg-main-branch-84deb20467/AmplifyBranchLinker/CustomResourceLambda/Code/Stage, bundle output is located at /codebuild/output/src3664567169/src/test/packages/backend/.amplify/artifacts/cdk.out/bundling-temp-6179cc509a26d893cc8c51e1ff8fe9d2840f1ea0578adcf1070f7cec38fa011d-error: Error: bash -c pnpm exec -- esbuild --bundle "/codebuild/output/src3664567169/src/test/node_modules/@aws-amplify/backend/lib/engine/branch-linker/lambda/branch_linker.js" --target=node18 --platform=node --outfile="/codebuild/output/src3664567169/src/test/packages/backend/.amplify/artifacts/cdk.out/bundling-temp-6179cc509a26d893cc8c51e1ff8fe9d2840f1ea0578adcf1070f7cec38fa011d/index.js" run in directory /codebuild/output/src3664567169/src/test exited with status 127

I've cleaned my project multiple times and regenerated lock files etc etc. Any help would be greatly appreciated!

josefaidt commented 5 months ago

Hey @euan-cowie what package manager are you using with turbo?

euan-cowie commented 5 months ago

Hey @josefaidt, sorry for the late response, was recovering from a nasty flu!

I'm using pnpm.

josefaidt commented 5 months ago

@euan-cowie can you try running that command with --debug? do you experience a similar issue locally or is this only failing in the automated builds?

stewartmoreland commented 3 months ago

Encountered the same error in a monorepo using turbo & yarn workspaces.

Adding esbuild to the root devDependencies resolved the issue for me as @josefaidt mentioned above. @euan-cowie I would advise using the check-dependency-version-consistency package to ensure your package versions are aligned.

Glad I finally found this post. It's been plaguing me for quite a while! 😆

aws-amplify / docs

document function build issues with esbuild in monorepos, with pnpm, yarn #7634