aws-amplify / amplify-backend

Home to all tools related to Amplify's code-first DX (Gen 2) for building fullstack apps on AWS
Apache License 2.0
164 stars 55 forks source link

Build fails without errors on logs #1101

Closed CGarces closed 3 months ago

CGarces commented 6 months ago

Environment information

System:
  OS: Windows 10 10.0.19045
  CPU: (8) x64 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
  Memory: 382.15 MB / 7.65 GB
Binaries:
  Node: 18.17.1 - C:\Program Files\nodejs\node.EXE
  Yarn: undefined - undefined
  npm: 10.1.0 - C:\Program Files\nodejs\npm.CMD
  pnpm: undefined - undefined
NPM Packages:
  @aws-amplify/backend: 0.13.0-beta.3
  @aws-amplify/backend-cli: 0.12.0-beta.3
  aws-amplify: 6.0.18
  aws-cdk: 2.131.0
  aws-cdk-lib: 2.131.0
  typescript: 5.3.3
AWS environment variables:
  AWS_NODEJS_CONNECTION_REUSE_ENABLED = 1
  AWS_SDK_LOAD_CONFIG = 1
  AWS_STS_REGIONAL_ENDPOINTS = regional
No CDK environment variables

Description

The amplify build stage fails without any error in the logs, so the deploy stage not is executed.

Last lines of the logs

2024-03-02T13:57:25.878Z [INFO]: ▲ Next.js 14.1.0
2024-03-02T13:57:25.879Z [INFO]: 
2024-03-02T13:57:25.953Z [INFO]: Creating an optimized production build ...
2024-03-02T13:57:59.833Z [INFO]: ✓ Compiled successfully
2024-03-02T13:57:59.839Z [INFO]: Linting and checking validity of types ...
2024-03-02T13:58:08.884Z [INFO]: Collecting page data ...
2024-03-02T13:58:10.352Z [INFO]: Generating static pages (0/16) ...
2024-03-02T13:58:10.642Z [INFO]: Generating static pages (4/16)
2024-03-02T13:58:12.701Z [INFO]: Generating static pages (8/16)
2024-03-02T13:58:13.153Z [INFO]: Generating static pages (12/16)
2024-03-02T13:58:13.348Z [INFO]: ✓ Generating static pages (16/16)
2024-03-02T13:58:13.970Z [INFO]: Finalizing page optimization ...
2024-03-02T13:58:13.970Z [INFO]: Collecting build traces ...
2024-03-02T13:58:28.709Z [INFO]: 

Maybe it's and out of memory error, not shown in the new interface?

blinkdaffer commented 6 months ago

run it with the --debug flag

CGarces-Apser commented 6 months ago

Thanks @blinkdaffer

--debug not add more details

> next build "--debug"
2024-03-03T16:43:13.966Z [INFO]: ⚠ No build cache found. Please configure build caching for faster rebuilds. Read more: https://nextjs.org/docs/messages/no-cache
2024-03-03T16:43:13.971Z [INFO]: Attention: Next.js now collects completely anonymous telemetry regarding usage.
2024-03-03T16:43:13.971Z [INFO]: This information is used to shape Next.js' roadmap and prioritize features.
                                 You can learn more, including how to opt-out if you'd not like to participate in this anonymous program, by visiting the following URL:
                                 https://nextjs.org/telemetry
2024-03-03T16:43:14.046Z [INFO]: ▲ Next.js 14.1.0
2024-03-03T16:43:14.046Z [INFO]: 
2024-03-03T16:43:14.111Z [INFO]: Creating an optimized production build ...
2024-03-03T16:43:44.251Z [INFO]: ✓ Compiled successfully
2024-03-03T16:43:44.256Z [INFO]: Linting and checking validity of types ...
2024-03-03T16:43:52.278Z [INFO]: Collecting page data ...
2024-03-03T16:43:53.523Z [INFO]: Generating static pages (0/16) ...
2024-03-03T16:43:53.795Z [INFO]: Generating static pages (4/16)
2024-03-03T16:43:55.680Z [INFO]: Generating static pages (8/16)
2024-03-03T16:43:55.921Z [INFO]: Generating static pages (12/16)
2024-03-03T16:44:02.943Z [INFO]: ✓ Generating static pages (16/16)
2024-03-03T16:44:03.133Z [INFO]: Finalizing page optimization ...
                                 Collecting build traces ...
2024-03-03T16:44:03.348Z [INFO]: Redirects
2024-03-03T16:44:03.349Z [INFO]: ┌ source: /:path+/
                                 ├ destination: /:path+
                                 └ permanent: true

2024-03-03T16:44:09.528Z [INFO]: 

The build is set as "failed" without more information image

CGarces commented 6 months ago

Just to add more context.

amplify.yaml ```yaml version: 1 backend: phases: build: commands: - "nvm use 20" - "npm install -g pnpm" - "pnpm install" - "npx amplify pipeline-deploy --branch $AWS_BRANCH --app-id $AWS_APP_ID" frontend: phases: preBuild: commands: - "pnpm install" build: commands: - "pnpm run build --debug" artifacts: baseDirectory: .next files: - "**/*" cache: paths: - "node_modules/**/*" ```
package.json ```json { "name": "foo-bar-frontend", "version": "0.1.0", "private": true, "scripts": { "dev": "next dev", "build": "next build", "start": "next start", "lint": "next lint" }, "dependencies": { "@ant-design/icons": "^5.3.0", "@ant-design/nextjs-registry": "^1.0.0", "@aws-amplify/adapter-nextjs": "^1.0.18", "@aws-sdk/client-cognito-identity-provider": "^3.515.0", "@tanstack/react-query": "^5.24.1", "amazon-cognito-identity-js": "^6.3.7", "antd": "^5.14.2", "aws-amplify": "^6.0.18", "bcrypt": "^5.1.1", "classnames": "^2.5.1", "echarts": "^5.5.0", "echarts-for-react": "^3.0.2", "immer": "^10.0.3", "next": "^14.1.0", "next-auth": "5.0.0-beta.13", "react": "^18.2.0", "react-dom": "^18.2.0", "zod": "^3.22.4", "zustand": "^4.5.1" }, "devDependencies": { "@aws-amplify/backend": "0.13.0-beta.3", "@aws-amplify/backend-cli": "^0.12.0-beta.3", "@types/node": "^20.11.20", "@types/react": "^18.2.58", "@types/react-dom": "^18.2.19", "autoprefixer": "^10.4.17", "aws-cdk": "^2", "aws-cdk-lib": "^2", "constructs": "^10.0.0", "esbuild": "^0.20.1", "eslint": "^8.56.0", "eslint-config-next": "^14.1.0", "postcss": "^8.4.35", "tailwindcss": "^3.4.1", "tsx": "^4.7.1", "typescript": "^5.3.3" } } ```
Jay2113 commented 6 months ago

@CGarces 👋 , thanks for reaching out. Can you share your Amplify app id?

CGarces commented 6 months ago

d3qvt5uw9k572 builds form 1 to 12, all failed without errors.

Jay2113 commented 6 months ago

@CGarces thanks for sharing that information. While reviewing the internal logs for build 12, I observed that it failed due to running out of memory.

Since the default build container has 7GB RAM, it should be able to support about 5.5GB max old space size assuming no other build process is running in parallel. Setting the max-old-space-size to >5.5GB for large builds would result in the OS killing the node process.

Thus, you can set a max old space size of 5.5GB in the buildSpec as follows:

export NODE_OPTIONS=--max-old-space-size=5632\n'

Additionally, you can verify whether NodeJS has correctly picked up the max-old-space-size config by including the following command at the preBuild step in the buildSpec:

node -e 'console.log(v8.getHeapStatistics().total_available_size/(1024*1024))'

Lastly, we are tracking the feature request to increase the compute capacity of the build containers/environments here: https://github.com/aws-amplify/amplify-hosting/issues/654


Can you download the logs for build 12 from the console and confirm if you are able to observe the OOM error within them? This will help us gauge if the appropriate error is surfaced in the logs.

johnpc commented 6 months ago

These docs exist for troubleshooting memory issues: https://docs.aws.amazon.com/amplify/latest/userguide/troubleshooting-ssr-deployment.html#out-of-memory

CGarces commented 6 months ago

Can you download the logs for build 12 from the console and confirm if you are able to observe the OOM error within them?

No, I can't observe the error. That's the point of this issue. I already suspect that was a OOM error (it's at the end of the 1º post) The problem is that the error not is show on the logs or the AWS Console interface.

ykethan commented 6 months ago

Marking this bug for error messaging and visibility improvements.

Jay2113 commented 3 months ago

Hi @CGarces 👋 , we have deployed a fix for this issue and with this change the builds should no longer fail without surfacing the appropriate OOM error/banner in the console. I am going to close this thread but please feel free to reopen it if you are still running into it. Thanks!