aws-amplify / amplify-hosting

AWS Amplify Hosting provides a Git-based workflow for deploying and hosting fullstack serverless web applications.
https://aws.amazon.com/amplify/hosting/
Apache License 2.0
448 stars 113 forks source link

Nextjs Turbo Monorepo 500 Error #3179

Closed Rafcin closed 1 year ago

Rafcin commented 1 year ago

Before opening, please confirm:

App Id

d2t18tt2z5krry

AWS Region

ca-central-1

Amplify Hosting feature

Build settings, Deployments, Frontend builds, Monorepo

Describe the bug

Howdy, I've discovered a bug with the new build system I had previously not thought to test. One of my main projects uses a monorepo structure. After some tinkering and dealing with the nightmare of a package manager known as pnpm, I seem unable to deploy the app; however, it builds correctly, and I can't trace the issue. The app builds correctly, finishes caching, and does all its post-build steps; it finds the correct .next folder, and then when I check the amplify domain, I get thrown a 500 error.

Expected behavior

I should be able to see my app working just fine; I think this could be the next standalone issue, although I'm wildly unsure.

Reproduction steps

I don't have a demo that can be used yet; I need to spin up a separate barebones version of my repo for this. However, my structure follows the Amplify-UI repo structure and rollup config almost the same way. The main difference is I have an apps folder, and all my apps locally reference the packages and not from npm because they aren't published. I also use hoist and auto peers shamefully...

If the Amplify team needs access to the repo, mention me, and I'll send you an invite over git.

Build Settings

Here is the Amplify YML config, you can ignore the `apps/main` section, that's old. `apps/help` is the new config.

  version: 1
  applications:
    - appRoot: apps/main
      env:
      variables:
        #AMPLIFY_DIFF_DEPLOY: true
      backend:
        phases:
          build:
            commands:
              - echo BACKEND MAIN BUILD
              - amplifyPush --simple
      frontend:
        phases:
          preBuild:
            commands:
              - echo FRONTEND MAIN PREBUILD
              # - node -e
              # - 'console.log(v8.getHeapStatistics().total_available_size/(1024*1024))'
              # Step out
              - cd ../../
              # We don't need to move folders, we already moved out to the root at the backend build step.
              #- nvm install 18
              - npx pnpm install
          build:
            commands:
              - echo FRONTEND MAIN BUILD
              # - aws s3 cp amplify.yml s3://main-sandbox/public/
              # - yarn amplify:clean:current-backend
              - npx pnpm run main:build
              # Step in
              - cd apps/main
          postBuild:
            commands:
              - echo END OF HELP BUILD
        artifacts:
          baseDirectory: .next
          files:
            - "**/*"
        cache:
          paths:
            - node_modules/**/*
            #- '!node_modules/.cache'
            #- '!.next/**/*'
    - appRoot: apps/help
      env:
      variables:
        #AMPLIFY_DIFF_DEPLOY: true
      backend:
        phases:
          build:
            commands:
              - echo NO BACKEND BUILD
      frontend:
        phases:
          preBuild:
            commands:
              - echo FRONTEND HELP PREBUILD
              - cd ../../
              - nvm install 16.13 # NodeJS 16.13+ is the latest version that Amplify Hosting supports
              - nvm use 16
              - node -v
              - echo -e "@oxygen:registry=https://registry.npmjs.org/\nstrict-peer-dependencies=false\nauto-install-peers=true\nshamefully-hoist=true" > .npmrc
          build:
            commands:
              - echo FRONTEND HELP BUILD
              # Cleans all pkg and app folders of nm, turbo, dist and .next
              - ./scripts/clean.sh
              - nvm install 16.13 # NodeJS 16.13+ is the latest version that Amplify Hosting supports
              - nvm use 16
              - node -v
              - npx pnpm install
              # - aws s3 cp amplify.yml s3://main-sandbox/public/
              # - yarn amplify:clean:current-backend
              - npx pnpm run help:build
          postBuild:
            commands:
              - echo END OF HELP BUILD
        artifacts:
          # Base directory should just be next. If you set it to apps/someapp when the build finishes it would then be apps/someapp/apps/someapp/.next and that is wrong.
          baseDirectory: .next
          files:
            - "**/*"
        cache:
          paths:
            - node_modules/**/*

### Log output

<details>

Logs

                             # Starting phase: preBuild
                             # Executing command: echo FRONTEND HELP PREBUILD
                             FRONTEND HELP PREBUILD
                             # Executing command: cd ../../

2022-12-05T21:13:44.852Z [INFO]: # Executing command: nvm install 16.13 2022-12-05T21:13:45.481Z [WARNING]: v16.13.2 is already installed. 2022-12-05T21:13:56.774Z [INFO]: Now using node v16.13.2 (npm v8.1.2) 2022-12-05T21:13:56.776Z [INFO]: Installing default global packages from /root/.nvm/default-packages... npm install -g --quiet yarn@1.22.0 2022-12-05T21:13:59.290Z [INFO]: changed 1 package, and audited 2 packages in 2s 2022-12-05T21:13:59.290Z [INFO]: found 0 vulnerabilities 2022-12-05T21:13:59.300Z [INFO]: # Executing command: nvm use 16 2022-12-05T21:13:59.660Z [INFO]: Now using node v16.18.1 (npm v8.19.2) 2022-12-05T21:13:59.660Z [INFO]: # Executing command: node -v 2022-12-05T21:13:59.663Z [INFO]: v16.18.1 2022-12-05T21:13:59.663Z [INFO]: # Executing command: echo -e "@oxygen:registry=https://registry.npmjs.org/\nstrict-peer-dependencies=false\nauto-install-peers=true\nshamefully-hoist=true" > .npmrc 2022-12-05T21:13:59.663Z [INFO]: # Completed phase: preBuild

Starting phase: build

                             # Executing command: echo FRONTEND HELP BUILD
                             FRONTEND HELP BUILD

2022-12-05T21:13:59.664Z [INFO]: # Executing command: ./scripts/clean.sh 2022-12-05T21:14:30.483Z [INFO]: # Executing command: nvm install 16.13 2022-12-05T21:14:31.111Z [WARNING]: v16.13.2 is already installed. 2022-12-05T21:14:31.415Z [INFO]: Now using node v16.13.2 (npm v8.1.2) 2022-12-05T21:14:31.415Z [INFO]: Installing default global packages from /root/.nvm/default-packages... npm install -g --quiet yarn@1.22.0 2022-12-05T21:14:31.989Z [INFO]: changed 1 package, and audited 2 packages in 463ms 2022-12-05T21:14:31.990Z [INFO]: found 0 vulnerabilities 2022-12-05T21:14:32.000Z [INFO]: # Executing command: nvm use 16 2022-12-05T21:14:32.378Z [INFO]: Now using node v16.18.1 (npm v8.19.2) 2022-12-05T21:14:32.378Z [INFO]: # Executing command: node -v 2022-12-05T21:14:32.381Z [INFO]: v16.18.1 2022-12-05T21:14:32.381Z [INFO]: # Executing command: npx pnpm install 2022-12-05T21:14:33.416Z [WARNING]: npm 2022-12-05T21:14:33.416Z [WARNING]: WARN exec The following package was not found and will be installed: pnpm@7.18.0 2022-12-05T21:14:34.396Z [INFO]: Scope: all 7 workspace projects 2022-12-05T21:14:34.740Z [INFO]: Progress: resolved 1, reused 0, downloaded 0, added 0 2022-12-05T21:14:35.748Z [INFO]: Progress: resolved 25, reused 0, downloaded 8, added 0 2022-12-05T21:14:36.753Z [INFO]: Progress: resolved 42, reused 0, downloaded 25, added 0 2022-12-05T21:14:37.754Z [INFO]: Progress: resolved 55, reused 0, downloaded 36, added 0 2022-12-05T21:14:38.266Z [INFO]: packages/design-system |  WARN  deprecated rollup-plugin-terser@7.0.2 2022-12-05T21:14:38.757Z [INFO]: Progress: resolved 70, reused 0, downloaded 52, added 0 2022-12-05T21:14:39.767Z [INFO]: Progress: resolved 95, reused 0, downloaded 72, added 0 2022-12-05T21:14:40.771Z [INFO]: Progress: resolved 109, reused 0, downloaded 92, added 0 2022-12-05T21:14:41.775Z [INFO]: Progress: resolved 125, reused 0, downloaded 109, added 0 2022-12-05T21:14:42.777Z [INFO]: Progress: resolved 151, reused 0, downloaded 136, added 0 2022-12-05T21:14:43.781Z [INFO]: Progress: resolved 194, reused 0, downloaded 165, added 0 2022-12-05T21:14:44.793Z [INFO]: Progress: resolved 202, reused 0, downloaded 173, added 0 2022-12-05T21:14:45.832Z [INFO]: Progress: resolved 217, reused 0, downloaded 186, added 0 2022-12-05T21:14:46.838Z [INFO]: Progress: resolved 232, reused 0, downloaded 200, added 0 2022-12-05T21:14:47.840Z [INFO]: Progress: resolved 262, reused 0, downloaded 227, added 0 2022-12-05T21:14:48.277Z [INFO]: packages/design-system |  WARN  deprecated @types/cssnano@5.1.0 2022-12-05T21:14:48.844Z [INFO]: Progress: resolved 293, reused 0, downloaded 258, added 0 2022-12-05T21:14:49.846Z [INFO]: Progress: resolved 333, reused 0, downloaded 292, added 0 2022-12-05T21:14:50.859Z [INFO]: Progress: resolved 381, reused 0, downloaded 329, added 0 2022-12-05T21:14:51.860Z [INFO]: Progress: resolved 413, reused 0, downloaded 362, added 0 2022-12-05T21:14:52.850Z [INFO]: apps/help |  WARN  deprecated sourcemap-codec@1.4.8 2022-12-05T21:14:52.864Z [INFO]: Progress: resolved 450, reused 0, downloaded 397, added 0 2022-12-05T21:14:53.458Z [INFO]: apps/main |  WARN  deprecated uuid@3.4.0 2022-12-05T21:14:53.865Z [INFO]: Progress: resolved 493, reused 0, downloaded 438, added 0 2022-12-05T21:14:54.050Z [INFO]: apps/main |  WARN  deprecated uuid@3.3.2 2022-12-05T21:14:54.777Z [INFO]: apps/main |  WARN  deprecated @aws-sdk/eventstream-marshaller@3.6.1 2022-12-05T21:14:54.871Z [INFO]: Progress: resolved 515, reused 0, downloaded 461, added 0 2022-12-05T21:14:55.789Z [INFO]: apps/main |  WARN  deprecated @aws-sdk/util-base64-browser@3.6.1 2022-12-05T21:14:55.806Z [INFO]: apps/main |  WARN  deprecated @aws-sdk/util-base64-node@3.6.1 2022-12-05T21:14:55.871Z [INFO]: Progress: resolved 555, reused 0, downloaded 495, added 0 2022-12-05T21:14:56.881Z [INFO]: Progress: resolved 578, reused 0, downloaded 526, added 0 2022-12-05T21:14:57.883Z [INFO]: Progress: resolved 618, reused 0, downloaded 555, added 0 2022-12-05T21:14:58.887Z [INFO]: Progress: resolved 675, reused 0, downloaded 612, added 0 2022-12-05T21:14:59.889Z [INFO]: Progress: resolved 798, reused 0, downloaded 705, added 0 2022-12-05T21:15:00.893Z [INFO]: Progress: resolved 884, reused 0, downloaded 786, added 0 2022-12-05T21:15:01.217Z [INFO]: apps/main |  WARN  deprecated @aws-sdk/util-base64-browser@3.186.0 2022-12-05T21:15:01.218Z [INFO]: apps/main |  WARN  deprecated @aws-sdk/util-base64-node@3.186.0 2022-12-05T21:15:01.892Z [INFO]: Progress: resolved 957, reused 0, downloaded 864, added 0 2022-12-05T21:15:02.892Z [INFO]: Progress: resolved 1013, reused 0, downloaded 913, added 0 2022-12-05T21:15:03.893Z [INFO]: Progress: resolved 1128, reused 0, downloaded 1015, added 0 2022-12-05T21:15:04.893Z [INFO]: Progress: resolved 1239, reused 0, downloaded 1122, added 0 2022-12-05T21:15:05.565Z [INFO]: apps/main |  WARN  deprecated querystring@0.2.0 2022-12-05T21:15:05.893Z [INFO]: Progress: resolved 1340, reused 0, downloaded 1222, added 0 2022-12-05T21:15:06.893Z [INFO]: Progress: resolved 1432, reused 0, downloaded 1302, added 0 2022-12-05T21:15:07.425Z [INFO]: packages/design-system |  WARN  deprecated stable@0.1.8 2022-12-05T21:15:07.894Z [INFO]: Progress: resolved 1526, reused 0, downloaded 1391, added 0 2022-12-05T21:15:08.941Z [INFO]: Progress: resolved 1568, reused 0, downloaded 1481, added 0 2022-12-05T21:15:09.947Z [INFO]: Progress: resolved 1617, reused 0, downloaded 1524, added 0 2022-12-05T21:15:10.961Z [INFO]: Progress: resolved 1670, reused 0, downloaded 1580, added 0 2022-12-05T21:15:12.033Z [INFO]: Progress: resolved 1756, reused 0, downloaded 1657, added 0 2022-12-05T21:15:13.034Z [INFO]: Progress: resolved 1851, reused 0, downloaded 1738, added 0 2022-12-05T21:15:13.123Z [INFO]: apps/main |  WARN  deprecated uglify-es@3.3.9 2022-12-05T21:15:13.286Z [INFO]: apps/main |  WARN  deprecated source-map-resolve@0.5.3 2022-12-05T21:15:13.444Z [INFO]: apps/main |  WARN  deprecated resolve-url@0.2.1 2022-12-05T21:15:13.461Z [INFO]: apps/main |  WARN  deprecated source-map-url@0.4.1 2022-12-05T21:15:13.475Z [INFO]: apps/main |  WARN  deprecated urix@0.1.0 2022-12-05T21:15:14.277Z [INFO]: Progress: resolved 1937, reused 0, downloaded 1847, added 0 2022-12-05T21:15:14.294Z [INFO]: Packages: +1905 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2022-12-05T21:15:15.405Z [INFO]: Progress: resolved 1937, reused 0, downloaded 1848, added 0 2022-12-05T21:15:15.518Z [INFO]: Packages are hard linked from the content-addressable store to the virtual store. Content-addressable store is at: /codebuild/output/.pnpm-store/v3 Virtual store is at: node_modules/.pnpm 2022-12-05T21:15:16.405Z [INFO]: Progress: resolved 1937, reused 0, downloaded 1895, added 137 2022-12-05T21:15:17.406Z [INFO]: Progress: resolved 1937, reused 0, downloaded 1895, added 164 2022-12-05T21:15:18.445Z [INFO]: Progress: resolved 1937, reused 0, downloaded 1895, added 165 2022-12-05T21:15:19.445Z [INFO]: Progress: resolved 1937, reused 0, downloaded 1897, added 473 2022-12-05T21:15:20.444Z [INFO]: Progress: resolved 1937, reused 0, downloaded 1898, added 769 2022-12-05T21:15:21.449Z [INFO]: Progress: resolved 1937, reused 0, downloaded 1898, added 1108 2022-12-05T21:15:22.449Z [INFO]: Progress: resolved 1937, reused 0, downloaded 1898, added 1564 2022-12-05T21:15:23.459Z [INFO]: Progress: resolved 1937, reused 0, downloaded 1898, added 1844 2022-12-05T21:15:24.459Z [INFO]: Progress: resolved 1937, reused 0, downloaded 1898, added 1904 2022-12-05T21:15:26.172Z [INFO]: Progress: resolved 1937, reused 0, downloaded 1899, added 1904 2022-12-05T21:15:26.472Z [INFO]: .../node_modules/core-js-pure postinstall$ node -e "try{require('./postinstall')}catch(e){}" 2022-12-05T21:15:26.513Z [INFO]: .../swiper@8.4.5/node_modules/swiper postinstall$ node -e "try{require('./postinstall')}catch(e){}" 2022-12-05T21:15:26.554Z [INFO]: .../node_modules/core-js-pure postinstall: Thank you for using core-js ( https://github.com/zloirock/core-js ) for polyfilling JavaScript standard library! 2022-12-05T21:15:26.556Z [INFO]: .../node_modules/core-js-pure postinstall: The project needs your help! Please consider supporting of core-js: .../node_modules/core-js-pure postinstall: > https://opencollective.com/core-js .../node_modules/core-js-pure postinstall: > https://patreon.com/zloirock .../node_modules/core-js-pure postinstall: > bitcoin: bc1qlea7544qtsmj2rayg0lthvza9fau63ux0fstcz .../node_modules/core-js-pure postinstall: Also, the author of core-js ( https://github.com/zloirock ) is looking for a good job -) 2022-12-05T21:15:26.558Z [INFO]: .../node_modules/core-js-pure postinstall: Done 2022-12-05T21:15:26.670Z [INFO]: .../swiper@8.4.5/node_modules/swiper postinstall: Done 2022-12-05T21:15:26.710Z [INFO]: .../node_modules/maplibre-gl postinstall$ node ./postinstall.js 2022-12-05T21:15:26.794Z [INFO]: .../node_modules/maplibre-gl postinstall: Done 2022-12-05T21:15:27.136Z [INFO]: .../.pnpm/turbo@1.6.3/node_modules/turbo postinstall$ node install.js 2022-12-05T21:15:27.137Z [INFO]: .../esbuild@0.15.18/node_modules/esbuild postinstall$ node install.js 2022-12-05T21:15:27.196Z [INFO]: Progress: resolved 1937, reused 0, downloaded 1899, added 1905, done 2022-12-05T21:15:27.229Z [INFO]: .../.pnpm/turbo@1.6.3/node_modules/turbo postinstall: Done 2022-12-05T21:15:27.305Z [INFO]: .../esbuild@0.15.18/node_modules/esbuild postinstall: Done 2022-12-05T21:15:29.220Z [INFO]: devDependencies:



</details>

### Additional information

_No response_
ghost commented 1 year ago

Hi @Rafcin 👋🏽 thanks for raising this issue and providing detailed information. We are investigating this issue with monorepos and SSR deployments and are working to find a solution as soon as possible. We will post an update once we have next steps.

Rafcin commented 1 year ago

@hloriana Awesome, much appreciated! Lmk if you need any thing!

jbreemhaar commented 1 year ago

@hloriana, @Rafcin Not sure if related but I fixed this issue in my project by adding the following to nextConfig.experimental. My project uses yarn workspaces, not Turbo though.

    outputFileTracingRoot: path.join(__dirname, "../.."),

I noticed my project throwing 500s when migration from SSG to SSR, within cloudwatch I noticed it was throwing the following error.

Error: Cannot find module 'next/dist/server/next-server'
--

There's more info in this thread: https://github.com/vercel/next.js/discussions/39432

Rafcin commented 1 year ago

@jbreemhaar lemme give it a try! I feel like I have this config, not sure.

Rafcin commented 1 year ago

For ref here is my NextJS config

/* eslint-disable turbo/no-undeclared-env-vars */
/* eslint-disable import/order */
const path = require('path')

const withBundleAnalyzer = require('@next/bundle-analyzer')({
  enabled: process.env.ANALYZE === 'true',
})
/**
 * @note(pnpm) pass ENV variable to determine if we should transpile
 *            ./src  == transpile
 *            ./dist != transpile
 */

const transpilePackages = ["@oxygen/design-system"]

const withMDX = require('@next/mdx')({
  extension: /\.mdx?$/,
  options: {
    remarkPlugins: [],
    rehypePlugins: [],
    // If you use `MDXProvider`, uncomment the following line.
    providerImportSource: "@mdx-js/react",
    pageExtensions: ['ts', 'tsx', 'js', 'jsx', 'md', 'mdx'],
  },
})

const PROTOCOL = {
  HTTP: 'http',
  HTTPS: 'https',
}
const protocol = PROTOCOL.HTTPS
/**
 * @note(pnpm) until we move "websites" into "packages"...
 *
 * When developing locally:
 * - pnpm dev:ds
 * - OR update .env => pnpm dev
 *
 * This maps all the externals required for proper localized
 *  files system path mapping.
 */
const isLocal = process.env.DESIGN_SYSTEM__LINK === 'true' ? true : false
const externals = [
  '@types/react',
  'react',
  'react-dom',
]
const isLocalDebugMessages = [
  `warn  - [ 📝 ]  pnpm link...`,
  `warn  - [ 🔗 ]  @oxygen/design-system`,
]

/**
 * @note The following environment variables are required
 */
const envRequired = [
  'GH_TOKEN',
  'NEXT_PUBLIC_SITE',
]
envRequired.map((item) => {
  if (!process.env[item]) {
    throw new Error(`process.env.${item} is not set in env`)
  }
})

/**
 * @type {import('next').NextConfig}
 **/
const nextConfig = {
  reactStrictMode: true,
  amp: {
    canonicalBase: undefined,
  },
  // analyticsId: 'SELF_HOSTED_ONLY',
  assetPrefix: undefined,
  basePath: '',
  cleanDistDir: true,
  compiler: {},
  compress: true,
  // crossOrigin: 'same-origin',
  devIndicators: { buildActivity: true, buildActivityPosition: 'bottom-right' },
  distDir: './.next',
  // env,
  eslint: {
    // @note(eslint) handled outside of next
    ignoreDuringBuilds: true,
  },
  excludeDefaultMomentLocales: true,
  experimental: {
    appDir: false,
    legacyBrowsers: false,
    esmExternals: "loose",
    transpilePackages,
  },
  httpAgentOptions: {
    keepAlive: true,
  },
  i18n: {
    locales: ['en'],
    defaultLocale: 'en',
  },
  images: {
    deviceSizes: [743, 744, 1127, 1128, 1440],
    imageSizes: [16, 32, 48, 64, 96, 128, 256, 384],
    domains: [
      "NA",
      "**.amazonws.com",
      "NA",
      "**.googleapis.com",
    ],
    remotePatterns: [
      {
        protocol,
        hostname: `**.${process.env.NEXT_PUBLIC_SITE}`,
      },
      // @note(remotePattern) Google Apis
      {
        protocol,
        hostname: `**.googleapis.com`,
      },
      // @note(remotePattern) Future proofing "other" websites
      {
        protocol,
        hostname: `**.NA.com`,
      },
      // @note(remotePattern) AWS
      {
        protocol,
        hostname: `**.amazonaws.com`,
      },
      // @note(remotePattern) Twitter
      {
        protocol,
        hostname: `pbs.twimg.com`,
      },
      // @note(remotePattern) Unsplash
      {
        protocol,
        hostname: `images.unsplash.com`,
      },
    ],
    formats: ["image/avif", "image/webp"],
    minimumCacheTTL: 18144000, // 1 month
  },
  onDemandEntries: {
    /**
     * @note(next)
     * period (in ms) where the server will keep pages in the buffer
     */
    maxInactiveAge: 15 * 1000,
    /**
     * @note(next)
     * number of pages that should be kept simultaneously without being disposed
     */
    pagesBufferLength: 2,
  },
  optimizeFonts: true,
  output: undefined,
  outputFileTracing: true,
  outputFileTracingRoot: path.join(__dirname, "../.."),
  pageExtensions: ["js", "jsx", "md", "mdx", "ts", "tsx"],
  poweredByHeader: false,
  productionBrowserSourceMaps: true,
  publicRuntimeConfig: {
    // @note(next) available on server and client
  },
  sassOptions: {},
  serverRuntimeConfig: {
    // @note(next) available on the server
  },
  staticPageGenerationTimeout: 60,
  swcMinify: true,
  trailingSlash: false,
  typescript: {
    // @note(typescript) handled outside of next
    ignoreBuildErrors: true,
  },
  // @note(next) false will block: ./pages
  useFileSystemPublicRoutes: true,
  // @ts-ignore
  webpack: (config, { buildId, dev, isServer, defaultLoaders, webpack }) => {
    // @note(pnpm)  path mapping if working locally
    if (isLocal) {
      isLocalDebugMessages.map((msg) =>
        console.debug('\x1b[33m%s\x1b[0m', 'warn', ' - ', msg)
      )
      externals.map((ext) => {
        console.debug('\x1b[33m%s\x1b[0m', 'warn', ' - [ 📦 ] ›  ', ext)
        // @note(npmrc) shamefully-hoist === node_modules at root
        // @todo(npmrc) would be nice to not shamefully-hoist
        config.resolve.alias[ext] = path.resolve(
          __dirname,
          '..',
          '..',
          'node_modules',
          ext
        )
      })
    }

    return config
  },
}

/**
 * @note
 * Plugins cannot handle their own Configuration at this time.
 */
 const plugins = [withBundleAnalyzer, withMDX]
 module.exports = plugins.reduce((config, plugin) => plugin(config), nextConfig)
Rafcin commented 1 year ago

@jbreemhaar Unfortunately, that did not change anything :(

jbreemhaar commented 1 year ago

Hmm bummer :/. Have you checked your cloudwatch logs?

Rafcin commented 1 year ago

I don't have access to anything behind the scenes

jbreemhaar commented 1 year ago

Ugh thats going to be a hard to find bug. Good luck!

Rafcin commented 1 year ago

We'll see, I'm not super familiar with the next JS standalone deployment. I also don't really have access to the behind the scenes so I don't know what the issue is.

Maybe it's a configuration issue with my next config

king0120 commented 1 year ago

I've also run into this issue when trying to build and deploy a nextjs app using NX as the monorepo. I attempted @jbreemhaar 's fix but unfortunately was met with a build error around the server.js file no longer existing when I set the outputFileTracing to be the root. It appears that where server.js normally is, the entrypoint seems to be apps/APP_NAME/server.js

Rafcin commented 1 year ago

@hloriana Have there been any updates to this issue? It currently prevents me from deploying my project using the new system.

Rafcin commented 1 year ago

Unsure if related however new canary build of Next had this bug fix reported: https://github.com/vercel/next.js/pull/43746

DuCanhGH commented 1 year ago

@Rafcin no, my PR is not related to your issue as it fixes an issue with appDir's route groups, which you don't seem to be using. Try moving this lỉne: outputFileTracingRoot: path.join(__dirname, "../.."), from your nextConfig to its experimental object as it is still an experimental feature. Also output: undefined won't do a single thing, please use output: "standalone" instead if you want to use Next.js's output file tracing. You may also want to add // @ts-check to the top of your next.config.js so that it warns you when you have an invalid config. I don't really know the issue, but this may help a bit :)

Rafcin commented 1 year ago

@DuCanhGH Good to know; my main app actually uses route groups, so I'll keep an eye on that. Let me see if your solution works magic man; give me one moment, and I'll let you know! I appreciate the help! Also, sorry if my reference to the PR pinged you; my apologies!

DuCanhGH commented 1 year ago

@Rafcin remember to update to the latest canary build :)

king0120 commented 1 year ago

@DuCanhGH @Rafcin So I can report from my end that putting outputFileTracingRoot in experimental and having output: 'standalone' hasn't worked on my end. The build ends up failing with the following error.

 !!! CustomerError: The standalone directory /codebuild/output/src974333853/src/ft-app-nextjs-demo/apps/flashtract-app/dist/.next/standalone is missing a server.js file. This file shoud have been created automatically by NextJS. Make sure you enable output standalone on your next.config.js file or set NEXT_PRIVATE_STANDALONE=true. https://nextjs.org/docs/advanced-features/output-file-tracing#automatically-copying-traced-files

Like mentioned above, the server.js that it looks like the app is looking for is nested inside of a couple other directories based on the name of apps inside my monorepo. I don't know if this helps at all, but I tried using the same NX config to deploy to vercel and things seemed to deploy just fine. But would much rather stay on Amplify, since pretty much everything else I use is AWS.

Rafcin commented 1 year ago

@king0120 @DuCanhGH Just ran into the same issue:

2022-12-14T02:48:29.269Z [ERROR]: !!! CustomerError: The standalone directory /codebuild/output/src291575553/src/fox/apps/help/.next/standalone is missing a server.js file. This file shoud have been created automatically by NextJS. Make sure you enable output standalone on your next.config.js file or set NEXT_PRIVATE_STANDALONE=true. https://nextjs.org/docs/advanced-features/output-file-tracing#automatically-copying-traced-files
DuCanhGH commented 1 year ago

@king0120 wait, can you try removing outputFileTracingRoot? I think it may help as the server.js file is output to .next/standalone/[your app's dir relative to the tracing root dir].

king0120 commented 1 year ago

@DuCanhGH so if I remove the outputFileTracingRoot the result is the 500 error mentioned at the beginning of the thread. The root of the 500 seems to be related to not being able to find the root of the projects node modules. Away from my computer for the evening, but the error message in cloudwatch is something along the lines of not being able to find ‘next/dist’

Rafcin commented 1 year ago

This is what the build ouput looks like when I build the app locally with standalone output enabled and outputFileTracingRoot: path.join(__dirname, "../..") inside apps/help/.next is the server.js file. Screenshot from 2022-12-13 19-25-09

DuCanhGH commented 1 year ago

@king0120 @Rafcin hmm, I think having outputFileTracingRoot makes sense, but it is weird that what seems to be aws-amplify is only checking for .next/standalone/server.js. Can you guys make it check for .next/standalone/[your app's dir relative to your root]/server.js instead? Like, by marking it the entrypoint instead of the aforementioned?

Rafcin commented 1 year ago

https://docs.aws.amazon.com/amplify/latest/userguide/build-settings.html These are the only configuration options you can define when deploying.

DuCanhGH commented 1 year ago

@Rafcin eh, I think aws-amplify should do something about monorepos. Maybe for now you can just symlink server.js from .next/standalone/apps/help to .next/standalone.

Edit: removed the copy suggestion since I think it wouldn't work

Rafcin commented 1 year ago

@DuCanhGH I'll give it a shot tomorrow morning, I think link will be better than copy. I'll let you know how it goes. I'll check in with the Amplify team and let them know about the output.

jbreemhaar commented 1 year ago

Apparently my issue wasn't fixed, so I'm back yay :'( @DuCanhGH @Rafcin Symlinking server.js doesn't work.

Just to be sure I wasn't making a mistake I ran another build that copied the server.js instead of symlinking it. Build success, but obviously it'll not work when deployed.

image

jbreemhaar commented 1 year ago

@hloriana any updates? I think what's actually going on is pretty clear now.

DuCanhGH commented 1 year ago

@jbreemhaar maybe you can try copying a fake server.js that actually invokes your app's server.js? I don't understand how aws-amplify still thinks that that file doesn't exist...

jbreemhaar commented 1 year ago

@DuCanhGH adding a server.js which requires the actual server.js gets built and works serverside but all resources throw 404 errors.

DuCanhGH commented 1 year ago

@jbreemhaar maybe you should try invoking it by using child_process.exec to run node applications/web-admins/server.js instead of requiring it? I think it'd work.

jbreemhaar commented 1 year ago

@DuCanhGH I've switched back to SSG for now because I need to release stuff.

I personally don't feel good about your suggestion though, and have doubts about if it'll actually fix the problem. Amplifys web_compute is a blackbox for me at the moment though.

DuCanhGH commented 1 year ago

@jbreemhaar well, that works too :)

Rafcin commented 1 year ago

I brought this up during office hours, hopefully we'll get some more eyes on it. I wonder, if you just move your app into the base directory and then specify your workspaces as your packages and then just the app itself. Will that fix the issue? I was looking around and the amplify UI uses turbo and yarn workspaces, and it seems they can deploy their app. However, the only difference is that their docks app is in the base directory.

Rafcin commented 1 year ago

I guess the drawback would be that you would have to have all your apps in the base directory instead of in an apps folder if that does work.

DuCanhGH commented 1 year ago

@Rafcin it may work, though I think your repo will look a bit ugly, and I'd go as far as saying a monorepo wouldn't be really needed anymore. Though I have to ask, does my suggestion just not work?

Rafcin commented 1 year ago

@DuCanhGH I have yet to try the symlink method, I thought it didn't work?

DuCanhGH commented 1 year ago

@Rafcin no, I mean the one after that, which is copying a fake server.js into .next/standalone that invokes your actual server.js using child_process.exec.

Rafcin commented 1 year ago

@DuCanhGH let me see if I get this right, what you're saying is after the build process is done I should inject a server.js file where Server . Should be and then that file executes the real file that's nested somewhere in stand alone.

DuCanhGH commented 1 year ago

@Rafcin yeah, pretty much something like that. You can use child_process.spawn if you want live logs as well.

Rafcin commented 1 year ago

@DuCanhGH That makes sense. Lemme make something to eat, I haven't eaten all day and then in like half an hour I'll test it out.

DuCanhGH commented 1 year ago

@Rafcin yeah, enjoy your meal :)

Rafcin commented 1 year ago

@DuCanhGH, so i've been doing some tinkering. That solution has no effect. I also ended up realizing why do I need outputFileTracingRoot, if I don't define that, then the .next folder is correctly created in the project dir, and Amplify sees the file correctly and ends the build. The project builds however, I still get the 500 error.

Question, in your fix #43746 did the 500 error occur only on pages inside groups or did all pages error out to 500?

DuCanhGH commented 1 year ago

@Rafcin that error only happened to pages in Route Groups. I have to ask though, what did aws-amplify say? Did it say that server.js didn't exist?

Rafcin commented 1 year ago

@DuCanhGH So this is a new build system, and I'm not sure how it works as opposed to the SLS package that it used to use; however, if there is no server.js file found, then we'd get the missing server.js file like the above, so since it found it, the build finished. Beyond that, I can't actually find out what's going on, since it's a fully managed system, I can't see any logs for Cloudfront or anything. The only control I have is changing the build steps in the YML config and changing the docker image used in the build between AWS Linux 1 & 2 or a custom image.

DuCanhGH commented 1 year ago

@Rafcin so the server started, but you are still getting the same 500 error right?

Rafcin commented 1 year ago

@DuCanhGH Aye

DuCanhGH commented 1 year ago

@Rafcin can you try requiring that server.js file instead? Like, in your .next/standalone/server.js, you should have require("./apps/helper/server.js");. It's really weird, like, I can get it running just fine locally with monorepos, so it should work with aws-amplify as well. If your assets are returning 404 in your deployment you can also consider copying (or maybe linking) apps/helper/.next/static and apps/helper/public to .next/static and public in your root folder (or apps/helper/.next/standalone/.next/static and apps/helper/.next/standalone/public if that doesn't work). All worked just fine for me locally, but I don't know about aws-amplify so...

I don't know how you got 500 in the first place though...

thijsdaniels commented 1 year ago

@Rafcin @jbreemhaar @king0120 @DuCanhGH I think I have a decent temporary fix :)

The server.js file imports files from "./.next", so that directory also needs to be copied over. If you include this postBuild command it should work. Take care to replace the {appRoot} of course, and note that the trailing . of the source path is important.

postBuild:
  commands:
    - cp -r .next/standalone/{appRoot}/. .next/standalone

@hloriana I hope AWS is OK with implementing something similar internally so that we can eventually remove the cp command on our end. My guess is that it should be relatively straightforward given that the {appRoot} is already part of the config. Either inserting /{appRoot} in the deployment path(s) or moving the files before deploying should do the trick I believe.

I'm not really sure why Next.js actually decided to include the app's directory in the output path. It seems redundant to me, since each app would have its own standalone directory anyway. So ideally, unless I'm wrong about this of course, Next.js would change that on their end at some point. I'll see if I can find/create an issue over at their GitHub and I'll link it here when I do.

DuCanhGH commented 1 year ago

@thijsdaniels I think that may work, if it does then just use it till they fix the issue. Also that probably is intentional, see this line. I'm not sure whether this statement "each app would have its own standalone directory anyway" is correct, maybe you can build in root and have a .next folder there? Idk, haven't tried that yet. Or maybe it is to avoid conflicts with other stuffs Next.js may find in root and copy to .next/standalone, I don't know either.

thijsdaniels commented 1 year ago

@DuCanhGH Ah right! Removing that exact line would also fix our deployment :D But if they added it that explicitly I'm sure they had a good reason.