Next.js 12 Output File Tracing Fails to Build

[Miikis]

Current Behavior

• create a new nx workspace w/ a next.js app
• turn on next.js output file tracing
• run nx build --prod
• build fails w/ the error message saying dist/apps/haha/.next/standalone/apps/haha/server.js doesn't exist (it doesn't)

Expected Behavior

The standalone folder should contain a server.js file.

Steps to Reproduce

I've created a repro repo here. Clone & run yarn nx build --prod to see error message.

Failure Logs

ENOENT: no such file or directory, open '/nx_next_standalone_repro/dist/apps/haha/.next/standalone/apps/haha/server.js'

Environment

Output of next info

    Operating System:
      Platform: darwin
      Arch: arm64
      Version: Darwin Kernel Version 21.2.0: Sun Nov 28 20:29:10 PST 2021; root:xnu-8019.61.5~1/RELEASE_ARM64_T8101
    Binaries:
      Node: 16.13.1
      npm: 8.1.2
      Yarn: 1.22.17
      pnpm: N/A
    Relevant packages:
      next: 12.1.0
      react: 17.0.2
      react-dom: 17.0.2

Output of nx report

   Node : 16.13.1
   OS   : darwin arm64
   yarn : 1.22.17

   nx : 13.8.2
   @nrwl/angular : undefined
   @nrwl/cli : 13.8.2
   @nrwl/cypress : 13.8.2
   @nrwl/detox : undefined
   @nrwl/devkit : 13.8.2
   @nrwl/eslint-plugin-nx : 13.8.2
   @nrwl/express : undefined
   @nrwl/jest : 13.8.2
   @nrwl/js : 13.8.2
   @nrwl/linter : 13.8.2
   @nrwl/nest : undefined
   @nrwl/next : 13.8.2
   @nrwl/node : undefined
   @nrwl/nx-cloud : undefined
   @nrwl/react : 13.8.2
   @nrwl/react-native : undefined
   @nrwl/schematics : undefined
   @nrwl/storybook : 13.8.2
   @nrwl/tao : 13.8.2
   @nrwl/web : 13.8.2
   @nrwl/workspace : 13.8.2
   typescript : 4.5.5
   rxjs : 6.6.7
   ---------------------------------------
   Community plugins:

[Miikis]

Update: I had upgraded the dependencies to latest after the initial workspace creation. Figured it'd be worth seeing if the error appears w/ the default nx workspace creation dependency versions. So I rolled it back. The build still fails but it's a different error message now:

Error occurred prerendering page "/500". Read more: https://nextjs.org/docs/messages/prerender-error
Error: Minified React error #321; visit https://reactjs.org/docs/error-decoder.html?invariant=321 for the full message or use the non-minified dev environment for full errors and additional helpful warnings.
...
Error occurred prerendering page "/404". Read more: https://nextjs.org/docs/messages/prerender-error
Error: Minified React error #321; visit https://reactjs.org/docs/error-decoder.html?invariant=321 for the full message or use the non-minified dev environment for full errors and additional helpful warnings.
...
Error occurred prerendering page "/". Read more: https://nextjs.org/docs/messages/prerender-error
Error: Minified React error #321; visit https://reactjs.org/docs/error-decoder.html?invariant=321 for the full message or use the non-minified dev environment for full errors and additional helpful warnings.

Seems like it's having trouble rendering pages 🤔

[Miikis]

This actually does seem to be a dependency mismatch issue. The last next.js version that output-file-tracing works for is 12.0.7 (which has a known security vulnerability according to snyk).

Does Nx document supported next.js versions anywhere? Just saw an issue asking for support for next.js@12.1.0, so I'm assuming its not officially supported?

[palmithor]

I reproduced a similar issue, including next config for standalone as well, using NX. It'd be very nice if you could see figure out what is wrong (setup or bug?). Thanks in advance.

Next Config:

const nextConfig = {
  nx: {
    // Set this to true if you would like to to use SVGR
    // See: https://github.com/gregberge/svgr
    svgr: false,
  },
  experimental: {
    outputStandalone: true,
    outputFileTracingRoot: path.join(__dirname, '../../'),
  },
};

Readme shows how to reproduce the bug. https://github.com/palmithor/nextjs-nx-standalone-bug

Error:

ENOENT: no such file or directory, open '****/nextjs-nx-standalone-bug/dist/apps/nextjs-nx-standalone-bug/.next/standalone/apps/nextjs-nx-standalone-bug/server.js'

[MagnusSafty]

I am facing this exact issue as well, any updates?

[brandoncroberts]

The Caveats section here documents monorepo

https://nextjs.org/docs/advanced-features/output-file-tracing#caveats It seems that on monorepo setups outputStandalone: true, isn't included on their next.config.js. only outputFileTracingRoot: path.join(__dirname, '../../')

[Miikis]

@plusplushq I read that caveat as being a subsequent step to enabling outputStandalone. When I tried it without outputStandalone, there is no standalone directory generated. Have you tried this and seen it work?

[always-maap]

I tried it today and it's working.

experimental: {
    outputStandalone: true,
    outputFileTracingRoot: path.join(__dirname, '../../')
  },

node apps/web-app/server.js

[Miikis]

@always-maap what's working exactly? Did you try it w/ the repro example above? Did you update any dependencies? That command that you're running at the end node apps/web-app/server.js — is server.js a custom server you've set up? If so, what does that have to do w/ output-tracing? The server.js that's missing above is a file that would normally be generated, on build, in nx's build-output directory.

[always-maap]

@always-maap what's working exactly? Did you try it w/ the repro example above? Did you update any dependencies? That command that you're running at the end node apps/web-app/server.js — is server.js a custom server you've set up? If so, what does that have to do w/ output-tracing? The server.js that's missing above is a file that would normally be generated, on build, in nx's build-output directory.

No, I tested against our org nx project. no, it is not a custom server.

nrwl/next: 13.1.3

[Miikis]

What version of next.js are you using? As far as I can tell, 12.0.7 is the last time output file-tracing worked w/ any version of nx. If that's not the case, do you mind posting a minimal reproduction where it's working?

[always-maap]

What version of next.js are you using? As far as I can tell, 12.0.7 is the last time output file-tracing worked w/ any version of nx. If that's not the case, do you mind posting a minimal reproduction where it's working?

We are using 12.1.0

[Miikis]

just tried dowgrading the repro nx repo above to use 13.1.3 (next.js is already at 12.1.0. same error: it can't find .next/standalone/apps/<app-name>/server.js

[palmithor]

For those running next.js in docker this is a groundbreaking feature so I would be very happy to have this issue resolved. The docker image will reduce in size by a lot.

In our case it will also reduce the build time very much as we are doing what we can to have the docker image as small as possible, and in order to achieve that we have to do npm install twice, first for the build, and then again for the generated package.json file for this app.

[Miikis]

I looked into this a bit more last weekend. It looks like the root-cause might be a race-condition of some sort. The step that writes the tiny server to the server.js file located here is throwing an error because it thinks that file hasn't been created yet.

On next.js' side, output-file-tracing is working fine (if you create-next-app and enable outputFileTracing, everything works as expected). So I'm thinking somewhere around this step, something in nx is maybe removing that server.js file.

Hope that helps.

[Miikis]

One thing that's interesting: at one point I hard-coded the distDir value as an absolute-path inside the next.js codebase (for that build process), and the correct standalone output was generated. Albeit, in the wrong directory lol it put the build output in apps/<my-app>/<absolute-path-from-home-dir>/<dist-folder-with-correct-standalone>.

[palmithor]

Seems like this isn't gaining any focus? 😔

[cocamm]

Changing the outputPath in production configuration works for me:

"configurations": { "production": { "outputPath": "apps/<my-app>/dist" } } Or

npx nx run my-app:build --configuration=production --outputPath="apps/<my-app>/dist"

[Miikis]

@cocamm how are you getting that to work for you? doesn't that output the build in your app's directory, where the standalone bundle is several layers deep? are you wrapping the next/build executor in a run-command executor and shuffling that bundle around till everything's in its place?

[cocamm]

@cocamm how are you getting that to work for you? doesn't that output the build in your app's directory, where the standalone bundle is several layers deep? are you wrapping the next/build executor in a run-command executor and shuffling that bundle around till everything's in its place?

Yes, you are right. But, I will run this command just in the CI step of my workflow and I've made some changes in my Dockerfile to get just the files from this folder.

I know it is not the best solution, but it was the only way that I could get this working.

[pstoica]

Hi all, I was wondering if there was an official fix for this. I'm currently leaving the build inside the Next.js directory (I just added Nx; not using @nrwl/next), but I was able to get it working with this Dockerfile (app is named dashboard):

  experimental: {
    outputStandalone: true,
    outputFileTracingRoot: path.join(__dirname, "../../"),
  },

# Automatically leverage output traces to reduce image size 
# https://nextjs.org/docs/advanced-features/output-file-tracing
COPY --chown=nextjs:nodejs /packages/dashboard/.next/standalone ./
COPY --chown=nextjs:nodejs /packages/dashboard/.next/static ./packages/dashboard/.next/static
COPY /packages/dashboard/public ./packages/dashboard/public
# These are partially installed for some reason. Remove or it won't work.
RUN rm -rf ./packages/dashboard/node_modules/react
RUN rm -rf ./packages/dashboard/node_modules/next

...

CMD ["node", "./packages/dashboard/server.js"]

Pretty weird combination of folders/files, but I hope it helps someone in the meantime, or reveals the source of the issue.

[always-maap]

I post mine maybe it helps

experimental: {
    outputStandalone: true,
    outputFileTracingRoot: path.join(__dirname, '../../'),
  },

# Dockerfile/zoomit-editorial/Dockerfile

FROM docker-repo.url.com/node:v-alpine

COPY dist/apps/zoomit-editorial/.next/standalone app/
# TODO: serve with CDN
COPY dist/apps/zoomit-editorial/public/ app/apps/zoomit-editorial/public/
COPY dist/apps/zoomit-editorial/.next/static app/dist/apps/zoomit-editorial/.next/static/

WORKDIR /app

ENV NODE_ENV production
ENV PORT 4200

CMD ["node", "apps/zoomit-editorial/server.js"]

[h4yha]

any update guys? i'm still getting this error even in the latest version

[ghost]

Error still exists in 13.9.5

[palmithor]

And in 13.9.7, reproduce the issue by cloning the following repository and run

npm install
npm run build

Errror message:

[MarcieMarc425]

Encountering same issue on Vercel with @nrwl/next 13.10.1 and next 12.1.4. Not sure if I should still continue Next development with nx considering this is pretty much breaking for me on UAT and PROD envs.

[amized]

Somewhat related but I'm stuck with an issue running a build command for next js. Config in package json is:

"nx": {
    "type": "application",
    "targets": {
      "build": {
        "options": {},
        "configurations": {
          "production": {
            "outputPath": "apps/consumer-app/dist"
          }
        }
      }
    }
  },

When I run

yarn nx run consumer-app:build --configuration=production

I get the following error:

> nx run consumer-app:build:production

$ next build --outputPath=apps/consumer-app/dist
Unknown or unexpected option: --outputPath
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

For some reason nx is passing the outputPath cli option into the next build command. Can anyone suggest how I can modify the output path for next js builds?

[LudovicPelleDMS]

First of all thank you for your work, so I also came across the same problem, I was able to find a Hack similar to the comments, it is indeed a problem of Path between Nx (root, outputPath) and the standalone of next,

"targets": {
  "build": {
      "executor": "@nrwl/next:build",
      "outputs": ["{options.outputPath}"],
      "defaultConfiguration": "production",
      "options": {
          "root": "apps/terminal",
          "outputPath": "apps/terminal"
      },

and

    experimental: {
        outputStandalone: true,
        outputFileTracingRoot: path.join(__dirname, "../../")
    }

finish with

this standalone functionality is a real "plus value" for the production of minimal image (docker, k8s, ...) I hope you will be able to look into it quickly, thank you

[fllprbt]

@always-maap works. awesome!

"next": "12.0.7",
"@nrwl/next": "13.7.3"

[ctrlx-altf4]

changing the output path did the trick. Thanks @LudovicPelleDMS

[westmark]

Thanks @LudovicPelleDMS - getting build output now. However, I also get an error message

Source and destination must not be the same.

and the build exits with error code 1. Are you also getting this? Not ideal for CI. Fix can't arrive soon enough

[LudovicPelleDMS]

Thanks @LudovicPelleDMS - getting build output now. However, I also get an error message

Source and destination must not be the same.

and the build exits with error code 1. Are you also getting this? Not ideal for CI. Fix can't arrive soon enough

Yeah, i got this message too, as i said it's a "hack" , and an experimental feature , deploy on production may be risky (for now it's seems ok for my project , cypress cover my back ^^)

Thanks to @pstoica solution's, you can have the good return code then moving the "weird" folder structure (move static and public ) cf @pstoica Dockerfile

[bboyz269]

It seems this issue won't be resolved for quite some time. I created a sample repo nx-next-i18next-standalone building a minimal docker image that works with multiple apps nx workspace. Thanks to all comments from this thread!
There's also next-i18next integration if you're interested.

[dukemai]

I have the same problem on Mac with "@nrwl/next": "14.1.7" and "next": "12.1.5". In my case I can generate if I update next.config.js as following: outputFileTracingRoot: join(__dirname, './dist/apps/web-app'), the server.js is generated directly under /dist which is I dont really want

[Yizhachok]

Just don't ask how I investigated this and how much time I spend to find out how to fix this. ⚠️ WARNING ⚠️ Next text for people who believe in magic, skeptics, please stop read this text right now. I warned you.

Steps to get working repo for standalone Next.js server with build using NX CLI.

Init example workspace (or use your own)

1. Run npx create-nx-workspace@latest --preset=next (tested on v14.1.9 workspace)
2. Answers to questions: ✔ Workspace name (e.g., org name): next-test ✔ Application name: site ✔ Default stylesheet format: scss
3. Go to next.config.js file and add to config:

   experimental: {
   outputStandalone: true,
   outputFileTracingRoot: path.join(__dirname, "../../")
   }

Preparation finished, now let's make real magic

Go to pages/index.tsx (or any page file) and add this lines (don't ask why, this is MAGIC):

import path from 'path';
path.resolve('./next.config.js');

Really can be any js file that can be resolved from current location (even empty). This code enough to insert only once.

Show magic to the world

Run npm run build. That's it! No errors!

Bonus

This is a VERY SIMPLE Docker config that show how to work with this build.

FROM node:16-alpine AS builder
RUN apk add --no-cache libc6-compat python3 make g++
WORKDIR /build
COPY . .
RUN npm i && npm run build

FROM node:16-alpine AS runner
ARG APP=site
WORKDIR /app

RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs

COPY --from=builder --chown=nextjs:nodejs /build/dist/apps/${APP}/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /build/dist/apps/${APP}/.next/static ./dist/apps/${APP}/.next/static
COPY --from=builder --chown=nextjs:nodejs /build/dist/apps/${APP}/public ./apps/${APP}/public

USER nextjs

EXPOSE 3000

ENV NODE_ENV=production
ENV APP_PATH=apps/${APP}/server.js
ENV PORT 3000

CMD node $APP_PATH

Thanks to @bboyz269 for working example, from his repository I found the issue why build didn't work on clean project. His repo works because of next-i18next lib code.

[westmark]

@Yizhachok I tried your magic and it actually worked! Then I wondered, what was the original error message? Because I had forgotten. So I removed the import and resolve in index.tsx and... the build worked again! The only other change I made was to change outputPath to dist/apps/myapp. Just to be sure, I removed the dist folder and the NX cache folder, and the build still worked.

I'm so confused right now.

[Yizhachok]

@westmark I have tested this on a new project, going through steps that described in first message and using rm -rf ./node_modules/.cache/nx && npm run build command (and then on my project). Actually, npm run build must execute nx build. Main hint as I found out is in running path.resolve with existing file name. If you do not add path.resolve then you will see the error described in the first message of this issue. I think this is related somehow to babel process and what it adds when see some dependencies (because if you will try to call path.join this will not work).

[westmark]

@Yizhachok Yes, I also tried it on a fresh project, and it absolutely works. An interesting tidbit is that resolving ./next.config.js makes the build succeed, but resolving ../next.config.js, i.e the correct path relative to pages/index.tsx, fails the build.

As for my own project I'm guessing I accidentally "solved" this problem in a similar fashion along the way without realizing it, since I was still using the outputPath-hack so the original error was obscured.

[Yizhachok]

@westmark you can even try to resolve ./1.js file and create empty one near next.config.js and this still will be work. Very interesting behavior and if somebody can use this information to find the root of the problem this will be great. For now for me this solution is enough.

[ShadiBlitz]

@Yizhachok solution worked for me. However I had to add some additional options because I got this error FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory during build time.

These were the changes :

in pages/index.tsx

const { resolve } = require('path')
resolve('./next.config.js')

in next.config.js

const { join } = require('path')

experimental: {
    outputStandalone: true,
    outputFileTracingRoot: join(__dirname, '../../'),
    esmExternals: false,
  },

and update next in package.json to canary

"next": "12.2.0",

And the docker build worked great. However, an official solution has to be done on NX side.. @ndcunningham

[dwfee]

Hooray! I almost works! Did what ShadiBlitz stated, but since there was a new version out I picked "next": "^12.2.0", to confirm things work I copied the files to my standalone server and tried with docker. Result: it would work if all static files could be read. The files are actually there. Files concerned: css generated from scss, images from optimizedImages and so on. Has somebody experienced the same and found a fix?

Me stupid, just in case somebody is running into the same issue: I have manually set the output dir in next.config.js to another dir, this is where the error came from. I removed it and now everything works as expected !!!

[moatorres]

Thanks everyone for your comments on this issue!

Wrapping up here what worked for me, as it took me a few hours to get it done. Make sure to change your-app to your app/project's name. The steps below consider that your workspace's layout is the default "empty" with apps/ and libs/ folders in the workspace root directory.

A few considerations:

• Good Not having to download all the dependencies inside Docker build stages, particularly when your app depends on publishable libraries that are mantained inside the workspace and may have been updated but not published yet.
• Maybe not good Not sure how the "ideal" workflow for a CI would be or how it would fit in setups like the ones described in this article from Creotip and this other one on Medium.

Build and deploy a Next.js app in standalone mode from an Nx workspace with Docker

See more

`package.json` — for reference ```json "next": "12.1.6", "@nrwl/next": "14.4.0" ``` `workspace.json` — for reference ``` { "$schema": "./node_modules/nx/schemas/workspace-schema.json", "version": 2, "projects": { "your-app": "apps/your-app", } } ``` `app/your-app/project.json` — for reference only, generated with `yarn nx g @nrwl/next:application your-app` ```json { "$schema": "../../node_modules/nx/schemas/project-schema.json", "sourceRoot": "apps/your-app", "projectType": "application", "targets": { "build": { "executor": "@nrwl/next:build", "outputs": ["{options.outputPath}"], "defaultConfiguration": "production", "options": { "root": "apps/your-app", "outputPath": "dist/apps/your-app" }, "configurations": { "development": {}, "production": {} } }, } } ``` 1. To ensure it builds with `outputStandalone: true`, add `path.resolve('./next.config.js')` to `apps/your-app/pages/index.tsx` as suggested by @Yizhachok ```tsx import path from 'path' // ← add this line import { useTheme } from '@my-lib/themes' path.resolve('./next.config.js') // ← and this line const Home = () => { const { theme } = useTheme() ... ``` 2. Update your `apps/your-app/next.config.js` as suggested by @LudovicPelleDMS ```js /** * @type {import('@nrwl/next/plugins/with-nx').WithNxOptions} **/ const nextConfig = { nx: { svgr: false, }, // add the lines below ↓ experimental: { outputStandalone: true, outputFileTracingRoot: path.join(__dirname, '../../'), } } ``` 3. Run `yarn nx build your-app` 4. Create a `Dockerfile` in the workspace root directory (I haven't managed to get it working anywhere else yet) as suggested by @always-maap ```Dockerfile FROM node:16-alpine ENV NODE_ENV=production COPY dist/apps/your-app/.next/standalone app/ COPY dist/apps/your-app/public/ app/apps/your-app/public/ COPY dist/apps/your-app/.next/static app/dist/apps/your-app/.next/static/ WORKDIR /app EXPOSE 3000 ENV PORT 3000 CMD ["node", "apps/your-app/server.js"] ``` 5. Run `docker buildx build -t ghcr.io/your-github-username/your-app .` 6. Run `docker run -p 3000:3000 ghcr.io/your-github-username/your-app` 7. Open `http://localhost:3000` and check if it works.

[krissetto]

Has anyone else noticed issues when it comes to the automatic environment variable expansion in the .env files when using the outputStandalone and outputFileTracingRoot options?

Example from the docs for clarity:

# .env
HOSTNAME=localhost
PORT=8080
HOST=http://$HOSTNAME:$PORT

Everything else seems ok, but my pages are dying because of incorrect configuration values (my env vars seem to be passed along as simple strings like $HOSTNAME instead of actually being substituted)

[trulysinclair]

@moatorres this works but unfortunately, I'm thinking of just waiting until I can use Next 12.2 in a Docker setup with Nx. Hopefully an official fix will come out soon.

[enchorb]

The steps by @moatorres work great except I had to change:

const nextConfig = {
  nx: {
    svgr: false,
  },
  // add the lines below ↓
  experimental: {
    outputStandalone: true,
    outputFileTracingRoot: path.join(__dirname, '../../'),
  }
}

to:

const nextConfig = {
  nx: {
    svgr: false,
  },
  // add the lines below ↓
  output: 'standalone',
  experimental: {
    outputFileTracingRoot: path.join(__dirname, '../../'),
  }
}

[AlexandreCassagne]

Preparation finished, now let's make real magic

Go to pages/index.tsx (or any page file) and add this lines (don't ask why, this is MAGIC):

import path from 'path';
path.resolve('./next.config.js');

Can confirm this fixed everything for me 😂

[raymondKelly]

Unfortunately the above workaround did not work for me while using nx: 14.5.15 and next:12.2.3. I'm just going to go without the new functionality for now until their is support for such with nx.

[bojanbass]

Unfortunately, none of workarounds work with nx 14.8.1 and next 12.3.1. I've also tried to debug Next CLI and change the output folder, but it looks like .nft files are not generated correctly. For sure the bug is present in NX and should be fixed, as Next standalone works fine.

[eglove]

I fixed this problem a few days ago by disabling NX build cache.

npx nx build my-app --prod --skip-nx-cache

I haven't had the issue since I started doing this.

[bojanbass]

Skipping NX cache shouldn't make any difference. Are you sure you're building standalone version of Next, have some other configuration in place or use an older version of Next?

[eglove]

Shouldn't but does. This issue is not specific to a standalone version. Build files are simply not being found on the initial build, but if you rerun it, it works.

Skipping cache, it works all of the time.