vercel / next.js

The React Framework
https://nextjs.org
MIT License
126.95k stars 26.98k forks source link

Missing webpack dependency when using output standalone + custom server #64031

Open maximelebastard opened 7 months ago

maximelebastard commented 7 months ago

Link to the code that reproduces this issue

https://github.com/maximelebastard/nextjs-issue-custom-server

To Reproduce

The issue is related to using the output: "standalone" with a custom server. See example repository.

  1. Clone the reproducing repository
  2. Run npm install then npm run build and npm run start
  3. When starting the app, an error will be triggered due to a missing webpack file on a webpack require (see the error below)
  4. To see the code triggering this issue, see this commit: https://github.com/maximelebastard/nextjs-issue-custom-server/commit/c47de3a03c6694ac760b748ed2fecc28b97bb2e5

Current vs. Expected behavior

Expected:

  1. I create a recent NextJS project (14.1.4) using a recent Node version (v20.12)
  2. I want to deploy my app using Docker, therefore I use the output: "standalone" configuration in next.config.js
  3. I want to use some logging middlewares, therefore I use a custom server.js
  4. When I run npm run build and copy my custom server code to the build dir, the app is properly built and the file is properly copied
  5. When requiring next from server.js, all the dependencies are present and the server starts

Actual:

  1. 1-4 are the same as expected
  2. .next/standalone/node_modules/next/dist/compiled/webpack/build5.js does not exists in the built app
  3. When requiring next from server.js, I'm getting the error below because webpack tries to require build5.js
> node .next/standalone/custom-server.js

node:internal/modules/cjs/loader:1143
  const err = new Error(message);
              ^

Error: Cannot find module './bundle5'
Require stack:
- /Users/maxime/workspace/issue-bundle5-nextjs/.next/standalone/node_modules/next/dist/compiled/webpack/webpack.js
- /Users/maxime/workspace/issue-bundle5-nextjs/.next/standalone/node_modules/next/dist/server/config-utils.js
- /Users/maxime/workspace/issue-bundle5-nextjs/.next/standalone/node_modules/next/dist/server/config.js
- /Users/maxime/workspace/issue-bundle5-nextjs/.next/standalone/node_modules/next/dist/server/next.js
- /Users/maxime/workspace/issue-bundle5-nextjs/.next/standalone/custom-server.js
    at Module._resolveFilename (node:internal/modules/cjs/loader:1143:15)
    at /Users/maxime/workspace/issue-bundle5-nextjs/.next/standalone/node_modules/next/dist/server/require-hook.js:55:36
    at Module._load (node:internal/modules/cjs/loader:984:27)
    at Module.require (node:internal/modules/cjs/loader:1231:19)
    at mod.require (/Users/maxime/workspace/issue-bundle5-nextjs/.next/standalone/node_modules/next/dist/server/require-hook.js:65:28)
    at require (node:internal/modules/helpers:179:18)
    at exports.init (/Users/maxime/workspace/issue-bundle5-nextjs/.next/standalone/node_modules/next/dist/compiled/webpack/webpack.js:24:28)
    at loadWebpackHook (/Users/maxime/workspace/issue-bundle5-nextjs/.next/standalone/node_modules/next/dist/server/config-utils.js:18:5)
    at loadConfig (/Users/maxime/workspace/issue-bundle5-nextjs/.next/standalone/node_modules/next/dist/server/config.js:642:46)
    at initialize (/Users/maxime/workspace/issue-bundle5-nextjs/.next/standalone/node_modules/next/dist/server/lib/router-server.js:55:46) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/Users/maxime/workspace/issue-bundle5-nextjs/.next/standalone/node_modules/next/dist/compiled/webpack/webpack.js',
    '/Users/maxime/workspace/issue-bundle5-nextjs/.next/standalone/node_modules/next/dist/server/config-utils.js',
    '/Users/maxime/workspace/issue-bundle5-nextjs/.next/standalone/node_modules/next/dist/server/config.js',
    '/Users/maxime/workspace/issue-bundle5-nextjs/.next/standalone/node_modules/next/dist/server/next.js',
    '/Users/maxime/workspace/issue-bundle5-nextjs/.next/standalone/custom-server.js'
  ]
}

Provide environment information

Operating System:
  Platform: darwin
  Arch: arm64
  Version: Darwin Kernel Version 23.1.0: Mon Oct  9 21:27:24 PDT 2023; root:xnu-10002.41.9~6/RELEASE_ARM64_T6000
Binaries:
  Node: 20.12.0
  npm: 10.5.0
  Yarn: N/A
  pnpm: N/A
Relevant Packages:
  next: 14.1.4
  eslint-config-next: N/A
  react: 18.2.0
  react-dom: 18.2.0
  typescript: N/A
Next.js Config:
  output: standalone

Which area(s) are affected? (Select all that apply)

Standalone mode (output: "standalone")

Which stage(s) are affected? (Select all that apply)

next start (local)

Additional context

A workaround I found is to disable standalone output. That probably disables tree shaking and relies on the complete node_modules directory. A big drawback is that it makes a heavy docker image.

cs-miller commented 6 months ago

I'm seeing the same issue

Edit by maintainer bot: Comment was automatically minimized because it was considered unhelpful. (If you think this was by mistake, let us know). Please only comment if it adds context to the issue. If you want to express that you have the same problem, use the upvote 👍 on the issue description or subscribe to the issue for updates. Thanks!

clementzhao commented 6 months ago

I have same issue. And I found the server.js which compiled with standalone can works. So I just copy these 2lines, and my custom server.js also works fine.

const nextConfig = {which from compiled server.js}
process.env.__NEXT_PRIVATE_STANDALONE_CONFIG = JSON.stringify(nextConfig)
hi-zp commented 6 months ago

Here's my code, and while it works successfully, I'm not sure it's going to be a hidden problem

import { createServer } from 'node:http';
import { parse } from 'node:url';
import NextServer from 'next/dist/server/next-server';
// @ts-ignore
import { config } from './.next/required-server-files.json';
import { NextConfig } from 'next';

const port = parseInt(process.env.PORT || '9000', 10);
const hostname = '0.0.0.0';
const dev = process.env.NODE_ENV !== 'production';

const app = new NextServer({
  hostname: 'localhost',
  port,
  dir: process.cwd(),
  dev,
  conf: {
    ...(config as NextConfig),
  },
});

const handle = app.getRequestHandler();

app.prepare().then(() => {
  createServer((req, res) => {
    try {
      const parsedUrl = parse(req.url!, true);
     handle(req, res, parsedUrl);
    } catch (err) {
      console.error('Error occurred handling', req.url, err);
      res.statusCode = 500;
      res.end('internal server error');
    }
  })
    .once('error', (err) => {
      logger.error(err);
      process.exit(1);
    })
    .listen(port, () => {
      console.log(
        `> Server listening at http://${hostname}:${port} as ${
          dev ? 'development' : process.env.NODE_ENV
        }`,
      );
    });
});
wq93 commented 6 months ago

I've met, too, frustrating

Node.js v18.20.2
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
yarn run v1.22.19
$ NODE_ENV=production node ./server/main.js
node:internal/modules/cjs/loader:1140
  const err = new Error(message);
              ^

Error: Cannot find module './bundle5'
Require stack:
yunsii commented 4 months ago

Already 3 months passed, there is still no any progress? 🫠

yunsii commented 4 months ago

Even use v14.2.4 still has the same error...

Edit by maintainer bot: Comment was automatically minimized because it was considered unhelpful. (If you think this was by mistake, let us know). Please only comment if it adds context to the issue. If you want to express that you have the same problem, use the upvote 👍 on the issue description or subscribe to the issue for updates. Thanks!

nectoscorp commented 4 months ago

I confirmed that this works for me: https://github.com/vercel/next.js/issues/64031#issuecomment-2078708340 So AWS lambda, I am using the following to extract the JSON: sed -n 12p .next/standalone/server.js | cut -c 20-

then require it inside my lambda handler file.

If I had more time I will figure out which particular parameter in the config made it work by commenting them out one at a time.

yunsii commented 4 months ago

I have same issue. And I found the server.js which compiled with standalone can works. So I just copy these 2lines, and my custom server.js also works fine.

const nextConfig = {which from compiled server.js}
process.env.__NEXT_PRIVATE_STANDALONE_CONFIG = JSON.stringify(nextConfig)

Thanks @nectoscorp

After I tested, it works 😂

nectoscorp commented 4 months ago

I wish the nextConfig json was just saved on a file in like .next/standalone/next.config.json so we don't have to resort to these hacks. But obviously, the best is solution is to find out why bundle5 couldn't be found.

Brandawg93 commented 2 months ago

Having the same issue. This seems to work well for me to allow for both local dev development and docker deployment:

const dev = process.env.NODE_ENV !== 'production'

if (!dev) {
  // eslint-disable-next-line @typescript-eslint/no-require-imports
  const { config } = require('./.next/required-server-files.json')
  process.env.__NEXT_PRIVATE_STANDALONE_CONFIG = JSON.stringify(config)
}
flawnn commented 1 month ago

Having the same issue. This seems to work well for me to allow for both local dev development and docker deployment:

const dev = process.env.NODE_ENV !== 'production'

if (!dev) {
  // eslint-disable-next-line @typescript-eslint/no-require-imports
  const { config } = require('./.next/required-server-files.json')
  process.env.__NEXT_PRIVATE_STANDALONE_CONFIG = JSON.stringify(config)
}

The ES6-conform version:

import("./.next/required-server-files.json")
        .then(({ default: { config } }) => {
        process.env.__NEXT_PRIVATE_STANDALONE_CONFIG = JSON.stringify(config);
        const app = next({ dev });
        return app.prepare().then(() => startServer(app) // startup func);
    })
        .catch((error) => {
        console.error("Failed to load required-server-files.json:", error);
        process.exit(1);
    });
SujithPriyamRajan commented 1 month ago

I have the same issue but surprisingly in PM2 as local & VM's was working fine

Can anyone summarize why this issue is ??

/** @type {import('next').NextConfig} */

const nextConfig = {
    output: 'standalone',
    typescript: {
        ignoreBuildErrors: true
    },
    images: { unoptimized: true },
    experimental: {
        serverActions: {
            serverActions: true
        },
        missingSuspenseWithCSRBailout: false
    },
    distDir: 'out',
    rewrites: async () => {
        return [
            {
                source: '/_next/static/:path*',
                destination: '/static/:path*'
            }
        ];
    }
};
export default nextConfig;
aczekajski commented 1 month ago

I found a universal way if adding missing dependencies to standalone output that I guess can also be used for custom server.

You can have a look here: https://github.com/vercel/next.js/discussions/66327#discussioncomment-10889930