search

vercel / next.js

The React Framework
https://nextjs.org
MIT License
127.27k stars 27.02k forks source link

'fs' not found using experimental node runtime #35636

Closed selfagency closed 2 years ago

selfagency commented 2 years ago

Verify canary release

Provide environment information

Please run next info in the root directory of your project and paste the results.

um yeah, that doesn't work.

📂 unstoppable-domains-website/client ⋅  dan/frontend-tests (066b118) +1 [$✘!+] ⋅ 📦 v1.0.0 ⋅  v16.13.2 
01:14:10 PM » yarn workspace client next info
yarn workspace v1.22.15
yarn run v1.22.15
$ /Users/daniel/code/ud/unstoppable-domains-website/node_modules/.bin/next info
> No such directory exists as the project root: /Users/daniel/code/ud/unstoppable-domains-website/client/info
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed.
Exit code: 1
Command: /Users/daniel/.nvm/versions/node/v16.13.2/bin/node
Arguments: /Users/daniel/.node/corepack/yarn/1.22.15/lib/cli.js next info
Directory: /Users/daniel/code/ud/unstoppable-domains-website/client
Output:

info Visit https://yarnpkg.com/en/docs/cli/workspace for documentation about this command.

per your cli's own output, the info command doesn't exist.

01:10:56 PM ✖1 » yarn next --help
yarn run v1.22.15
$ /Users/daniel/code/ud/unstoppable-domains-website/node_modules/.bin/next --help

    Usage
      $ next <command>

    Available commands
      build, start, export, dev, lint, telemetry

    Options
      --version, -v   Version number
      --help, -h      Displays this message

    For more information run a command with the --help flag
      $ next build --help

✨  Done in 0.14s.

What browser are you using? (if relevant)

No response

How are you deploying your application? (if relevant)

No response

Describe the Bug

i am using "next": "^12.1.2-canary.0" and "react": "^18.0.0-rc.3". my nextjs config explicitly says to use nodejs and not the edge runtime. however, the build is failing saying that 'fs' can't be found.

  experimental: {
    runtime: 'nodejs',
  },
01:08:29 PM ⏱️ 2m15s » yarn workspace client build
yarn workspace v1.22.15
yarn run v1.22.15
$ NODE_OPTIONS=--max_old_space_size=4096 next build
info  - Loaded env from /Users/daniel/code/ud/unstoppable-domains-website/client/.env
warn  - You have enabled experimental feature(s).
warn  - Experimental features are not covered by semver, and may cause unexpected or broken application behavior. Use them at your own risk.

info  - Skipping validation of types  
warn  - You are using the experimental Node.js Runtime with `experimental.runtime`.
info  - Creating an optimized production build  
Failed to compile.

../node_modules/node-gyp-build/index.js:1:0
Module not found: Can't resolve 'fs'

Import trace for requested module:
../node_modules/sodium-native/index.js
../node_modules/@zilliqa-js/crypto/dist/index.umd.js
./utils/zilliqaLoader.ts
./actions/guestManageActions.ts
./pages/deadhead-collection.page.tsx

https://nextjs.org/docs/messages/module-not-found

> Build failed because of webpack errors
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed.
Exit code: 1
Command: /Users/daniel/.nvm/versions/node/v16.13.2/bin/node
Arguments: /Users/daniel/.node/corepack/yarn/1.22.15/lib/cli.js build
Directory: /Users/daniel/code/ud/unstoppable-domains-website/client
Output:

Expected Behavior

for next to be able to recognize node internals when building for node

To Reproduce

use

  experimental: {
    runtime: 'nodejs',
  },

and import a package that uses node internals

selfagency commented 2 years ago

warn - You are using the experimental Node.js Runtime with experimental.runtime. Module not found: Can't resolve 'fs' Native Node.js APIs are not supported in the Edge Runtime. Found fs imported.

welp that makes a ton of sense. looks like even if you specify the node runtime, it uses the edge runtime.

$ APP_ENV=development NODE_OPTIONS=--max_old_space_size=4096 next
ready - started server on 0.0.0.0:3000, url: http://localhost:3000
info  - Loaded env from /Users/daniel/code/ud/unstoppable-domains-website/client/.env
warn  - You have enabled experimental feature(s).
warn  - Experimental features are not covered by semver, and may cause unexpected or broken application behavior. Use them at your own risk.

warn  - You are using the experimental Node.js Runtime with `experimental.runtime`.
wait  - compiling...
error - ../node_modules/node-gyp-build/index.js:1:0
Module not found: Can't resolve 'fs'

Import trace for requested module:
../node_modules/sodium-native/index.js
../node_modules/@zilliqa-js/crypto/dist/random.js
../node_modules/@zilliqa-js/crypto/dist/schnorr.js
./utils/schnorrSign.ts
./redux/web3.ts
./contexts/web3.tsx
./utils/error.ts
./pages/_app.page.tsx

https://nextjs.org/docs/messages/module-not-found
Native Node.js APIs are not supported in the Edge Runtime. Found `fs` imported.
MauriceArikoglu commented 2 years ago

@selfagency it seems you are trying to use mix features not intended to be mixed. What are you trying to accomplish with the experimental runtime? If it is Streaming, why do you need it? Did you make sure to point the next compiler to your server only files? https://nextjs.org/docs/advanced-features/react-18/server-components#server-components-conventions

From the documentation:

This option determines the environment in which streaming SSR will be happening. When setting to 'edge', the server will be running entirely in the Edge Runtime.

I interprete this as "part of the server is running in edge runtime when specifying 'nodejs' as setting". In Edge Runtime, NodeJS APIs are limited and file system access is prohibited (hence fs will not work)

balazsorban44 commented 2 years ago

Regarding next info, if you are getting an error/indication that this command is not available, you seem to be referencing an older version of Next.js.

The output of next --help on newer versions should look like this:

Usage
      $ next <command>

    Available commands
      build, start, export, dev, lint, telemetry, info # <--

    Options
      --version, -v   Version number
      --help, -h      Displays this message

    For more information run a command with the --help flag
      $ next build --help

(You could try next --version as well)

It might explain your issues with runtime switching as a lot has changed since previous versions. :thinking:

next info was added in 12.0.8: https://github.com/vercel/next.js/releases/tag/v12.0.8

Could you create an isolated reproduction to be sure you are actually using the canary version? Maybe without a workspace first.

selfagency commented 2 years ago

@MauriceArikoglu I don't want to use the experimental runtime. I want to enable swc on my project and am having a hell of a time figuring out how to do so because it's throwing compile errors at me that lead me to GitHub issues that have me trying to do weird stuff I probably shouldn't have to.

@balazsorban44 What I don't understand is, I don't have Next.js installed globally. And I'm completely deleting the node_modules folders in every workspace and running a fresh install, so why would it be pulling down the incorrect version of the cli?

MauriceArikoglu commented 2 years ago

@selfagency swc should be enabled by default, if you are using next > 12. If not, you can update next by running npm install next@latest or the equivalent yarn command.

You don't need to have next installed globally. Please check your package.json dependencies (or devDependencies) for next and its version. You are probably using a local version (which is always reinstalled on npm i, so deleting node_modules does not make any difference

selfagency commented 2 years ago

You missed the compilation errors part. I upgraded to next@latest. Then I tried to build my project and it told me fs couldn't be found. How do I resolve that issue?

MauriceArikoglu commented 2 years ago

@selfagency I can't do remote diagnosis. Make sure you remove the experimental features and start with a clear config. Post relevant files:

balazsorban44 commented 2 years ago

My point from https://github.com/vercel/next.js/issues/35636#issuecomment-1081690834 still stands, we would need a proper reproduction. So far it's not verified what version of Next.js you are using, or how the code actually looks.

I noticed from the error message, it looks like you are using fs in ./pages/deadhead-collection.page.tsx and/or ./pages/_app.page.tsx.

So the issue might be unrelated to the runtime config. Maybe you are using these APIs on the client-side. As browsers also lack fs, the above error will be thrown. In that case, you have to make sure that the libraries you are using are browser compatible.

Could you please link to your code or create a reproduction? Otherwise, we cannot investigate further.

riboher commented 2 years ago

Crossposting this https://github.com/vercel/next.js/issues/36237#issuecomment-1101356231 here, since it seems a related issue. Several Node APIs are making the build fail even though none of them are being used on client side. This has happened after updating to react v18

riboher commented 2 years ago

Any updates on this? Even with the latest next version 12.1.6, my build fails due to some modules not being found because they aren't supported by the edge runtime, even though I'm specifying the experimental runtime to be nodejs and I'm not using any middleware function, so I don't know why that error is being thrown.

Here's my package.json

{
  "name": "fleet-admin",
  "version": "0.1.0",
  "private": true,
  "scripts": {},
  "dependencies": {
    "@material-ui/core": "4.12.4",
    "@next/env": "12.1.5",
    "@product/aker-grpc-client": "0.0.3",
    "@product/auth-client": "0.0.13",
    "@product/fleet-companies-client": "0.0.21",
    "@product/fleet-earnings-client": "0.0.3",
    "@product/formularious-hotkeys": "2.3.2",
    "@product/marshal": "21.1.0",
    "@product/rocky": "4.0.0",
    "@product/skye": "4.0.0",
    "@product/users-grpc-client": "0.0.8",
    "@product/users-search-grpc-client": "0.0.3",
    "cache-manager": "3.6.1",
    "cache-manager-ioredis": "2.1.0",
    "camelcase-keys": "7.0.2",
    "classnames": "2.3.1",
    "core-js": "3.21.1",
    "find-up": "5.0.0",
    "google-spreadsheet": "3.2.0",
    "http-graceful-shutdown": "3.1.7",
    "lodash": "4.17.21",
    "next": "12.1.6",
    "next-auth": "4.3.1",
    "next-connect": "0.12.2",
    "next-http-proxy-middleware": "1.2.4",
    "node-mocks-http": "1.11.0",
    "nprogress": "0.2.0",
    "on-headers": "1.0.2",
    "pino": "7.10.0",
    "pino-http": "6.6.0",
    "prom-client": "14.0.1",
    "react": "18.0.0",
    "react-dom": "18.0.0",
    "react-hook-form": "7.29.0",
    "response-time": "2.3.2",
    "rollbar": "2.24.1",
    "serialize-error": "8.1.0",
    "swr": "1.3.0",
    "threads": "1.7.0",
    "typescript": "4.6.3",
    "url-value-parser": "2.1.0",
    "utility-types": "3.10.0",
    "yaml": "1.10.2"
  },
  "devDependencies": {
    "@apidevtools/swagger-parser": "10.0.3",
    "@cabify/eslint-config": "1.0.2",
    "@product/stylelint-preset": "15.0.0",
    "@product/tsconfig": "15.0.0",
    "@stoplight/prism-cli": "4.8.0",
    "@storybook/addon-essentials": "6.4.21",
    "@storybook/addon-postcss": "2.0.0",
    "@storybook/addons": "6.4.21",
    "@storybook/builder-webpack5": "6.4.21",
    "@storybook/manager-webpack5": "6.4.21",
    "@storybook/preset-scss": "1.0.3",
    "@storybook/react": "6.4.21",
    "@testing-library/jest-dom": "5.16.4",
    "@testing-library/react": "12.1.5",
    "@testing-library/react-hooks": "7.0.2",
    "@testing-library/user-event": "14.1.0",
    "@types/cache-manager": "3.4.3",
    "@types/cache-manager-ioredis": "2.0.2",
    "@types/classnames": "2.3.1",
    "@types/cypress": "1.1.3",
    "@types/google-spreadsheet": "3.2.0",
    "@types/jest": "27.4.1",
    "@types/next-auth": "3.15.0",
    "@types/node": "17.0.23",
    "@types/nprogress": "0.2.0",
    "@types/on-headers": "1.0.0",
    "@types/pino-http": "5.8.1",
    "@types/react": "17.0.44",
    "@types/response-time": "2.3.5",
    "@types/webpack": "5.28.0",
    "@types/yaml": "1.9.6",
    "autoprefixer": "10.4.4",
    "babel-plugin-module-resolver": "4.1.0",
    "camelcase": "6.3.0",
    "concurrently": "7.1.0",
    "cssnano": "5.1.7",
    "cypress": "9.5.4",
    "dotenv-types-generator": "1.1.2",
    "eslint": "8.13.0",
    "eslint-config-next": "12.1.5",
    "eslint-import-resolver-typescript": "2.7.1",
    "eslint-plugin-cypress": "2.12.1",
    "eslint-plugin-jest-dom": "4.0.1",
    "eslint-plugin-sort-keys-fix": "1.1.2",
    "eslint-plugin-testing-library": "5.3.1",
    "eslint-plugin-typescript-sort-keys": "2.1.0",
    "hygen": "6.2.0",
    "inflection": "1.13.2",
    "jest": "27.5.1",
    "jest-environment-jsdom": "27.5.1",
    "jest-environment-jsdom-global": "3.0.0",
    "jest-simple-dot-reporter": "1.0.5",
    "local-ts-repl": "0.1.3",
    "loki": "0.29.0",
    "msw": "0.39.2",
    "msw-storybook-addon": "1.6.3",
    "pino-pretty": "7.6.1",
    "postcss": "8.4.12",
    "postcss-loader": "6.2.1",
    "postcss-plugin-px2rem": "0.8.1",
    "postcss-preset-env": "7.4.3",
    "prettier": "2.6.2",
    "prettier-plugin-package": "1.3.0",
    "prettier-plugin-sort-json": "0.0.2",
    "prettier-plugin-tsconfig": "0.0.1",
    "sass": "1.50.0",
    "sass-loader": "12.6.0",
    "storybook-addon-next-router": "3.1.1",
    "stylelint": "14.6.1",
    "stylelint-config-prettier": "9.0.3",
    "stylelint-order": "5.0.0",
    "stylelint-prettier": "2.0.0",
    "ts-node": "10.7.0",
    "typed-scss-modules": "6.2.0",
    "webpack": "5.72.0",
    "whatwg-fetch": "3.6.2"
  },
  "msw": {
    "workerDirectory": "public"
  },
  "packageManager": "yarn@3.2.0",
  "volta": {
    "node": "16.14.2",
    "yarn": "1.22.18"
  }
}

And my config file

/* eslint-disable import/no-default-export */

// IMPORTANT NOTE: variables defined here are always read at runtime

const fs = require('fs');
const { PHASE_PRODUCTION_BUILD } = require('next/constants');

module.exports = function getConfig(phase = PHASE_PRODUCTION_BUILD) {
  /**
   * @type {import('next').NextConfig}
   */
  const nextConfig = {
    basePath: process.env.BASE_PATH,
    experimental: {
      pageEnv: true,
      runtime: 'nodejs',
    },
    generateEtags: false,
    pageExtensions: ['page.tsx', 'endpoint.ts'],
    poweredByHeader: false,
    publicRuntimeConfig: {},
    reactStrictMode: true,
    async redirects() {},
    serverRuntimeConfig: {
      serviceVersion: process.env.SERVICE_VERSION,
    },
    webpack: (config, { dev, isServer, webpack }) => {
      /* eslint-disable no-param-reassign */
      if (dev) {
        config.ignoreWarnings = [
          { message: /clean: Invalid font values/ },
          { message: /clean: Invalid character\(s\)\s'top:/ },
        ];
      }
      if (isServer) {
        return {
          ...config,
          async entry() {
            const workers = fs
              .readdirSync('src/services/workers')
              .filter((file) => file.endsWith('.worker.ts'));

            return config.entry().then((entry) => {
              const newEntry = { ...entry };
              workers.forEach((worker) => {
                newEntry[
                  `workers/${worker.replace('.ts', '')}`
                ] = `src/services/workers/${worker}`;
              });
              return newEntry;
            });
          },
        };
      }

      config.plugins?.push(
        new webpack.IgnorePlugin({
          resourceRegExp: /fetch-blob$/,
        }),
      );

      config.plugins?.push(
        new webpack.NormalModuleReplacementPlugin(/\.server/, (resource) => {
          resource.request = resource.request.replace('server', 'browser');
        }),
      );
      return config;
    },
  };

  return nextConfig;
};

Thanks everyone!

balazsorban44 commented 2 years ago

This issue has been automatically closed because it received no activity for a month and had no reproduction to investigate. If you think this was closed by accident, please leave a comment. If you are running into a similar issue, please open a new issue with a reproduction. Thank you.

github-actions[bot] commented 2 years ago

This closed issue has been automatically locked because it had no new activity for a month. If you are running into a similar issue, please create a new issue with the steps to reproduce. Thank you.