Nextjs app fails to find next.config.js when exports_directories_only=True

[ewianda]

🐞 bug report

Affected Rule

npm_install

The issue is caused by the rule: ### Is this a regression? No it is not

Description

getConfig function of nextjs returns on undefined when exports_directories_only=True

A clear and concise description of the problem... ## 🔬 Minimal Reproduction clone https://github.com/ewianda/bazel-next `baze build //.next` ``` INFO: Analyzed target //:.next (0 packages loaded, 0 targets configured). INFO: Found 1 target... INFO: From Action .next: warn - No ESLint configuration detected. Run next lint to begin setup info - Checking validity of types... info - Creating an optimized production build... info - Compiled successfully info - Collecting page data... { publicRuntimeConfig: { TEST: 'Test' }, serverRuntimeConfig: {} } { publicRuntimeConfig: { TEST: 'Test' }, serverRuntimeConfig: {} } info - Generating static pages (0/4) info - Generating static pages (1/4) info - Generating static pages (2/4) { serverRuntimeConfig: {}, publicRuntimeConfig: { TEST: 'Test' } } { serverRuntimeConfig: {}, publicRuntimeConfig: { TEST: 'Test' } } info - Generating static pages (3/4) info - Generating static pages (4/4) info - Finalizing page optimization... Page Size First Load JS ┌ ○ / 2.88 kB 74.4 kB ├ /_app 0 B 71.5 kB ├ ○ /404 194 B 71.7 kB └ ○ /posts/first-post 2.78 kB 74.3 kB + First Load JS shared by all 71.5 kB ├ chunks/framework-91d7f78b5b4003c8.js 42 kB ├ chunks/main-eab312c0bf2a7270.js 28.2 kB ├ chunks/pages/_app-73483fad2904193b.js 508 B ├ chunks/webpack-514908bffb652963.js 770 B └ css/ccfdf9d4db962a53.css 272 B ○ (Static) automatically rendered as static HTML (uses no initial props) Target //:.next up-to-date: bazel-bin/.next INFO: Elapsed time: 8.450s, Critical Path: 8.35s INFO: 2 processes: 1 internal, 1 linux-sandbox. INFO: Build completed successfully, 2 total actions ``` `bazel build //:.broken_next` ``` INFO: Analyzed target //:.broken_next (0 packages loaded, 0 targets configured). INFO: Found 1 target... ERROR: /home/ewianda/projects/next-bazel/BUILD.bazel:28:12: Action .broken_next failed: (Exit 1): next.sh failed: error executing command bazel-out/host/bin/external/npm_breaks/next/bin/next.sh build '--bazel_node_modules_manifest=bazel-out/k8-fastbuild/bin/_.broken_next.module_mappings.json' Use --sandbox_debug to see verbose messages from the sandbox warn - No ESLint configuration detected. Run next lint to begin setup > Build error occurred TypeError: Cannot read property 'publicRuntimeConfig' of undefined at Object.363 (/home/ewianda/.cache/bazel/_bazel_ewianda/4b31098c41477cc5850b10bfee232ae4/sandbox/linux-sandbox/20/execroot/create_react_app/.next/server/chunks/363.js:68:47) at __webpack_require__ (/home/ewianda/.cache/bazel/_bazel_ewianda/4b31098c41477cc5850b10bfee232ae4/sandbox/linux-sandbox/20/execroot/create_react_app/.next/server/webpack-runtime.js:25:42) at Object.171 (/home/ewianda/.cache/bazel/_bazel_ewianda/4b31098c41477cc5850b10bfee232ae4/sandbox/linux-sandbox/20/execroot/create_react_app/.next/server/pages/posts/first-post.js:20:76) at __webpack_require__ (/home/ewianda/.cache/bazel/_bazel_ewianda/4b31098c41477cc5850b10bfee232ae4/sandbox/linux-sandbox/20/execroot/create_react_app/.next/server/webpack-runtime.js:25:42) at __webpack_exec__ (/home/ewianda/.cache/bazel/_bazel_ewianda/4b31098c41477cc5850b10bfee232ae4/sandbox/linux-sandbox/20/execroot/create_react_app/.next/server/pages/posts/first-post.js:176:39) at /home/ewianda/.cache/bazel/_bazel_ewianda/4b31098c41477cc5850b10bfee232ae4/sandbox/linux-sandbox/20/execroot/create_react_app/.next/server/pages/posts/first-post.js:177:74 at Function.__webpack_require__.X (/home/ewianda/.cache/bazel/_bazel_ewianda/4b31098c41477cc5850b10bfee232ae4/sandbox/linux-sandbox/20/execroot/create_react_app/.next/server/webpack-runtime.js:108:21) at /home/ewianda/.cache/bazel/_bazel_ewianda/4b31098c41477cc5850b10bfee232ae4/sandbox/linux-sandbox/20/execroot/create_react_app/.next/server/pages/posts/first-post.js:177:47 at Object. (/home/ewianda/.cache/bazel/_bazel_ewianda/4b31098c41477cc5850b10bfee232ae4/sandbox/linux-sandbox/20/execroot/create_react_app/.next/server/pages/posts/first-post.js:180:3) at Module._compile (internal/modules/cjs/loader.js:1072:14) { type: 'TypeError' } info - Checking validity of types... info - Creating an optimized production build... info - Compiled successfully info - Collecting page data... undefined Target //:.broken_next failed to build Use --verbose_failures to see the command lines of failed build steps. INFO: Elapsed time: 4.282s, Critical Path: 4.19s INFO: 2 processes: 2 internal. FAILED: Build did NOT complete successfully ```

🔥 Exception or Error

🌍 Your Environment

Operating System:

  
Ubuntu 20.04.3 LTS
  

Output of bazel version:

  
Build label: 5.0.0
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Wed Jan 19 14:08:54 2022 (1642601334)
Build timestamp: 1642601334
Build timestamp as int: 1642601334

  

Rules_nodejs version:

(Please check that you have matching versions between WORKSPACE file and @bazel/* npm packages.)

  
4.3.0
5.0.0
  

Anything else relevant?

[alexeagle]

Thanks for the repro.

First, your "working" case doesn't produce any outputs, because you're running next in the source tree (even though the comment there acknowledges that it has to run in the output tree.)

See https://docs.aspect.build/bazelbuild/rules_nodejs/4.5.1/docs/builtins.html#npm_package_bin-chdir

This fixes it:

diff --git a/BUILD.bazel b/BUILD.bazel
index 6e3e8a5..00429bb 100644
--- a/BUILD.bazel
+++ b/BUILD.bazel
@@ -1,3 +1,4 @@
+load("@build_bazel_rules_nodejs//:index.bzl", "copy_to_bin")
 load("@npm//next:index.bzl", "next")
 load("@npm_breaks//next:index.bzl", broken_next="next")

@@ -6,14 +7,9 @@ load("@npm_breaks//next:index.bzl", broken_next="next")
 # As a workaround, we can run it with a working directory in the output folder
 # so it produces a directory in the place bazel expects.

-next(
-    # Note: this must be named ".next" since Next CLI hard-codes that as the output dir
-    name = ".next",
-    args = [
-        "build",
-    ],
-    chdir = package_name(),
-    data = glob([
+copy_to_bin(
+    name = "workaround",
+    srcs = glob([
         "components/*",
         "pages/**",
         "public/**",
@@ -22,6 +18,14 @@ next(
         "package.json",
         "next.config.js",
     ],
+)
+
+next(
+    # Note: this must be named ".next" since Next CLI hard-codes that as the output dir
+    name = ".next",
+    args = ["build"],
+    chdir = "$(RULEDIR)",
+    data = [":workaround"],
     output_dir = True,
 )

[alexeagle]

As for the difference with exports_directories_only, I see that as well. Happens under rules_nodejs 5.0.0 also. Something about webpack's implementation of the require function (they don't use the node stdlib for it?) doesn't work, and next build for some reason tries to execute your application during the build (maybe to do prerendering?)

This is the code failing:

/* harmony import */ var next_config__WEBPACK_IMPORTED_MODULE_3___default = /*#__PURE__*/__webpack_require__.n(next_config__WEBPACK_IMPORTED_MODULE_3__);
const { publicRuntimeConfig: { TEST  } ,  } = next_config__WEBPACK_IMPORTED_MODULE_3___default()(); // Returns undefined when  exports_directories_only = True,

so the __webpack_require__.n helper can't resolve the node_modules/next/config.js file under Bazel's execroot.

[ewianda]

Thanks for the repro.

First, your "working" case doesn't produce any outputs, because you're running next in the source tree (even though the comment there acknowledges that it has to run in the output tree.)

See https://docs.aspect.build/bazelbuild/rules_nodejs/4.5.1/docs/builtins.html#npm_package_bin-chdir

This fixes it:

diff --git a/BUILD.bazel b/BUILD.bazel
index 6e3e8a5..00429bb 100644
--- a/BUILD.bazel
+++ b/BUILD.bazel
@@ -1,3 +1,4 @@
+load("@build_bazel_rules_nodejs//:index.bzl", "copy_to_bin")
 load("@npm//next:index.bzl", "next")
 load("@npm_breaks//next:index.bzl", broken_next="next")

@@ -6,14 +7,9 @@ load("@npm_breaks//next:index.bzl", broken_next="next")
 # As a workaround, we can run it with a working directory in the output folder
 # so it produces a directory in the place bazel expects.

-next(
-    # Note: this must be named ".next" since Next CLI hard-codes that as the output dir
-    name = ".next",
-    args = [
-        "build",
-    ],
-    chdir = package_name(),
-    data = glob([
+copy_to_bin(
+    name = "workaround",
+    srcs = glob([
         "components/*",
         "pages/**",
         "public/**",
@@ -22,6 +18,14 @@ next(
         "package.json",
         "next.config.js",
     ],
+)
+
+next(
+    # Note: this must be named ".next" since Next CLI hard-codes that as the output dir
+    name = ".next",
+    args = ["build"],
+    chdir = "$(RULEDIR)",
+    data = [":workaround"],
     output_dir = True,
 )

Just to make sure I am not doing something unusual. My complete setup is

next(
    name = ".next",
    args = [
        "build",
        "--no-lint",
        "$(RULEDIR)",
    ],
    data = DEPS,
    output_dir = True,
)

next(
    name = "start",
    args = [
        "start -p 8090 ./benchsci/frontend/ern",
        "--node_options=--preserve-symlinks-main",
    ],
    data = [
        ":.next",
    ] + DEPS,
    tags = [
        "manual",
        # Tell ibazel not to restart the devserver when its deps change.
        "ibazel_notify_changes",
        # Tell ibazel to serve the live reload script, since we expect a browser will connect to
        # this program.
        "ibazel_live_reload",
    ],
    templated_args = ["--bazel_patch_module_resolver"],
)

nodejs_image(
    name = "client_image",
    args = ["start"],
    data = [
        ":.next",
    ] + DEPS,
    entry_point = "@npm//:node_modules/next/dist/bin/next",
    templated_args = ["--bazel_patch_module_resolver"],
)

Which seems to work.

[ewianda]

As for the difference with exports_directories_only, I see that as well. Happens under rules_nodejs 5.0.0 also. Something about webpack's implementation of the require function (they don't use the node stdlib for it?) doesn't work, and next build for some reason tries to execute your application during the build (maybe to do prerendering?)

This is the code failing:

/* harmony import */ var next_config__WEBPACK_IMPORTED_MODULE_3___default = /*#__PURE__*/__webpack_require__.n(next_config__WEBPACK_IMPORTED_MODULE_3__);
const { publicRuntimeConfig: { TEST  } ,  } = next_config__WEBPACK_IMPORTED_MODULE_3___default()(); // Returns undefined when  exports_directories_only = True,

so the __webpack_require__.n helper can't resolve the node_modules/next/config.js file under Bazel's execroot.

Is this related to how runfiles are setup with exports_directories_only=True ?

[ewianda]

Hello @alexeagle .

Sorry I am way out of my depth here. My javascript knowledge is not adequate to even reason about this issue. I am only trying to setup bazel build for our javascript code base. Is the conclusion here that there is an issue with nextjs internals or is there something else that needs to be addressed.

[ewianda]

Adding --bazel_run_from_execroot seems to fix the build

https://github.com/ewianda/bazel-next/pull/1/files

Not sure exactly what the difference is