Gatsby breaks if any parent directory is called node_modules

[rakeshpai]

Description

When gatsby is run in any child directory of a directory called node_modules, npm run develop breaks with the following output to stdout:

success open and validate gatsby-configs — 0.012 s
success load plugins — 0.255 s
success onPreInit — 0.399 s
success delete html and css files from previous builds — 0.123 s
success initialize cache — 0.006 s
success copy gatsby files — 0.030 s
success onPreBootstrap — 0.006 s
success source and transform nodes — 0.094 s
success building schema — 0.355 s
success createPages — 0.001 s
success createPagesStatefully — 0.061 s
success onPreExtractQueries — 0.003 s
success update schema — 0.207 s
success extract queries from components — 0.147 s
success run graphql queries — 0.132 s — 8/8 61.30 queries/second
success write out page data — 0.005 s
success write out redirect data — 0.001 s
⡀ onPostBootstrapdone generating icons for manifest
success onPostBootstrap — 0.235 s

info bootstrap finished - 5.542 s

here ./.cache/develop-static-entry.js 102:6
Module parse failed: Unexpected token (102:6)
You may need an appropriate loader to handle this file type.
|     body: ``,
|     headComponents: headComponents.concat([
>       <script key={`io`} src="/socket.io/socket.io.js" />,
|     ]),
|     htmlAttributes,
error There was an error compiling the html.js component for the development server.

See our docs page on debugging HTML builds for help https://gatsby.dev/debug-html

  Error: ./.cache/develop-static-entry.js 102:6
  Module parse failed: Unexpected token (102:6)
  You may need an appropriate loader to handle this file type.
  |     body: ``,
  |     headComponents: headComponents.concat([
  >       <script key={`io`} src="/socket.io/socket.io.js" />,
  |     ]),
  |     htmlAttributes,

I'm creating a gatsby project inside a node_modules, because I want to use gatsby in a monorepo, alongside other code. To share code within the monorepo, instead of using Lerna, I'm using an approach similar to the one used by pouchdb, details here. Essentially, this involves having a node_modules directory somewhere in the path, which then causes node's require resolution algorithm to include all its children directories However, gatsby doesn't seem to like this node_modules in the path.

Steps to reproduce

mkdir -p test-gatsby/node_modules
cd test-gatsby/node_modules
npx gatsby new gatsby-site
cd gatsby-site
npm run develop # breaks

This doesn't break if node_modules is not in the path.

Expected result

The parent directory structure shouldn't matter. node_modules somewhere in the parent should be valid.

Environment

System: OS: Linux 4.15 Ubuntu 18.04.2 LTS (Bionic Beaver) CPU: (4) x64 Intel(R) Core(TM) i3-5010U CPU @ 2.10GHz Shell: 4.4.19 - /bin/bash Binaries: Node: 8.8.1 - /usr/local/bin/node npm: 6.8.0 - /usr/local/bin/npm

[rakeshpai]

npm run build also breaks with a similar error:

success open and validate gatsby-configs — 0.010 s
success load plugins — 0.243 s
success onPreInit — 0.426 s
success delete html and css files from previous builds — 0.124 s
success initialize cache — 0.007 s
success copy gatsby files — 0.046 s
success onPreBootstrap — 0.007 s
success source and transform nodes — 0.088 s
success building schema — 0.305 s
success createPages — 0.001 s
success createPagesStatefully — 0.058 s
success onPreExtractQueries — 0.003 s
success update schema — 0.213 s
success extract queries from components — 0.167 s
success run graphql queries — 0.092 s — 7/7 77.61 queries/second
success write out page data — 0.004 s
success write out redirect data — 0.001 s
⢀ onPostBootstrapdone generating icons for manifest
success onPostBootstrap — 0.257 s

info bootstrap finished - 4.249 s

error Generating JavaScript bundles failed

  Error: ./.cache/production-app.js 43:8
  Module parse failed: Unexpected token (43:8)
  You may need an appropriate loader to handle this file type.
  | 
  |       return (
  >         <EnsureResources location={location}>
  |           {({ pageResources, location }) => (
  |             <RouteUpdates location={location}>

More:

• This doesn't break if node_modules is not a parent.
• No matter how high up the node_modules folder is located, gatsby breaks. (I've tested 2 levels of depth, ie: some-dir/node_modules/some-other-dir/gatsby-site.)

[rakeshpai]

It looks like this regex might be the culprit. If node_modules is the parent of the cwd, all files will match this regex, and so nothing will get built. It should be possible to modify this regex to only match after process.cwd(). If that's hard to do with regexes, it should be possible to change it to a function.

Disclaimer: This is from like 15 mins of reading around the code. I haven't run it, and I'm not familiar with the code. I would have issued a PR, but I'm having trouble getting the gatsby repo running locally.

[DSchau]

@rakeshpai I think you're on the correct path, there.

Gatsby is currently excluding node_modules for transpilation, which will manifest the issue you're seeing--essentially non-transpiled code making its way into the browser.

I can't say I fully understand the use case of developing in a nested node_modules folder, but I think the fix here would be to modify that regex that you mention to be a little more specific, e.g. to refer to the current directory (program.directory) and the node_modules within that directory. I wonder if this would break the use case of monorepos though, e.g. when you're developing in a monorepo and the node_modules directory could/is hoisted in a relative directory above the current. This could start transpiling node_modules in a parent directory, which we don't really want to necessarily support.

With this approach you're following, are there nested node_modules folders (within the Gatsby app) or are they hoisted?

[gatsbot[bot]]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.

If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!

Thanks for being a part of the Gatsby community! 💪💜

[gatsbot[bot]]

Hey again!

It’s been 30 days since anything happened on this issue, so our friendly neighborhood robot (that’s me!) is going to close it.

Please keep in mind that I’m only a robot, so if I’ve closed this issue in error, I’m HUMAN_EMOTION_SORRY. Please feel free to reopen this issue or create a new one if you need anything else.

Thanks again for being part of the Gatsby community!

[davidrichard23]

I can't say I fully understand the use case of developing in a nested node_modules folder

@DSchau we have run into this issue as well. Our use case is that we're using npm as a package manager for a plugin system inside an electron app. One of those plugins is a gatsby website that enables a user to take data generated within the electron app and build/deploy a gatsby site.

We've been testing with npm link which seems to have circumvented this issue, but now we're on a pretty tight deadline and this issue breaks a significant portion of our offering. Any news on a fix?