Local plugin "Cannot find module" error

[dteif]

Current Behavior

Using a local plugin generator fails with error "Cannot find module", as shown below.

Expected Behavior

As written in the docs, a plugin should be able to be used locally.

Steps to Reproduce

Run the following commands to create a new nx workspace with a local plugin package. yarn has been used, but the same result is obtained with npm.

npx create-nx-workspace@latest --preset=core my-workspace
cd my-workspace
rm -r node_modules
rm package-lock.json
echo '{"version": 2, "projects": {}}' >> workspace.json
yarn add -D @nrwl/nx-plugin -W
yarn nx g @nrwl/nx-plugin:plugin my-plugin --importPath @my-workspace/my-plugin
yarn nx g @nrwl/nx-plugin:generator app --project my-plugin
yarn nx g @my-workspace/my-plugin:app testapp --verbose

The last command fails with the error specified in Failure Logs.

As a workaround, the workspace files can be removed from the node_modules folder. With the additional following commands, the generator works as expected:

rm -r node_modules/@my-workspace/my-plugin
yarn nx g @my-workspace/my-plugin:app testapp

Failure Logs

...  my-workspace % yarn nx g @my-workspace/my-plugin:app testapp --verbose

yarn run v1.22.15
$ /Users/davide/dev/temp/nx-issue/my-workspace/node_modules/.bin/nx g @my-workspace/my-plugin:app testapp --verbose
Cannot find module '/Users/davide/dev/temp/nx-issue/my-workspace/packages/my-plugin/src/generators/app/generator'
Require stack:
- /Users/davide/dev/temp/nx-issue/my-workspace/node_modules/nx/src/config/workspaces.js
- /Users/davide/dev/temp/nx-issue/my-workspace/node_modules/nx/src/command-line/generate.js
- /Users/davide/dev/temp/nx-issue/my-workspace/node_modules/nx/src/command-line/nx-commands.js
- /Users/davide/dev/temp/nx-issue/my-workspace/node_modules/nx/bin/init-local.js
- /Users/davide/dev/temp/nx-issue/my-workspace/node_modules/nx/bin/nx.js
Error: Cannot find module '/Users/davide/dev/temp/nx-issue/my-workspace/packages/my-plugin/src/generators/app/generator'
Require stack:
- /Users/davide/dev/temp/nx-issue/my-workspace/node_modules/nx/src/config/workspaces.js
- /Users/davide/dev/temp/nx-issue/my-workspace/node_modules/nx/src/command-line/generate.js
- /Users/davide/dev/temp/nx-issue/my-workspace/node_modules/nx/src/command-line/nx-commands.js
- /Users/davide/dev/temp/nx-issue/my-workspace/node_modules/nx/bin/init-local.js
- /Users/davide/dev/temp/nx-issue/my-workspace/node_modules/nx/bin/nx.js
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15)
    at Function.Module._load (node:internal/modules/cjs/loader:778:27)
    at Module.require (node:internal/modules/cjs/loader:999:19)
    at require (/Users/davide/dev/temp/nx-issue/my-workspace/node_modules/v8-compile-cache/v8-compile-cache.js:159:20)
    at /Users/davide/dev/temp/nx-issue/my-workspace/node_modules/nx/src/config/workspaces.js:122:28
    at Object.<anonymous> (/Users/davide/dev/temp/nx-issue/my-workspace/node_modules/nx/src/command-line/generate.js:127:40)
    at Generator.next (<anonymous>)
    at fulfilled (/Users/davide/dev/temp/nx-issue/my-workspace/node_modules/tslib/tslib.js:114:62)
error Command failed with exit code 1.

Environment

   Node : 17.8.0
   OS   : darwin arm64
   yarn : 1.22.15

   nx : 13.10.1
   @nrwl/angular : Not Found
   @nrwl/cypress : Not Found
   @nrwl/detox : Not Found
   @nrwl/devkit : 13.10.1
   @nrwl/eslint-plugin-nx : 13.10.1
   @nrwl/express : Not Found
   @nrwl/jest : 13.10.1
   @nrwl/js : 13.10.1
   @nrwl/linter : 13.10.1
   @nrwl/nest : Not Found
   @nrwl/next : Not Found
   @nrwl/node : Not Found
   @nrwl/nx-cloud : Not Found
   @nrwl/nx-plugin : 13.10.1
   @nrwl/react : Not Found
   @nrwl/react-native : Not Found
   @nrwl/schematics : Not Found
   @nrwl/storybook : Not Found
   @nrwl/web : Not Found
   @nrwl/workspace : 13.10.1
   typescript : 4.6.3
   rxjs : 6.6.7
   ---------------------------------------

[AgentEnder]

Hey @dteif, we have tests for this, and I use the local plugin resolution on one of my plugin repos, is there a chance you can post a link to a reproduction that exhibits this problem?

Reading through the issue, I notice you mention the plugin files inside node_modules... Can you explain how/why that is the case? The only way Nx knows to load your plugin as a local plugin instead of an installed plugin is due to the original call to read the generators.json / executors.json / package.json file failing and then falling back on local resolution. If the plugin is in node_modules for some reason, this call will not fail, and Nx will fail when trying to read the generator since it only resolves typescript files for local plugins.

[dteif]

Hi @AgentEnder, thank you for the reply. Here is a reproduction repo of the issue: https://github.com/dteif/issues-nx-local-plugin

Reading through the issue, I notice you mention the plugin files inside node_modules... Can you explain how/why that is the case?

I guess that since plugin files are generated inside packages folder, and this is a workspaces root folder as specified in root package.json file, yarn installs the package inside node_modules. I also tried to move those generated files into another folder (e.g. tools) and updating workspace.json file accordingly, but now every nx command used to perform operations on that plugin project fails.

[dteif]

As a partial workaround, I just noticed that by changing workspaces configuration in root package.json file to exclude the plugin project folder, everything seems to work as expected. Not sure however if this change may affect other repo parts (like testing). The new config in package.json is

{
  //...
  "workspaces": [
      "packages/**",
      "!(packages/my-plugin)"
    ]
}

However, I manage to make it work only with modern version of Yarn, specifically Yarn v3.2.0. Tested with both Yarn v1.x (1.22.15) and npm v8.5.5 and it still fails since plugin package is still installed in node_modules folder.

I created a new branch with the working configuration.

[AgentEnder]

Interesting, yeah this looks like a bug caused by yarn / npm workspaces. I wasn't aware that they linked node modules like that.

I'll have to look into fixing this, probably will need to swap check order (i.e. check if local plugin exists first)

[72gm]

I got the error "Cannot find module 'nx/src/config/workspaces'" when migrating to 14.0.5.... basically none of the nx commands would work... the cli had been left at 13.9.4 , upgrading this to 14.0.5 solved the issue

[mathis-m]

the issue is still persisting in 14.3.6. Inside node_modules there is a symlink to the plugin project. After deleting it, the issue is resolved. @AgentEnder is there any way to work arround this issue?

[mrcreatist]

Facing same issue. Able to create one project but while adding the second project, this error occurs.

[Hotell]

facing same issue with yarn 1 + yarn workspaces

workaround

make sure to manually create javascript file with the same file as you executor/generator implementation with following content:

// @filename generator.js
const { workspaceRoot } = require('nx/src/utils/workspace-root');
const { registerTsProject } = require('nx/src/utils/register');

registerTsProject(workspaceRoot, 'tsconfig.base.json');

module.exports = require('./generator.ts');

fix

~not sure if this can be fixed - require needs to be overriden by ts-node/swc while symlinking from node_modules (forced by workspaces).~ ~I've tried few things in nx source without any luck.~

UPDATE:

explicitly setting node options will properly override require so that should be possible:

NODE_OPTIONS="-r ts-node/register" yarn nx generate @myorg/workspace-extensions:workspace-extensions zzz --dry-run --verbose "just works"

UPDATE 2:

Looks like i figured it out! readPluginPackageJson function needs to be refactored to something like this

export function readPluginPackageJson(
  pluginName: string,
  paths = [workspaceRoot]
): {
  path: string;
  json: PackageJson;
} {
  try {
    const result = readModulePackageJson(pluginName, paths);
    // this will check if the plugin is 3rd party or if its local symlinked to node_modules via workspaces
    if (result.path.includes('node_modules')) {
      return {
        json: result.packageJson,
        path: result.path,
      };
    }
    return registerLocalPlugin();
  } catch (e) {
    if (e.code === 'MODULE_NOT_FOUND') {
      return registerLocalPlugin();
    }
    throw e;
  }

  function registerLocalPlugin() {
    const localPluginPath = resolveLocalNxPlugin(pluginName);
    if (localPluginPath) {
      const localPluginPackageJson = path.join(
        localPluginPath.path,
        'package.json'
      );
      return {
        path: localPluginPackageJson,
        json: readJsonFile(localPluginPackageJson),
      };
    }
  }
}

🫡

@AgentEnder should I send PR or you'll take it from here ? cheers

[aloysb]

@Hotell Thank you, running in the same issue on a vanilla install. :tada:

[AgentEnder]

@Hotell this is an interesting solution... we had debated in the past simply reversing the order of the checks (i.e. it would become is there a local plugin? no -> is there an installed plugin? instead of vice versa). This solution did work (and was merged in #9913), but there were some issues that cropped up when running @nrwl/js:tsc and @nrwl/web:webpack from local sources. Those issues were fixed by #9956...

Let me think on if we want to simply reapply #9913 now, or if its worth exploring the check for node_modules in the manner that you expressed... The bigger issue with doing so would be that it would need to be reworked when evaluating yarn pnp support, but I'm not sure how vital that is at this point in time.

[andyjessop]

I'm seeing this same bug, and @Hotell's fix works. We're also using yarn.

[schirrel]

any news on this?

[dustyhorizon]

I am hitting the same issue too

@AgentEnder not sure if its relevant but it seems that I start getting issues regarding plugin resolution on 14.7.18 onwards. 14.7.17 reports no issues.

For context, the following is configured in my workspace:

• my local project graph plugin written in TS has an entry in paths under tsconfig.base.json at workspaceRoot
• configured plugins with the name of the entry I configured in tsconfig.base.json

Cannot find module "xx" appears on nx 14.7.18 onwards, downgrading to 14.7.17 seems fine to me.

Aside from this, I am also getting unable to find tsconfig.base.json or tsconfig.json on 14.7.18 onwards too. If I remove the local plugin from nx.json, there are no errors.

[AgentEnder]

@dustyhorizon different bug, see #https://github.com/nrwl/nx/issues/12340

[dan-cooke]

This is bonkers. Its very confusing on how to reproduce this but it seems to enter a non-recoverable state by doing something like this:

1. Create a new plugin
2. Rename the generator folder in the new plugin, and update the genreators.json file
3. Generate the files
4. Rename the generator folder again

This now completely breaks generators, no mattetr what I do I cannot generate

I have tried clearing node modules , aligning NX verisons, nx reset to restart the daemon`

This is the most frustrating bug I've encountered with NX, and it basically locks off the Generator functionality for me.

This has happened in 2 projects now, and It seems the only way to use Generatoirs, is to go with the default naming convention of

@company/plugin:plugin which is very ugly

[AgentEnder]

The issue in this case is explicitly caused by npm workspaces, your issue sounds to be different. Can you open a new issue and include a reproduction? I've not seen this.

[dwelch2344]

Props to @hotell for https://github.com/nrwl/nx/issues/9823#issuecomment-1207397040 – new to NX and spent a good 2 hours figuring out why this wasn't working (and couldn't reproduce fresh)

For any looking for the quick fix:

Manually create a generator.js or executor.js like the following:

// @filename generator.js
// @see https://github.com/nrwl/nx/issues/9823#issuecomment-1207397040
const { workspaceRoot } = require('nx/src/utils/workspace-root');
const { registerTsProject } = require('nx/src/utils/register');

registerTsProject(workspaceRoot, 'tsconfig.base.json');

module.exports = require('./generator.ts');

// @filename executor.js
// @see https://github.com/nrwl/nx/issues/9823#issuecomment-1207397040
const { workspaceRoot } = require('nx/src/utils/workspace-root');
const { registerTsProject } = require('nx/src/utils/register');

registerTsProject(workspaceRoot, 'tsconfig.base.json');

module.exports = require('./executor.ts');

[forivall]

Alternatively, I've been using the following workaround:

// @filename executor.js
const { resolveLocalNxPlugin } = require('nx/src/utils/nx-plugin');
resolveLocalNxPlugin(require('../package.json').name);

module.exports = require('./executor.ts');

[Spencer-Easton]

I was stuck on this for a while. I realized tsconfig.base.json path never updated when plugin's package.json name param was changed. It produced the same error as the OP.

[SimonKunz]

Any news on this? Unfortunately the workarounds described above won't work on my project.

[github-actions[bot]]

This issue has been closed for more than 30 days. If this issue is still occuring, please open a new issue with more recent context.