search

nrwl / nx

Smart Monorepos · Fast CI
https://nx.dev
MIT License
23.56k stars 2.36k forks source link

Nx CLI is very slow when local plugins implement `createNodes` #26297

Closed kylebjordahl closed 5 months ago

kylebjordahl commented 5 months ago

Current Behavior

Running any command which utilizes the project graph results in very slow CLI results. nx graph currently takes over 1 minute to complete; the same happens when running basic project targets like lint and test. Running time nx report for this issue took over 9 minutes

It appears that our custom plugins are taking orders of magnitude longer to load than 1st and 3rd party Nx plugins; while I cannot share the source code as this is a private repo, I can say that I have tried removing our local plugins from the nx.json and this restores the CLI to more acceptable speeds.

I have also tried creating a wholly new plugin with the following createNodes implementation, and it still takes an unbelievably long time to load the plugin (it has zero dependencies and only the one index.ts file).

perf test plugin

import { CreateNodes, CreateNodesContext, readJsonFile } from '@nx/devkit';
import { dirname, join } from 'path';

export const createNodes: CreateNodes = [
  '**/project.json',
  (projectJsonFilePath: string, _, context: CreateNodesContext) => {
    const project = readJsonFile(
      join(context.workspaceRoot, projectJsonFilePath)
    );
    const projectDir = dirname(projectJsonFilePath);
    return {
      projects: {
        [projectDir]: {
          name: project.name,
          targets: {
            foo: {
              command: 'echo "foo"',
            },
          },
        },
      },
    };
  },
];

Logs

time NX_PERF_LOGGING=true NX_DAEMON=false NX_CACHE_PROJECT_GRAPH=false NX_ISOLATE_PLUGINS=true pnpm nx graph --file=tmp_file3.json --verbose

Time for 'loading dotenv files' 1.7580829999999992
Time for 'workspace context init' 7.420375000000007
Time for 'Load Nx Plugin: @nx/eslint/plugin' 10.54504099999997
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.9.1_@swc+core@1.5.24/node_modules/nx/src/plugins/js' 22.390166999999963
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.9.1_@swc+core@1.5.24/node_modules/nx/src/plugins/project-json/build-nodes/project-json' 0.38924999999994725
Time for 'Load Nx Plugin: @nx/storybook/plugin' 244.241584
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.9.1_@swc+core@1.5.24/node_modules/nx/src/plugins/project-json/build-nodes/package-json-next-to-project-json' 0.43087500000001455
Time for 'Load Nx Plugin: @nx/remix/plugin' 327.78691699999996
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.9.1_@swc+core@1.5.24/node_modules/nx/src/plugins/package-json-workspaces' 1.257333000000017
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.9.1_@swc+core@1.5.24/node_modules/nx/src/plugins/target-defaults/target-defaults-plugin' 0.3419579999999769
Time for 'Load Nx Plugin: @nxlv/python' 483.18112499999995
Time for 'Load Nx Plugin: @nx/vite/plugin' 247.12970799999994
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.9.1_@swc+core@1.5.24/node_modules/nx/src/plugins/package-json-workspaces' 0.33345800000000736
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.9.1_@swc+core@1.5.24/node_modules/nx/src/plugins/project-json/build-nodes/package-json-next-to-project-json' 0.38599999999996726
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.9.1_@swc+core@1.5.24/node_modules/nx/src/plugins/target-defaults/target-defaults-plugin' 0.40458300000000236
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.9.1_@swc+core@1.5.24/node_modules/nx/src/plugins/js' 5.743292000000054
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.9.1_@swc+core@1.5.24/node_modules/nx/src/plugins/project-json/build-nodes/project-json' 0.4152090000000044
Time for 'workspace context init' 5.568457999999964
Time for 'nx/core/package-json:createNodes' 364.0932500000001
Time for 'nx/core/project-json:createNodes' 340.63258400000007
Time for 'nx/core/package-json-workspaces:createNodes' 348.03945899999997
Time for 'nx/js/dependencies-and-lockfile:createNodes' 417.00225
Time for 'nx/core/target-defaults:createNodes' 674.1791260000001
Time for 'build-project-configs' 713.261833
Time for 'createNodes:merge' 6.6293329999998605
Time for 'Load Nx Plugin: @rh-plugins/perf-test' 62013.113946
Time for '@nx/remix/plugin:createNodes' 0.16854099999909522
Time for '@rh-plugins/perf-test:createNodes' 178.17366700000275
Time for 'nx/core/package-json:createNodes' 249.3558750000011
Time for 'nx/core/project-json:createNodes' 161.3224160000027
Time for 'nx/core/package-json-workspaces:createNodes' 212.95116699999926
Time for 'nx/js/dependencies-and-lockfile:createNodes' 354.3757089999999
Time for 'workspace context init' 4.7882080000053975
Time for 'nx/core/target-defaults:createNodes' 416.31229199999507
The CJS build of Vite's Node API is deprecated. See https://vitejs.dev/guide/troubleshooting.html#vite-cjs-node-api-deprecated for more details.
[---------------PRUNED A BUNCH OF Nx Vite TsPaths logs here----------]
Time for '@nx/vite/plugin:createNodes' 1812.2412919999915
Time for '@nx/eslint/plugin:createNodes' 2292.9875840000095
Time for 'workspace context init' 9.815834000000905
The CJS build of Vite's Node API is deprecated. See https://vitejs.dev/guide/troubleshooting.html#vite-cjs-node-api-deprecated for more details.
Time for '@nx/storybook/plugin:createNodes' 85170.74633300002
Time for 'workspace context init' 10.473000000005413
Time for 'build-project-configs' 85281.55054
Time for 'createNodes:merge' 51.932832999998936
Time for 'native-file-deps' 0.11845799998263828
Time for 'get-workspace-files' 4.320082999998704
Time for 'get-all-workspace-files' 11.562000000005355
Time for '@nx/storybook/plugin:createDependencies' 59.274749999982305
Time for '@nx/remix/plugin:createDependencies' 64.09587499999907
Time for '@nx/vite/plugin:createDependencies' 63.51991600001929
Time for '@nxlv/python:createDependencies' 226.33183400001144
Time for 'build typescript dependencies' 241.74520799997845

 NX   JSON output created in /home/vscode/workspace

/home/vscode/workspace/tmp_file3.json

NX_PERF_LOGGING=true NX_DAEMON=false NX_CACHE_PROJECT_GRAPH=false =true pnpm   1.20s user 0.48s system 1% cpu 2:30.80 total

Expected Behavior

Nx CLI should perform commands in a timely fashion that does not dramatically hinder developer productivity.

GitHub Repo

No response

Steps to Reproduce

I've tried to reproduce in the nx-examples lib, and while I can see a slight change in CLI time when I have a custom plugin, it's not nearly as dramatic as we are seeing in our repo. It could be because of the difference in scale (we have nearly 100 projects)

Nx Report

workspace git:(kb/RN-769-upgrade-to-nx-19) time nx report

 NX   Report complete - copy this into the issue template

Node   : 20.14.0
OS     : linux-arm64
pnpm   : 8.15.7

nx                 : 19.1.1
@nx/js             : 19.1.1
@nx/jest           : 19.1.1
@nx/linter         : 19.1.1
@nx/eslint         : 19.1.1
@nx/workspace      : 19.1.1
@nx/cypress        : 19.1.1
@nx/devkit         : 19.1.1
@nx/eslint-plugin  : 19.1.1
@nx/plugin         : 19.1.1
@nx/react          : 19.1.1
@nx/remix          : 19.1.1
@nx/storybook      : 19.1.1
@nrwl/tao          : 19.1.1
@nx/vite           : 19.1.1
@nx/web            : 19.1.1
nx-cloud           : 19.0.0
typescript         : 5.3.3
---------------------------------------
Registered Plugins:
@nx/storybook/plugin
@nx/remix/plugin
@nx/eslint/plugin
@nxlv/python
@rh-plugins/python
@rh-plugins/cdk-python
@rh-plugins/cdktf-python
@rh-plugins/ruff
@nx/vite/plugin
---------------------------------------
Community plugins:
@nxlv/python : 18.0.1
---------------------------------------
Local workspace plugins:
         @rh-plugins/cdktf-python
         @rh-plugins/cdk-python
         @rh-plugins/protobuf
         @rh-plugins/python

nx report  36.35s user 57.34s system 16% cpu 9:11.72 total

Failure Logs

time NX_PERF_LOGGING=true NX_DAEMON=false NX_CACHE_PROJECT_GRAPH=false NX_ISOLATE_PLUGINS=true pnpm nx graph --file=tmp_file2.json --verbose

Time for 'loading dotenv files' 1.9366660000000024
Time for 'workspace context init' 4.815333999999993
Time for 'Load Nx Plugin: @nx/eslint/plugin' 37.29208299999999
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.8.0_@swc+core@1.3.107/node_modules/nx/src/plugins/project-json/build-nodes/package-json-next-to-project-json' 0.41433399999999665
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.8.0_@swc+core@1.3.107/node_modules/nx/src/plugins/js' 20.81762500000002
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.8.0_@swc+core@1.3.107/node_modules/nx/src/plugins/target-defaults/target-defaults-plugin' 11.796625000000006
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.8.0_@swc+core@1.3.107/node_modules/nx/src/plugins/project-json/build-nodes/project-json' 0.3847499999999968
Time for 'Load Nx Plugin: @nx/vite/plugin' 253.25066699999996
Time for 'Load Nx Plugin: @nx/storybook/plugin' 353.37341699999996
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.8.0_@swc+core@1.3.107/node_modules/nx/src/plugins/package-json-workspaces' 29.327499999999986
Time for 'Load Nx Plugin: @nx/remix/plugin' 419.44741700000003
Time for 'Load Nx Plugin: @nxlv/python' 640.0362910000001
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.8.0_@swc+core@1.3.107/node_modules/nx/src/plugins/project-json/build-nodes/package-json-next-to-project-json' 0.3523329999999305
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.8.0_@swc+core@1.3.107/node_modules/nx/src/plugins/js' 7.149833000000001
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.8.0_@swc+core@1.3.107/node_modules/nx/src/plugins/project-json/build-nodes/project-json' 2.5868750000000773
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.8.0_@swc+core@1.3.107/node_modules/nx/src/plugins/js' 33.41737499999999
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.8.0_@swc+core@1.3.107/node_modules/nx/src/plugins/project-json/build-nodes/project-json' 0.37845899999990706
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.8.0_@swc+core@1.3.107/node_modules/nx/src/plugins/project-json/build-nodes/package-json-next-to-project-json' 0.4613329999999678
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.8.0_@swc+core@1.3.107/node_modules/nx/src/plugins/target-defaults/target-defaults-plugin' 0.4339999999999691
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.8.0_@swc+core@1.3.107/node_modules/nx/src/plugins/target-defaults/target-defaults-plugin' 0.4881669999999758
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.8.0_@swc+core@1.3.107/node_modules/nx/src/plugins/project-json/build-nodes/package-json-next-to-project-json' 0.37004099999990103
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.8.0_@swc+core@1.3.107/node_modules/nx/src/plugins/package-json-workspaces' 10.988749999999982
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.8.0_@swc+core@1.3.107/node_modules/nx/src/plugins/target-defaults/target-defaults-plugin' 0.4730420000000777
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.8.0_@swc+core@1.3.107/node_modules/nx/src/plugins/project-json/build-nodes/package-json-next-to-project-json' 0.3638750000000073
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.8.0_@swc+core@1.3.107/node_modules/nx/src/plugins/project-json/build-nodes/project-json' 1.1877080000000433
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.8.0_@swc+core@1.3.107/node_modules/nx/src/plugins/target-defaults/target-defaults-plugin' 0.3692910000000893
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.8.0_@swc+core@1.3.107/node_modules/nx/src/plugins/package-json-workspaces' 0.4823329999999828
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.8.0_@swc+core@1.3.107/node_modules/nx/src/plugins/js' 5.596708000000035
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.8.0_@swc+core@1.3.107/node_modules/nx/src/plugins/js' 7.971333000000072
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.8.0_@swc+core@1.3.107/node_modules/nx/src/plugins/package-json-workspaces' 0.39387499999998
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.8.0_@swc+core@1.3.107/node_modules/nx/src/plugins/package-json-workspaces' 0.34941600000001927
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.8.0_@swc+core@1.3.107/node_modules/nx/src/plugins/project-json/build-nodes/project-json' 0.3890830000000278
Time for 'workspace context init' 5.462082999999893
Time for 'workspace context init' 14.914167000000134
Time for 'workspace context init' 7.9191249999998945
Time for 'workspace context init' 7.22441700000013
Time for 'nx/js/dependencies-and-lockfile:createNodes' 101.05445800000007
Time for 'nx/js/dependencies-and-lockfile:createNodes' 125.00791600000002
Time for 'nx/js/dependencies-and-lockfile:createNodes' 139.89987500000007
Time for 'nx/core/project-json:createNodes' 179.64724999999999
Time for 'nx/js/dependencies-and-lockfile:createNodes' 161.38587500000006
Time for 'nx/core/project-json:createNodes' 166.9258749999999
Time for 'nx/core/project-json:createNodes' 235.31633399999987
Time for 'nx/core/project-json:createNodes' 185.86195899999996
Time for 'nx/core/package-json:createNodes' 396.74408400000004
Time for 'nx/core/package-json-workspaces:createNodes' 444.327125
Time for 'nx/core/package-json:createNodes' 556.4193750000001
Time for 'nx/core/package-json-workspaces:createNodes' 538.2411670000001
Time for 'nx/core/package-json:createNodes' 565.447876
Time for 'nx/core/package-json-workspaces:createNodes' 460.57462599999997
Time for 'nx/core/package-json:createNodes' 613.3223759999998
Time for 'nx/core/package-json-workspaces:createNodes' 613.796209
Time for 'nx/core/target-defaults:createNodes' 657.409042
Time for 'nx/core/target-defaults:createNodes' 645.193417
Time for 'nx/core/target-defaults:createNodes' 689.925292
Time for 'nx/core/target-defaults:createNodes' 651.9341250000001
Time for 'build-project-configs' 729.381625
Time for 'createNodes:merge' 9.729624999999942
Time for 'Load Nx Plugin: @rh-plugins/cdktf-python' 56972.125734999994
Time for 'build-project-configs' 693.9262500000002
Time for 'createNodes:merge' 8.004165999999941
Time for 'Load Nx Plugin: @rh-plugins/cdk-python' 57065.629569
Time for 'build-project-configs' 711.949501
Time for 'createNodes:merge' 7.878707999999733
Time for 'Load Nx Plugin: @rh-plugins/ruff' 57040.010527
Time for 'build-project-configs' 727.270417
Time for 'createNodes:merge' 8.386416000000281
Time for 'Load Nx Plugin: @rh-plugins/python' 57085.224068999996
Time for '@nx/remix/plugin:createNodes' 0.14279099999839673
Time for '@rh-plugins/cdktf-python:createNodes' 13.411208999998053
Time for '@rh-plugins/cdk-python:createNodes' 59.52062499999738
Time for '@rh-plugins/python:createNodes' 166.61512499999662
Time for 'nx/js/dependencies-and-lockfile:createNodes' 110.97000000000116
Time for 'nx/core/package-json:createNodes' 302.8212080000012
Time for '@rh-plugins/ruff:createNodes' 229.4657920000027
Time for 'nx/core/project-json:createNodes' 199.08883400000195
Time for 'nx/core/package-json-workspaces:createNodes' 284.01420900000085
Time for '@nx/storybook/plugin:createNodes' 464.2682080000013
Time for 'workspace context init' 16.303333000003477
Time for '@nx/vite/plugin:createNodes' 412.40845800000534
Time for 'workspace context init' 8.899166999995941
Time for 'nx/core/target-defaults:createNodes' 422.37024999999994
Time for '@nx/eslint/plugin:createNodes' 1781.4180420000048
Time for 'workspace context init' 7.531042000002344
Time for 'build-project-configs' 1901.2991670000047
Time for 'createNodes:merge' 12.72004200000083
Time for 'native-file-deps' 0.08587499999703141
Time for 'get-workspace-files' 2.0747919999994338
Time for 'get-all-workspace-files' 11.310082999996666
Time for '@nx/remix/plugin:createDependencies' 37.059000000001106
Time for '@nx/vite/plugin:createDependencies' 35.09933300000557
Time for '@nx/storybook/plugin:createDependencies' 59.76391599999624
Time for '@nxlv/python:createDependencies' 167.86887600000045
Time for 'build typescript dependencies' 200.22995799999626

 NX   JSON output created in /home/vscode/workspace

/home/vscode/workspace/tmp_file2.json

NX_PERF_LOGGING=true NX_DAEMON=false NX_CACHE_PROJECT_GRAPH=false =true pnpm   0.91s user 0.33s system 2% cpu 1:01.95 total

Package Manager Version

pnpm@8.15.7

Operating System

Additional Information

Running in a devcontainer on MacOS 14.3.1 (M1 chip), devcontainer OS is Ubuntu 22.04

This issue was opened after working with Miro from Nx Cloud Support via email, on their recommendation

kylebjordahl commented 5 months ago

After more tinkering with my perf-test plugin, I have discovered that if I build the plugin and place it within the node_modules dir to approximate installing a 3rd party plugin, it loads remarkably fast. (Logs below)

It seems there is something wrong with loading local TS plugins

workspace git:(kb/RN-769-upgrade-to-nx-19) time NX_PERF_LOGGING=true NX_DAEMON=false NX_CACHE_PROJECT_GRAPH=false NX_ISOLATE_PLUGINS=true pnpm nx graph --file=tmp_file3.json --verbose

Time for 'loading dotenv files' 1.1174579999999992
Time for 'workspace context init' 3.954625000000007
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.9.1_@swc+core@1.5.24/node_modules/nx/src/plugins/project-json/build-nodes/package-json-next-to-project-json' 0.44529099999999744
Time for 'Load Nx Plugin: @nx/eslint/plugin' 2.4542920000000095
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.9.1_@swc+core@1.5.24/node_modules/nx/src/plugins/target-defaults/target-defaults-plugin' 2.9343749999999886
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.9.1_@swc+core@1.5.24/node_modules/nx/src/plugins/js' 7.256458000000009
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.9.1_@swc+core@1.5.24/node_modules/nx/src/plugins/package-json-workspaces' 0.47204099999999016
Time for 'Load Nx Plugin: @nx/vite/plugin' 254.830208
Time for 'Load Nx Plugin: @rh-plugins/perf-test' 51.146124999999984
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/nx@19.1.1_@swc-node+register@1.9.1_@swc+core@1.5.24/node_modules/nx/src/plugins/project-json/build-nodes/project-json' 0.37720899999999347
Time for 'Load Nx Plugin: @nx/storybook/plugin' 213.89875
Time for 'Load Nx Plugin: @nx/remix/plugin' 76.697
Time for 'Load Nx Plugin: @nxlv/python' 194.76016700000002
Time for '@nx/remix/plugin:createNodes' 0.046625000000062755
Time for 'nx/js/dependencies-and-lockfile:createNodes' 40.426874999999995
Time for '@rh-plugins/perf-test:createNodes' 139.61029199999996
Time for 'nx/core/package-json:createNodes' 223.16429199999993
Time for 'nx/core/package-json-workspaces:createNodes' 177.91375000000005
Time for 'nx/core/project-json:createNodes' 176.405125
Time for '@nx/storybook/plugin:createNodes' 321.95854199999997
Time for 'workspace context init' 5.202041000000008
Time for '@nx/vite/plugin:createNodes' 328.05991700000004
Time for 'workspace context init' 10.170041999999967
Time for 'nx/core/target-defaults:createNodes' 341.43791699999997
Time for '@nx/eslint/plugin:createNodes' 1881.7331680000002
Time for 'workspace context init' 5.841625000000022
Time for 'build-project-configs' 1942.886418
Time for 'createNodes:merge' 7.167707999999948
Time for 'native-file-deps' 0.1515409999997246
Time for 'get-workspace-files' 2.2273339999997006
Time for 'get-all-workspace-files' 10.753375000000233
Time for '@nx/storybook/plugin:createDependencies' 98.18079200000011
Time for '@nx/remix/plugin:createDependencies' 84.69016600000032
Time for '@nx/vite/plugin:createDependencies' 62.47445800000014
Time for '@nxlv/python:createDependencies' 239.52779200000032
Time for 'build typescript dependencies' 549.6482080000001

 NX   JSON output created in /home/vscode/workspace

/home/vscode/workspace/tmp_file3.json

NX_PERF_LOGGING=true NX_DAEMON=false NX_CACHE_PROJECT_GRAPH=false =true pnpm   0.66s user 0.31s system 26% cpu 3.720 total
kylebjordahl commented 5 months ago

Narrowing it down even further, I added some performance marks in my local nx install, specifically in node_modules/nx/src/plugins/js/utils/typescript.js:readTsConfig and it looks as though this is where all the time is getting spent on plugin loading, specifically in tsModule.parseJsonConfigFileContent

Time for 'Load Nx Plugin: @rh-plugins/ruff' 62171.26652999999
Time for 'readTsConfig - readConfigFile total' 5.7970000000000255
Time for 'readTsConfig - Total' 61845.963154

So, going out on a wild limb, I tried making a few under-the-hood tweaks and managed to get some pretty significant speed improvements by avoiding the call to tsModule.parseJsonConfigFileContent altogether; from what I can tell, registerPluginTSTranspiler doesn't actually need most of the output generated by parseJsonConfigFileContent, it really just wants the contents of the tsconfig.json file.

I'll provide a PR that represents what I've found, and hopefully this can get sorted out quickly

kylebjordahl commented 5 months ago

After investigating the performance of parseJsonConfigFileContent (mostly by running tsc --showConfig) I finally diagnosed this as an issue in our tsconfig.base.json excludes.

I ended up adding some extra excludes, it might be ideal to include some of these in the default which is generated for new workspaces: 

"exclude": ["**/node_modules", "**/.*/", "tmp", "**/cdk.out", "dist"]
github-actions[bot] commented 3 months ago

This issue has been closed for more than 30 days. If this issue is still occuring, please open a new issue with more recent context.