search

nrwl / nx

Smart Monorepos · Fast CI
https://nx.dev
MIT License
23.69k stars 2.36k forks source link

Dependency Graph with disconnected nodes (Library to Library dependency & nested folders) #2955

Closed georgiee closed 4 years ago

georgiee commented 4 years ago

Expected Behavior

I want to use subfolders is libs/ to group my packages and I expect a correct dependency independent of being flat or nested. With a wrong dependency graph the builds with --with-deps will always fail. So you can use the nx dep-graph utility for a quick check without an actual build.

Current Behavior

  1. Flat projects, where the packages are located in libs/ always generate a correct graph ✅
  2. Nested libs/ generate a correct graph on OSX ✅
  3. Nested libs/ always fails to generate a correct graph on Windows 10 Enterprise🛑 The nested dependencies are not recognized, only the flat ones.

The graph will look differently depending of the bug to happen or not:

Good ✅ osx-nested

Bad (Windows 10 Enterprise)🛑 windows-nested

Failure Information (for bugs)

I think this is clearly a bug. It's working on all OSX machines but fails on Windows 10 Enterprise for many colleagues from abroad. My testing capacity is therefore limited but I captured as much information I could. I'm happy to forward other test cases if you need more information.

Steps to Reproduce

  1. Create an Nx Workspace
  2. Add four libraries a, b, c, d and make a dependent on b and c(a->b) dependent on d (c->d)
  3. Confirm the graph is good (nx dep-graph)
  4. Confirm the build works for a & c (nx run-many --target=build --projects=a,c --with-deps)

(Windows) Bug is appearing now:

  1. Move d into lib/nested/d and adjust all configs (or I should have used --directory which I didn't think of)
  2. Create a Graph (nx dep-graph) which shows a dangling d on affected systems. Check the screenshots in the Failure Logs section or in the example repository.

Context

I prepared a carefully curated demo application with a Readme listing all results and tags to have your own try. I will use some of the content of the Readme to complete this issue for your convenience: https://github.com/georgiee/nx-debug-nested-dep-graph

It's really quick to check as the only thing you need to run is the dep-graph utility. If the graph is wrong the build will fail. I also added a monkey patch for an internal nx util to add some more console logs. Those will reveal that the dependency is not found and therefore the entire build has a wrong baseline.

Working OSX:

Node: v12.10.0
OSX 10.15.4
Latest Nx

Failing Windows:

Node: 12.14.1
Win 10 Enterprise Version 1607
Latest Nx

Failure Logs

Copied from the Readme in the example project: https://github.com/georgiee/nx-debug-nested-dep-graph

Baseline

That's how I created the project, which means is a standard project.

create-nx-workspace debug-nx-deps-problem --preset angular --appName debug-nx-deps-problem --style=css --cli=nx
cd debug-nx-deps-problem

yarn nx g @nrwl/angular:lib a --publishable --style=css
yarn nx g @nrwl/angular:lib b --publishable --style=css
yarn nx g @nrwl/angular:lib c --publishable --style=css
yarn nx g @nrwl/angular:lib d --publishable --style=css

# Prepare: Add actual dependencies to a (dep on b) and c (dep on d).

Nx has no expressive verbose mode, so I prepared a debug script that will monkey patch Nx's core file buildable-libs-utils.js which is involved in providing utilities for the dependency graph. You will get the following output per package during a build in addition to the other logs.

> nx run a:build
---- checkDependentProjectsHaveBeenBuilt: [
  '/debug-nx-deps-problem/dist/libs/b/package.json'
]
---- updatePaths. dependencies: [
  {
    name: '@debug-nx-deps-problem/b',
    outputs: [ 'dist/libs/b' ],
    node: { name: 'b', type: 'lib', data: [Object] }
  }
]

Context: OSX (✅)

Node: v12.10.0
OSX 10.15.4

Here the report for OSX. Everything works as expected.

Flat (✅)
git co scenario-flat
yarn nx dep-graph

osx-flat

Build works:

yarn debug-nx-build

# runs this:
# yarn nx run-many --target=build --projects=a,c --with-deps --skip-nx-cache
Bash Log (with debug task) $ nx run-many --target=build --projects=a,c --with-deps --skip-nx-cache > NX Running target build for projects: - b - a - d - c ——————————————————————————————————————————————— > nx run d:build ---- updatePaths. dependencies: [] Building Angular Package ****************************************************************************** It is not recommended to publish Ivy libraries to NPM repositories. Read more here: https://v9.angular.io/guide/ivy#maintaining-library-compatibility ****************************************************************************** ------------------------------------------------------------------------------ Building entry point '@debug-nx-deps-problem/d' ------------------------------------------------------------------------------ Compiling TypeScript sources through ngc Bundling to FESM2015 Bundling to FESM5 Bundling to UMD Minifying UMD bundle Writing package metadata Built @debug-nx-deps-problem/d ------------------------------------------------------------------------------ Built Angular Package - from: debug-nx-deps-problem/libs/d - to: debug-nx-deps-problem/dist/libs/d ------------------------------------------------------------------------------ > nx run b:build ---- updatePaths. dependencies: [] Building Angular Package ****************************************************************************** It is not recommended to publish Ivy libraries to NPM repositories. Read more here: https://v9.angular.io/guide/ivy#maintaining-library-compatibility ****************************************************************************** ------------------------------------------------------------------------------ Building entry point '@debug-nx-deps-problem/b' ------------------------------------------------------------------------------ Compiling TypeScript sources through ngc Bundling to FESM2015 Bundling to FESM5 Bundling to UMD Minifying UMD bundle Writing package metadata Built @debug-nx-deps-problem/b ------------------------------------------------------------------------------ Built Angular Package - from: debug-nx-deps-problem/libs/b - to: debug-nx-deps-problem/dist/libs/b ------------------------------------------------------------------------------ > nx run c:build ---- checkDependentProjectsHaveBeenBuilt: [ 'debug-nx-deps-problem/dist/libs/d/package.json' ] ---- updatePaths. dependencies: [ { name: '@debug-nx-deps-problem/d', outputs: [ 'dist/libs/d' ], node: { name: 'd', type: 'lib', data: [Object] } } ] updatePaths of: @debug-nx-deps-problem/d current value: libs/d/src/index.ts overriden value: dist/libs/d Building Angular Package ****************************************************************************** It is not recommended to publish Ivy libraries to NPM repositories. Read more here: https://v9.angular.io/guide/ivy#maintaining-library-compatibility ****************************************************************************** ------------------------------------------------------------------------------ Building entry point '@debug-nx-deps-problem/c' ------------------------------------------------------------------------------ Compiling TypeScript sources through ngc Bundling to FESM2015 Bundling to FESM5 Bundling to UMD WARNING: No name was provided for external module '@debug-nx-deps-problem/d' in output.globals – guessing 'd' Minifying UMD bundle Writing package metadata Built @debug-nx-deps-problem/c ------------------------------------------------------------------------------ Built Angular Package - from: debug-nx-deps-problem/libs/c - to: debug-nx-deps-problem/dist/libs/c ------------------------------------------------------------------------------ > nx run a:build ---- checkDependentProjectsHaveBeenBuilt: [ 'debug-nx-deps-problem/dist/libs/b/package.json' ] ---- updatePaths. dependencies: [ { name: '@debug-nx-deps-problem/b', outputs: [ 'dist/libs/b' ], node: { name: 'b', type: 'lib', data: [Object] } } ] updatePaths of: @debug-nx-deps-problem/b current value: libs/b/src/index.ts overriden value: dist/libs/b Building Angular Package ****************************************************************************** It is not recommended to publish Ivy libraries to NPM repositories. Read more here: https://v9.angular.io/guide/ivy#maintaining-library-compatibility ****************************************************************************** ------------------------------------------------------------------------------ Building entry point '@debug-nx-deps-problem/a' ------------------------------------------------------------------------------ Compiling TypeScript sources through ngc Bundling to FESM2015 Bundling to FESM5 Bundling to UMD WARNING: No name was provided for external module '@debug-nx-deps-problem/b' in output.globals – guessing 'b' Minifying UMD bundle Writing package metadata Built @debug-nx-deps-problem/a ------------------------------------------------------------------------------ Built Angular Package - from: debug-nx-deps-problem/libs/a - to: debug-nx-deps-problem/dist/libs/a ------------------------------------------------------------------------------ ——————————————————————————————————————————————— > NX SUCCESS Running target "build" succeeded ✨ Done in 11.90s.
Nested (✅)

Moved d into subfolder. Graph looks good. The option "group by folder" is supported in the graph viewer. So it's kind of expected to introduce folders for grouping ?

git co scenario-nested
yarn nx dep-graph

osx-nested

Build works:

yarn debug-nx-build

# runs this:
# yarn nx run-many --target=build --projects=a,c --with-deps --skip-nx-cache
Bash Log (with debug task) $ yarn debug-nx-build $ sh debug.sh Debugging Nx Workspace Debug Patch for console output patching file node_modules/@nrwl/workspace/src/utils/buildable-libs-utils.js Clean Dist Folder nx run-many --target=build --projects=a,c --with-deps --skip-nx-cache > NX Running target build for projects: - b - a - d - c ——————————————————————————————————————————————— > nx run d:build ---- updatePaths. dependencies: [] Building Angular Package ****************************************************************************** It is not recommended to publish Ivy libraries to NPM repositories. Read more here: https://v9.angular.io/guide/ivy#maintaining-library-compatibility ****************************************************************************** ------------------------------------------------------------------------------ Building entry point '@debug-nx-deps-problem/d' ------------------------------------------------------------------------------ Compiling TypeScript sources through ngc Bundling to FESM2015 Bundling to FESM5 Bundling to UMD Minifying UMD bundle Writing package metadata Built @debug-nx-deps-problem/d ------------------------------------------------------------------------------ Built Angular Package - from: debug-nx-deps-problem/libs/nested/d - to: debug-nx-deps-problem/dist/libs/nested/d ------------------------------------------------------------------------------ > nx run b:build ---- updatePaths. dependencies: [] Building Angular Package ****************************************************************************** It is not recommended to publish Ivy libraries to NPM repositories. Read more here: https://v9.angular.io/guide/ivy#maintaining-library-compatibility ****************************************************************************** ------------------------------------------------------------------------------ Building entry point '@debug-nx-deps-problem/b' ------------------------------------------------------------------------------ Compiling TypeScript sources through ngc Bundling to FESM2015 Bundling to FESM5 Bundling to UMD Minifying UMD bundle Writing package metadata Built @debug-nx-deps-problem/b ------------------------------------------------------------------------------ Built Angular Package - from: debug-nx-deps-problem/libs/b - to: debug-nx-deps-problem/dist/libs/b ------------------------------------------------------------------------------ > nx run c:build ---- checkDependentProjectsHaveBeenBuilt: [ 'debug-nx-deps-problem/dist/libs/nested/d/package.json' ] ---- updatePaths. dependencies: [ { name: '@debug-nx-deps-problem/d', outputs: [ 'dist/libs/nested/d' ], node: { name: 'd', type: 'lib', data: [Object] } } ] updatePaths of: @debug-nx-deps-problem/d current value: libs/nested/d/src/index.ts overriden value: dist/libs/nested/d Building Angular Package ****************************************************************************** It is not recommended to publish Ivy libraries to NPM repositories. Read more here: https://v9.angular.io/guide/ivy#maintaining-library-compatibility ****************************************************************************** ------------------------------------------------------------------------------ Building entry point '@debug-nx-deps-problem/c' ------------------------------------------------------------------------------ Compiling TypeScript sources through ngc Bundling to FESM2015 Bundling to FESM5 Bundling to UMD WARNING: No name was provided for external module '@debug-nx-deps-problem/d' in output.globals – guessing 'd' Minifying UMD bundle Writing package metadata Built @debug-nx-deps-problem/c ------------------------------------------------------------------------------ Built Angular Package - from: debug-nx-deps-problem/libs/c - to: debug-nx-deps-problem/dist/libs/c ------------------------------------------------------------------------------ > nx run a:build ---- checkDependentProjectsHaveBeenBuilt: [ 'debug-nx-deps-problem/dist/libs/b/package.json' ] ---- updatePaths. dependencies: [ { name: '@debug-nx-deps-problem/b', outputs: [ 'dist/libs/b' ], node: { name: 'b', type: 'lib', data: [Object] } } ] updatePaths of: @debug-nx-deps-problem/b current value: libs/b/src/index.ts overriden value: dist/libs/b Building Angular Package ****************************************************************************** It is not recommended to publish Ivy libraries to NPM repositories. Read more here: https://v9.angular.io/guide/ivy#maintaining-library-compatibility ****************************************************************************** ------------------------------------------------------------------------------ Building entry point '@debug-nx-deps-problem/a' ------------------------------------------------------------------------------ Compiling TypeScript sources through ngc Bundling to FESM2015 Bundling to FESM5 Bundling to UMD WARNING: No name was provided for external module '@debug-nx-deps-problem/b' in output.globals – guessing 'b' Minifying UMD bundle Writing package metadata Built @debug-nx-deps-problem/a ------------------------------------------------------------------------------ Built Angular Package - from: debug-nx-deps-problem/libs/a - to: debug-nx-deps-problem/dist/libs/a ------------------------------------------------------------------------------ ——————————————————————————————————————————————— > NX SUCCESS Running target "build" succeeded ✨ Done in 12.04s.

Context: Windows (🛑)

Node: 12.14.1
Win 10 Enterprise Version 1607
Flat (✅)

Graph is good:

git co scenario-nested
yarn nx dep-graph

windows-flat

Build works:


yarn debug-nx-build

# runs this:
# yarn nx run-many --target=build --projects=a,c --with-deps --skip-nx-cache
📙 Bash Log (Debug) ``` $ nx run-many --target=build --projects=a,c --with-deps --skip-nx-cache > NX Running target build for projects: - b - a - d - c ——————————————————————————————————————————————— > nx run d:build Building Angular Package ****************************************************************************** It is not recommended to publish Ivy libraries to NPM repositories. Read more here: https://v9.angular.io/guide/ivy#maintaining-library-compatibility ****************************************************************************** ------------------------------------------------------------------------------ Building entry point '@debug-nx-deps-problem/d' ------------------------------------------------------------------------------ Compiling TypeScript sources through ngc Bundling to FESM2015 Bundling to FESM5 Bundling to UMD Minifying UMD bundle Writing package metadata Built @debug-nx-deps-problem/d ------------------------------------------------------------------------------ Built Angular Package - from: nx-debug-nested-dep-graph-scenario-flat\libs\d - to: nx-debug-nested-dep-graph-scenario-flat\dist\libs\d ------------------------------------------------------------------------------ > nx run b:build Building Angular Package ****************************************************************************** It is not recommended to publish Ivy libraries to NPM repositories. Read more here: https://v9.angular.io/guide/ivy#maintaining-library-compatibility ****************************************************************************** ------------------------------------------------------------------------------ Building entry point '@debug-nx-deps-problem/b' ------------------------------------------------------------------------------ Compiling TypeScript sources through ngc Bundling to FESM2015 Bundling to FESM5 Bundling to UMD Minifying UMD bundle Writing package metadata Built @debug-nx-deps-problem/b ------------------------------------------------------------------------------ Built Angular Package - from: nx-debug-nested-dep-graph-scenario-flat\libs\b - to: nx-debug-nested-dep-graph-scenario-flat\dist\libs\b ------------------------------------------------------------------------------ > nx run c:build Building Angular Package ****************************************************************************** It is not recommended to publish Ivy libraries to NPM repositories. Read more here: https://v9.angular.io/guide/ivy#maintaining-library-compatibility ****************************************************************************** ------------------------------------------------------------------------------ Building entry point '@debug-nx-deps-problem/c' ------------------------------------------------------------------------------ Compiling TypeScript sources through ngc Bundling to FESM2015 Bundling to FESM5 Bundling to UMD WARNING: No name was provided for external module '@debug-nx-deps-problem/d' in output.globals – guessing 'd' Minifying UMD bundle Writing package metadata Built @debug-nx-deps-problem/c ------------------------------------------------------------------------------ Built Angular Package - from: nx-debug-nested-dep-graph-scenario-flat\libs\c - to: nx-debug-nested-dep-graph-scenario-flat\dist\libs\c ------------------------------------------------------------------------------ > nx run a:build Building Angular Package ****************************************************************************** It is not recommended to publish Ivy libraries to NPM repositories. Read more here: https://v9.angular.io/guide/ivy#maintaining-library-compatibility ****************************************************************************** ------------------------------------------------------------------------------ Building entry point '@debug-nx-deps-problem/a' ------------------------------------------------------------------------------ Compiling TypeScript sources through ngc Bundling to FESM2015 Bundling to FESM5 Bundling to UMD WARNING: No name was provided for external module '@debug-nx-deps-problem/b' in output.globals – guessing 'b' Minifying UMD bundle Writing package metadata Built @debug-nx-deps-problem/a ------------------------------------------------------------------------------ Built Angular Package - from: nx-debug-nested-dep-graph-scenario-flat\libs\a - to: nx-debug-nested-dep-graph-scenario-flat\dist\libs\a ------------------------------------------------------------------------------ ——————————————————————————————————————————————— > NX SUCCESS Running target "build" succeeded Done in 95.17s. ```
Nested (🛑)

Moved d into subfolder. Graph is wrong.

git co scenario-nested
yarn nx dep-graph

windows-nested

Build fails:


yarn debug-nx-build

# runs this:
# yarn nx run-many --target=build --projects=a,c --with-deps --skip-nx-cache

'rootDir' is expected to contain all source files. because the nested dependency d is not recognized as the dependency of c. That's why c is not build and the tsconfig is not updated properly.

📙 Bash Log (Debug) ``` $ yarn debug-nx-build $ sh debug.sh Debugging Nx Workspace Debug Patch for console output patching file node_modules/@nrwl/workspace/src/utils/buildable-libs-utils.js Clean Dist Folder $ nx run-many --target=build --projects=a,c --with-deps --skip-nx-cache > NX Running target build for projects: - b - a - c ——————————————————————————————————————————————— > nx run c:build ---- updatePaths. dependencies: [] Building Angular Package ****************************************************************************** It is not recommended to publish Ivy libraries to NPM repositories. Read more here: https://v9.angular.io/guide/ivy#maintaining-library-compatibility ****************************************************************************** ------------------------------------------------------------------------------ Building entry point '@debug-nx-deps-problem/c' ------------------------------------------------------------------------------ Compiling TypeScript sources through ngc ERROR: libs/nested/d/src/index.ts:1:15 - error TS6059: File 'nx-debug-nested-dep-graph-scenario-nested/libs/nested/d/src/lib/d.module.ts' is not under 'rootDir' 'nx-debug-nested-dep-graph-scenario-nested\libs\c\src'. 'rootDir' is expected to contain all source files. 1 export * from './lib/d.module'; ~~~~~~~~~~~~~~~~ libs/c/src/index.ts:2:19 - error TS6059: File 'nx-debug-nested-dep-graph-scenario-nested/libs/nested/d/src/index.ts' is not under 'rootDir' 'nx-debug-nested-dep-graph-scenario-nested\libs\c\src'. 'rootDir' is expected to contain all source files. 2 import { D } from '@debug-nx-deps-problem/d'; ~~~~~~~~~~~~~~~~~~~~~~~~~~ libs/nested/d/src/index.ts:1:15 - error TS6059: File 'nx-debug-nested-dep-graph-scenario-nested/libs/nested/d/src/lib/d.module.ts' is not under 'rootDir' 'nx-debug-nested-dep-graph-scenario-nested\libs\c\src'. 'rootDir' is expected to contain all source files. 1 export * from './lib/d.module'; ~~~~~~~~~~~~~~~~ libs/c/src/index.ts:2:19 - error TS6059: File 'nx-debug-nested-dep-graph-scenario-nested/libs/nested/d/src/index.ts' is not under 'rootDir' 'nx-debug-nested-dep-graph-scenario-nested\libs\c\src'. 'rootDir' is expected to contain all source files. 2 import { D } from '@debug-nx-deps-problem/d'; ~~~~~~~~~~~~~~~~~~~~~~~~~~ > nx run b:build ---- updatePaths. dependencies: [] Building Angular Package ****************************************************************************** It is not recommended to publish Ivy libraries to NPM repositories. Read more here: https://v9.angular.io/guide/ivy#maintaining-library-compatibility ****************************************************************************** ------------------------------------------------------------------------------ Building entry point '@debug-nx-deps-problem/b' ------------------------------------------------------------------------------ Compiling TypeScript sources through ngc Bundling to FESM2015 Bundling to FESM5 Bundling to UMD Minifying UMD bundle Writing package metadata Built @debug-nx-deps-problem/b ------------------------------------------------------------------------------ Built Angular Package - from: nx-debug-nested-dep-graph-scenario-nested\libs\b - to: nx-debug-nested-dep-graph-scenario-nested\dist\libs\b ------------------------------------------------------------------------------ ——————————————————————————————————————————————— > NX ERROR Running target "build" failed Failed projects: - c - a You can isolate the above projects by passing: --only-failed error Command failed with exit code 1. info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command. Revert Debug Patch patching file node_modules/@nrwl/workspace/src/utils/buildable-libs-utils.js Done in 27.16s. ```

Other

This errors happens in production and the only workaround for the Windows crew so far is adding the nested dependency as an Implicit Dependency to nx.json. I'm unsure about the implications that's why I decided to raise this issue.

Related:

Thank you for your help 🙏

jbjhjm commented 4 years ago

Alright, I did not realize this issue might be exactly the same as mine but after looking into the example I think it is. I've run the test setup provided by @georgiee and got the same results.

Setup

My Node and Win versions are strongly different to gorgiee's, so the problem seems to affect at least a large range of node and win10 versions:

Win10 Pro 1903 Node v10.20.0 Nx 9.2.4 Within my own repo, I also experienced the problem with 9.1.2 before updating to 9.2.4.

Results

Here are screenshots and the relevant pieces of json data received by using dep-graph --file.

Flat setup works:

    "dependencies": {
      "debug-nx-deps-problem-e2e": [...],
      "debug-nx-deps-problem": [],
      "a": [
        {
          "type": "static",
          "source": "a",
          "target": "b"
        }
      ],
      "b": [],
      "c": [
        {
          "type": "static",
          "source": "c",
          "target": "d"
        }
      ],
      "d": []
    }

depgraph_issue_flat

As soon as package d is located in libs/nested/d, dependency detection fails. Maybe worth a mention: Comparing the json output one can see in nested setup, package d is listed BEFORE package a b and c. Seems the evaluation order changes. This might be a hint on what's happening.

    "dependencies": {
      "debug-nx-deps-problem-e2e": [...],
      "debug-nx-deps-problem": [],
      "d": [],
      "a": [
        {
          "type": "static",
          "source": "a",
          "target": "b"
        }
      ],
      "b": [],
      "c": []
    }

depgraph_issue_nested

georgiee commented 4 years ago

Thank you @jbjhjm for confirming this issue. The order of the json output is the direct result of the wrong dependency map so I think that's just another manifestation of the problem and not the source.

I already did some debugging but I have no affected machine at my hands so it was always remotely on a colleague's machine which is pretty cumbersome. A quick check in a Windows 10 VM did not show the same effects (although I went over this pretty quickly and I should try this again).

I believe the error is buried in here: https://github.com/nrwl/nx/blob/10282bbcc9afb99cd6ef453ebd09b693ecc558b0/packages/workspace/src/core/project-graph/build-dependencies/explicit-project-dependencies.ts#L10-L35

Especially in the referenced TypeScriptImportLocator. Your confirmation gives me some motivation to dig again through the sources. If you will adventurous you could try monkey patching the files in node_modules/@nrwl/workspace/src/core/project-graph/* or node_modules/@nrwl/workspace/src/utils/buildable-libs-utils.jswith some console logs ( I did that in the example repo to output some debugging notes)

Then run the graph over and over again and check your output for something conspicuous. I will check it too but I need to let it fail locally somehow.

jbjhjm commented 4 years ago

I might give it a try. Would love to have this fixed before continueing app & lib development

jbjhjm commented 4 years ago

ok, I was able to track it down.

tl;dr: On windows, backslashes are preventing path-based detection to work correctly. in utils/app-root::pathInner(), sanitize backslashes by converting to slashes.

findProjectWithImport strategy

TargetProjectLocator.findProjectWithImport function located in workspace/src/core/target-project-locator.js checks if the import points to an actual project within the workspace by doing three things:

  1. isWorkspaceProject -- returns false if project type is neither of app lib or e2e.

  2. resolvedModule.startsWith() -- compares import paths with paths of known libs & apps. Broken on windows.

  3. comparision by scope. This answers the initial question of my own open ticket. It will compare the import path to a GUESSED typescript import alias being @(/)/project. If you'd import from d using "@debug-nx-deps-problem/nested/d", it should detect the dependency.

the issue

resolvedModule.startsWith(p.data.root)

ResolvedModule is the ABSOLUTE path on windows. So startWith can NEVER start with RELATIVE p.data.root.

the bug

resolvedModule path should be relative. It tries to replace absolute path using utils.app_root. As app_root contains backslashes on windows, it fails to convert path to relative path.

return resolvedModule.resolvedFileName.replace(`${app_root_1.appRootPath}/`, '');

Fix for issue

in utils/app-root::pathInner(), sanitize backslashes by converting to slashes. Edit: Yeah well there already is a note this needs to be fixed... // TODO: vsavkin normalize the path

Possible fix:

exports.appRootPath = sanitize(pathInner(__dirname));
function sanitize(dir) {
    return dir.replace(/\\/g,'/')
}

scope issue

tracked in https://github.com/nrwl/nx/issues/2982

georgiee commented 4 years ago

Wow great analysis & conclusion @jbjhjm. That you found the TODO: vsavkin normalize the path where the bug happens is the great irony of this story 😄 Do you think you could craft a small PR for this change as you already have pointed out a possible fix? Depending on the given CI/CD pipeline it could be as easy as editing the file via GitHub and let the CI confirm it's working. A single test pinning this scenario to prevent any future regression would make it perfect.

I'm happy to review it or contribute a test tomorrow and I bet @vsavkin would be happy to review such a PR too.

georgiee commented 4 years ago

Hello @jbjhjm,

Regarding :

findProjectWithImport comparision by scope. This answers the initial question of my own open ticket. It will compare the import path to a GUESSED typescript import alias being @(/)/project. If you'd import from d using "@debug-nx-deps-problem/nested/d", it should detect the dependency.

I dived into the sources this morning. One addition for the wild guessing would be finding the package.json if it's publishable. This would yield the correct result.

const packageCandidate = path.resolve(p.data.root, './package.json');

if(fs.existsSync(packageCandidate)) {
    const publishablePackageName = require(packageCandidate).name
    if(normalizedImportExpr.startsWith(publishablePackageName)) {
        return true;
    }
}

I did a quick check where I removed the resolvedModule check entirely. The dep-graph yielded a perfect graph (because many of our packages are publishable). So this is indeed a good fallback. On the other hand, if resolvedModule is fixed and correctly retrieved. Then resolvedModule.startsWith(p.data.root) should work as expected and we don't need any guessing at all, do we ? I suspect the guessing is in place because of the path issues in the first place.

jbjhjm commented 4 years ago

Opened pull request https://github.com/nrwl/nx/pull/3028 @FrozenPandaz / @philipjfulcher can you check?

georgiee commented 4 years ago

Some more notes:

  1. PR addressing the issue and opened by @jbjhjm https://github.com/nrwl/nx/pull/3028

  2. Prior Art by @rhutchison https://github.com/nrwl/nx/pull/2941

Related Issues linked by the PR in @rhutchison

georgiee commented 4 years ago

Update: One of the pending PRs with the fix has been merged. Let's wait for the release to see this issue being fixed 🙏

philipjfulcher commented 4 years ago

@georgiee Have you pulled in our latest releases? Was the issue resolved?

georgiee commented 4 years ago

Hello and thanks for your question. We couldn't upgrade to Nx 10 yet due to some weird errors with Angular CLI aorund the solution style tsconfig changes and the angular cli vs workspace.json issues.

https://github.com/nrwl/nx/issues/3464#issuecomment-669012951

I always expected this will hinder me to upgrade the example app — I will give it a try so we can check the status of this issue. Thanks.

philipjfulcher commented 4 years ago

@georgiee This fix actually came in during 9.x development, no need to update to 10. I updated an older version of your example repo to 9.6 and 10.1 and the dep graph appears correct in both versions. It looks like your example repo has also been updated to Nx 10 on the master branch. Are you seeing the correct graph there?

Are you able to test this with your actual repo instead of just the example? I think this is resolved but wanted to check.

philipjfulcher commented 4 years ago

This should be fixed in the latest version of 9.x and 10.x of Nx. Please open another bug if you see it again after upgrading to the latest version.

github-actions[bot] commented 1 year ago

This issue has been closed for more than 30 days. If this issue is still occuring, please open a new issue with more recent context.