search

nrwl / nx

Smart Monorepos · Fast CI
https://nx.dev
MIT License
23.65k stars 2.36k forks source link

nx 18's built types include problematic references to node's fs #21658

Open ianwremmel opened 9 months ago

ianwremmel commented 9 months ago

Current Behavior

nx 18 induces the following type errors:

node_modules/@types/node/fs.d.ts:21:1 - error TS6200: Definitions of the following identifiers conflict with those in another file: Type, promises, PathLike, PathOrFileDescriptor, TimeLike, NoParamCallback, BufferEncodingOption, EncodingOption, OpenMode, Mode, Stats, StatsFs, Dirent, Dir, ReadStream, WriteStream, statSync, lstatSync, ReadPosition, WriteFileOptions, WatchEventType, WatchListener, StatsListener, BigIntStatsListener, F_OK, R_OK, W_OK, X_OK, COPYFILE_EXCL, COPYFILE_FICLONE, COPYFILE_FICLONE_FORCE, O_RDONLY, O_WRONLY, O_RDWR, O_CREAT, O_EXCL, O_NOCTTY, O_TRUNC, O_APPEND, O_DIRECTORY, O_NOATIME, O_NOFOLLOW, O_SYNC, O_DSYNC, O_SYMLINK, O_DIRECT, O_NONBLOCK, S_IFMT, S_IFREG, S_IFDIR, S_IFCHR, S_IFBLK, S_IFIFO, S_IFLNK, S_IFSOCK, S_IRWXU, S_IRUSR, S_IWUSR, S_IXUSR, S_IRWXG, S_IRGRP, S_IWGRP, S_IXGRP, S_IRWXO, S_IROTH, S_IWOTH, S_IXOTH, UV_FS_O_FILEMAP

21 declare module "fs" {
   ~~~~~~~

  node_modules/@types/node/ts4.8/fs.d.ts:21:1
    21 declare module "fs" {
       ~~~~~~~
    Conflicts are in this file.

node_modules/@types/node/ts4.8/fs.d.ts:21:1 - error TS6200: Definitions of the following identifiers conflict with those in another file: Type, promises, PathLike, PathOrFileDescriptor, TimeLike, NoParamCallback, BufferEncodingOption, EncodingOption, OpenMode, Mode, Stats, StatsFs, Dirent, Dir, ReadStream, WriteStream, statSync, lstatSync, ReadPosition, WriteFileOptions, WatchEventType, WatchListener, StatsListener, BigIntStatsListener, F_OK, R_OK, W_OK, X_OK, COPYFILE_EXCL, COPYFILE_FICLONE, COPYFILE_FICLONE_FORCE, O_RDONLY, O_WRONLY, O_RDWR, O_CREAT, O_EXCL, O_NOCTTY, O_TRUNC, O_APPEND, O_DIRECTORY, O_NOATIME, O_NOFOLLOW, O_SYNC, O_DSYNC, O_SYMLINK, O_DIRECT, O_NONBLOCK, S_IFMT, S_IFREG, S_IFDIR, S_IFCHR, S_IFBLK, S_IFIFO, S_IFLNK, S_IFSOCK, S_IRWXU, S_IRUSR, S_IWUSR, S_IXUSR, S_IRWXG, S_IRGRP, S_IWGRP, S_IXGRP, S_IRWXO, S_IROTH, S_IWOTH, S_IXOTH, UV_FS_O_FILEMAP

21 declare module "fs" {
   ~~~~~~~

  node_modules/@types/node/fs.d.ts:21:1
    21 declare module "fs" {
       ~~~~~~~
    Conflicts are in this file.

node_modules/nx/src/native/index.d.ts:48:18 - error TS2304: Cannot find name 'JsInputs'.

48   inputs?: Array<JsInputs>
                    ~~~~~~~~

node_modules/nx/src/native/index.d.ts:55:38 - error TS2304: Cannot find name 'JsInputs'.

55   namedInputs?: Record<string, Array<JsInputs>>
                                        ~~~~~~~~

node_modules/nx/src/native/index.d.ts:114:38 - error TS2304: Cannot find name 'JsInputs'.

114   namedInputs?: Record<string, Array<JsInputs>>
                                         ~~~~~~~~

node_modules/nx/src/native/index.d.ts:131:19 - error TS2304: Cannot find name 'ProjectFiles'.

131   projectFileMap: ProjectFiles
                      ~~~~~~~~~~~~

node_modules/nx/src/native/index.d.ts:136:32 - error TS2304: Cannot find name 'ProjectFiles'.

136   projectFiles: ExternalObject<ProjectFiles>
                                   ~~~~~~~~~~~~

node_modules/nx/src/native/index.d.ts:145:19 - error TS2304: Cannot find name 'ProjectFiles'.

145   projectFileMap: ProjectFiles
                      ~~~~~~~~~~~~

node_modules/nx/src/native/index.d.ts:163:68 - error TS2552: Cannot find name 'JsExternal'. Did you mean 'External'?

163   getPlansReference(taskIds: Array<string>, taskGraph: TaskGraph): JsExternal
                                                                       ~~~~~~~~~~

  node_modules/typescript/lib/lib.dom.d.ts:8252:13
    8252 declare var External: {
                     ~~~~~~~~
    'External' is declared here.

node_modules/nx/src/native/index.d.ts:166:113 - error TS2304: Cannot find name 'ProjectFiles'.

166   constructor(workspaceRoot: string, projectGraph: ExternalObject<ProjectGraph>, projectFileMap: ExternalObject<ProjectFiles>, allWorkspaceFiles: ExternalObject<Array<FileData>>, tsConfig: Buffer, tsConfigPaths: Record<string, Array<string>>, options?: HasherOptions | undefined | null)
                                                                                                                    ~~~~~~~~~~~~

node_modules/nx/src/native/index.d.ts:167:60 - error TS2304: Cannot find name 'HashInstruction'.

167   hashPlans(hashPlans: ExternalObject<Record<string, Array<HashInstruction>>>, jsEnv: Record<string, string>): NapiDashMap
                                                               ~~~~~~~~~~~~~~~

node_modules/nx/src/native/index.d.ts:167:112 - error TS2304: Cannot find name 'NapiDashMap'.

167   hashPlans(hashPlans: ExternalObject<Record<string, Array<HashInstruction>>>, jsEnv: Record<string, string>): NapiDashMap
                                                                                                                   ~~~~~~~~~~~

node_modules/nx/src/native/index.d.ts:189:43 - error TS2304: Cannot find name 'ProjectRootMappings'.

189   updateProjectFiles(projectRootMappings: ProjectRootMappings, projectFiles: ExternalObject<ProjectFiles>, globalFiles: ExternalObject<Array<FileData>>, updatedFiles: Record<string, string>, deletedFiles: Array<string>): UpdatedWorkspaceFiles
                                              ~~~~~~~~~~~~~~~~~~~

node_modules/nx/src/native/index.d.ts:189:93 - error TS2304: Cannot find name 'ProjectFiles'.

189   updateProjectFiles(projectRootMappings: ProjectRootMappings, projectFiles: ExternalObject<ProjectFiles>, globalFiles: ExternalObject<Array<FileData>>, updatedFiles: Record<string, string>, deletedFiles: Array<string>): UpdatedWorkspaceFiles
                                                                                                ~~~~~~~~~~~~

Found 14 errors.

Expected Behavior

no type errors

GitHub Repo

No response

Steps to Reproduce

  1. update a monorepo to nx 18.0.1 or 18.0.2
  2. run the typescript compiler

Nx Report

>  NX   Report complete - copy this into the issue template

   Node   : 20.11.0
   OS     : darwin-arm64
   npm    : 8.19.4

   nx          : 18.0.2
   @nx/devkit  : 18.0.2
   @nrwl/tao   : 18.0.2
   typescript  : 5.3.3
   ---------------------------------------
   Local workspace plugins:
     @code-like-a-carpenter/foundation-cli
     @code-like-a-carpenter/tooling-deps
     @clc/nx

Failure Logs

No response

Package Manager Version

10.2.4

Operating System

Additional Information

From what I can tell, there are two issues here:

  1. a number of types are missed when src/native/index.d.ts is generated from rust code. This has been a known issue since atleast 17.2.0.
  2. src/utils/fileutils.d.ts and src/generators/tree.d.ts include /// <reference types="@types/node/ts4.8/fs" />, which induces a second import of the typedefs for node's fs.

The following diff fixes the issues:

diff --git a/node_modules/nx/bin/nx-cloud.js b/node_modules/nx/bin/nx-cloud.js
old mode 100755
new mode 100644
diff --git a/node_modules/nx/src/generators/tree.d.ts b/node_modules/nx/src/generators/tree.d.ts
index 2ac8002..633eb18 100644
--- a/node_modules/nx/src/generators/tree.d.ts
+++ b/node_modules/nx/src/generators/tree.d.ts
@@ -1,6 +1,3 @@
-/// <reference types="node" />
-/// <reference types="@types/node/ts4.8/fs" />
-/// <reference types="node" />
 import type { Mode } from 'fs';
 /**
  * Options to set when writing a file in the Virtual file system tree.
diff --git a/node_modules/nx/src/native/index.d.ts b/node_modules/nx/src/native/index.d.ts
index 3a3fc09..a9b209e 100644
--- a/node_modules/nx/src/native/index.d.ts
+++ b/node_modules/nx/src/native/index.d.ts
@@ -1,6 +1,13 @@
 /* tslint:disable */
 /* eslint-disable */

+type JsInputs = unknown
+type JsExternal = unknown
+type HashInstruction = unknown
+type ProjectFiles = unknown
+type ProjectRootMappings = unknown
+type NapiDashMap = unknown
+
 /* auto-generated by NAPI-RS */

 export class ExternalObject<T> {
diff --git a/node_modules/nx/src/utils/fileutils.d.ts b/node_modules/nx/src/utils/fileutils.d.ts
index db1decb..4993774 100644
--- a/node_modules/nx/src/utils/fileutils.d.ts
+++ b/node_modules/nx/src/utils/fileutils.d.ts
@@ -1,5 +1,3 @@
-/// <reference types="node" />
-/// <reference types="@types/node/ts4.8/fs" />
 import type { JsonParseOptions, JsonSerializeOptions } from './json';
 import { PathLike } from 'fs';
 export interface JsonReadOptions extends JsonParseOptions {
Humpelig commented 9 months ago

+1, same Behavior on Windows. My build is blocked on typedoc --validation.

FrozenPandaz commented 6 months ago

Hm, we don't have those comments in our code... @JamesHenry do you know what would cause tsc to put these comments in? :thinking:

JamesHenry commented 5 months ago

We seem to have an old node types reference in our package.json -> "@types/node": "18.19.8", - I think that could be dragging in old typings (I'm basing that on the pnpm reference that gets created):

node_modules/.pnpm/@types+node@18.19.8/node_modules/@types/node/ts4.8/fs.d.ts

image
JamesHenry commented 5 months ago

Actually it seems the issue with the old typings is already resolved:

https://www.npmjs.com/package/nx/v/19.1.1?activeTab=code

image

I opened a node 20 typings PR just in case we want that but feel free to close again if we don't @FrozenPandaz: https://github.com/nrwl/nx/pull/26260

Somebody will need to look into why the JsInputs etc types are not being generated by napi-rs, because that still seems to be an issue on the latest.

ianwremmel commented 5 months ago

It looks like the latest release introduces more invalid typings. Somewhere along the way | undefined got added to a bunch of things, so the current nested referencing of e.g. NxJsonConfiguration['release']['changelog'] is invalid. ~Here's the latest patch I'm using:~

edit: deleted patch because it was large and not relevant

ianwremmel commented 5 months ago

Actually, wait, I take that back. That's for the latest 18.x release. I haven't upgrade to 19 yet. Let me do that and get back to you.

ianwremmel commented 5 months ago

ok, yea, the type references have been fixed, but the generated types have indeed gotten more problematic. Here's that patch I need with v19.1.1 to make typescript happy:

diff --git a/node_modules/nx/bin/nx-cloud.js b/node_modules/nx/bin/nx-cloud.js
old mode 100755
new mode 100644
diff --git a/node_modules/nx/src/command-line/release/config/config.d.ts b/node_modules/nx/src/command-line/release/config/config.d.ts
index 04c2921..e31b942 100644
--- a/node_modules/nx/src/command-line/release/config/config.d.ts
+++ b/node_modules/nx/src/command-line/release/config/config.d.ts
@@ -25,6 +25,11 @@ type RemoveBooleanFromPropertiesOnEach<T, K extends keyof T[keyof T]> = {
     [U in keyof T]: RemoveBooleanFromProperties<T[U], K>;
 };
 export declare const IMPLICIT_DEFAULT_RELEASE_GROUP = "__default__";
+
+type DefiniteRelease = Exclude<NxJsonConfiguration['release'], undefined>;
+type DefiniteChangeLog = Exclude<DefiniteRelease['changelog'], undefined>;
+type DefiniteConventionalCommits = Exclude<DefiniteRelease['conventionalCommits'], undefined>;
+type DefiniteTypes = Exclude<DefiniteConventionalCommits['types'], undefined>;
 /**
  * Our source of truth is a deeply required variant of the user-facing config interface, so that command
  * implementations can be sure that properties will exist and do not need to repeat the same checks over
@@ -34,11 +39,11 @@ export declare const IMPLICIT_DEFAULT_RELEASE_GROUP = "__default__";
  * it easier to work with (the user could be specifying a single string, and they can also use any valid matcher
  * pattern such as directories and globs).
  */
-export type NxReleaseConfig = Omit<DeepRequired<NxJsonConfiguration['release'] & {
-    groups: DeepRequired<RemoveTrueFromPropertiesOnEach<EnsureProjectsArray<NxJsonConfiguration['release']['groups']>, 'changelog'>>;
-    changelog: RemoveTrueFromProperties<DeepRequired<NxJsonConfiguration['release']['changelog']>, 'workspaceChangelog' | 'projectChangelogs'>;
+export type NxReleaseConfig = Omit<DeepRequired<DefiniteRelease & {
+    groups: DeepRequired<RemoveTrueFromPropertiesOnEach<EnsureProjectsArray<DefiniteRelease['groups']>, 'changelog'>>;
+    changelog: RemoveTrueFromProperties<DeepRequired<DefiniteChangeLog>, 'workspaceChangelog' | 'projectChangelogs'>;
     conventionalCommits: {
-        types: RemoveBooleanFromPropertiesOnEach<DeepRequired<RemoveBooleanFromProperties<DeepRequired<NxJsonConfiguration['release']['conventionalCommits']['types']>, string>>, 'changelog'>;
+        types: RemoveBooleanFromPropertiesOnEach<DeepRequired<RemoveBooleanFromProperties<DeepRequired<DefiniteTypes>, string>>, 'changelog'>;
     };
 }>, 'projects'>;
 export interface CreateNxReleaseConfigError {
diff --git a/node_modules/nx/src/command-line/release/config/filter-release-groups.d.ts b/node_modules/nx/src/command-line/release/config/filter-release-groups.d.ts
index bffd713..4a50e9c 100644
--- a/node_modules/nx/src/command-line/release/config/filter-release-groups.d.ts
+++ b/node_modules/nx/src/command-line/release/config/filter-release-groups.d.ts
@@ -1,6 +1,7 @@
 import { ProjectGraph } from '../../../config/project-graph';
 import { NxReleaseConfig } from './config';
-export type ReleaseGroupWithName = NxReleaseConfig['groups'][string] & {
+type DefiniteGroups = Exclude<NxReleaseConfig['groups'], undefined>;
+export type ReleaseGroupWithName = DefiniteGroups[string] & {
     name: string;
 };
 export declare function filterReleaseGroups(projectGraph: ProjectGraph, nxReleaseConfig: NxReleaseConfig, projectsFilter?: string[], groupsFilter?: string[]): {
diff --git a/node_modules/nx/src/native/index.d.ts b/node_modules/nx/src/native/index.d.ts
index 3bd096b..71f4d62 100644
--- a/node_modules/nx/src/native/index.d.ts
+++ b/node_modules/nx/src/native/index.d.ts
@@ -1,6 +1,13 @@
 /* tslint:disable */
 /* eslint-disable */

+type JsInputs = unknown
+type JsExternal = unknown
+type HashInstruction = unknown
+type ProjectFiles = unknown
+type ProjectRootMappings = unknown
+type NapiDashMap = unknown
+
 /* auto-generated by NAPI-RS */

 export class ExternalObject<T> {
ZeroZeroOne-dev commented 4 months ago

Any update on a permanent fix? We're currently experiencing these issues migrating our workspace from nx 15 to 19