Open lorenzodejong opened 6 months ago
I ran into the same issue with an Angular project with some "publishable" libraries. #21560
When I publish them (via either "nx release" or via "nx run my-library:nx-release-publish") the resulting npm package is the source code i.e. the "my-library/src/" folder. I expected it to publish the the compiled "dist/my-library"
@lorenzodejong are you trying to have a standalone nx workspace? Or a nx workspace that could eventually contain multiple packages? Your description and steps to reproduce are not completely clear for me.
However, as I'm an nx enthusiast, I saw your issue and tried to see if I could be of any help. Therefor I have prepared a github repository containing a standalone typescript nx workspace:
https://github.com/thdk/nx-release-standalone-ts-library/tree/nrwl/nx/issues/21855
I have also documented my steps in the readme file in order to get it published using nx release
command.
The final output looks like:
β― npx nx release publish --dry-run
> NX Running target nx-release-publish for project my-lib:
- my-lib
With additional flags:
--dryRun=true
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
> nx run my-lib:nx-release-publish
π¦ @my-org/my-lib@0.0.1
=== Tarball Contents ===
1.1kB LICENSE
10.3kB README.md
30B dist/index.d.ts
200B dist/index.js
116B dist/index.js.map
41B dist/lib/my-lib.d.ts
200B dist/lib/my-lib.js
172B dist/lib/my-lib.js.map
10.6kB dist/README.md
1.0kB package.json
=== Tarball Details ===
name: @my-org/my-lib
version: 0.0.1
filename: my-org-my-lib-0.0.1.tgz
package size: 4.3 kB
unpacked size: 23.8 kB
shasum: 759b65eeb426b540ea9d93f946c0854f352dcd4b
integrity: sha512-KLpammVULAbED[...]xZDugQ6nIdk4w==
total files: 10
Would publish to https://registry.npmjs.org/ with tag "latest", but [dry-run] was set
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
> NX Successfully ran target nx-release-publish for project my-lib
There is one downside on my setup, it doesn't use the generated package.json
file from the build
target but used package.json
file from the workspace root. I have found another way to make it work with the generate package.json
as well but still unsure if that approach would be valid. As in, not documented, probably likely to break.
Update: below is the solution which sets the packageDir
option for the nx-release-publish target to use the generated package.json
for publishing:
Either configure per project in project.json
"targets": {
"nx-release-publish": {
"options": {
"packageRoot": "dist/{packageName}"
}
}
},
or globally as targetDefaults
in nx.json
"targetDefaults": {
"nx-release-publish": {
"options": {
"packageRoot": "dist/{packageName}"
}
}
},
The result output for running nx release publish
:
β― npx nx release publish --dry-run
NX Running target nx-release-publish for project my-lib:
- my-lib
With additional flags: --dryRun=true
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
nx run my-lib:nx-release-publish
π¦ @my-org/my-lib@1.1.0 === Tarball Contents ===
276B CHANGELOG.md
10.9kB README.md
30B index.d.ts
200B index.js
116B index.js.map
41B lib/my-lib.d.ts
200B lib/my-lib.js
172B lib/my-lib.js.map
976B package.json
=== Tarball Details ===
name: @my-org/my-lib
version: 1.1.0
filename: my-org-my-lib-1.1.0.tgz
package size: 3.6 kB
unpacked size: 12.9 kB
shasum: dceb978cfcb47d8f9f34d361e7dc6cec8dd9d80e
integrity: sha512-JlZ33VTeH+dD7[...]i+Gqn+593Rrgw==
total files: 9
Would publish to https://registry.npmjs.org/ with tag "latest", but [dry-run] was set
@thdk thanks for the extensive answer! I think my original message was indeed incorrect, i'm creating a monorepo with multiple packages. I think you have to choose for either a Package-based monorepo
or a Integrated Monorepo
to get this same behavior. I'll add that in my original reproduction steps.
The point you touch upon with the generated package.json
might be a related issue indeed, where the package.json
from the source is being used instead of the one from the build output.
Think I found the solution.
When running npx nx release publish
you see eventually the following:
> NX Successfully ran target nx-release-publish for project my-lib
Note the name of the target.
When I now look at the nx.json
file from @nrwl/nx
(this repo). I see the following:
"targetDefaults": {
"nx-release-publish": {
"options": {
"packageRoot": "build/packages/{projectName}"
}
},
}
So assuming you have a folder packages
containing your ts / js package source code, and each project is build output is placed in dist/packages/{projectName}
you can adjust this configuration to match your own dist
folder.
"targetDefaults": {
"nx-release-publish": {
"options": {
"packageRoot": "dist/packages/{projectName}"
}
},
}
We can also tell Nx to bump the version in the generated package.json
using the following config in nx.json
:
"release": {
"git": {
"commit": true,
"tag": true
},
"projectsRelationship": "independent",
"version": {
"generatorOptions": {
"specifierSource": "conventional-commits",
"currentVersionResolver": "git-tag",
"fallbackCurrentVersionResolver": "disk",
"packageRoot": "dist/packages/{projectName}"
}
},
}
Using conventional commits here to get the new version and git tags to find previously released versions since the version will not be bumped in the project's package.json
file but only in the generated package.json
. I believe other options are possible as well such as fetching existing version from the registry.
Example repo: https://github.com/thdk/nx-release-ts-packages
I'm running into the same root issue. After adding "packageRoot": "dist/libs/{projectName}"
to my nx.json, that solves the problem of the correct compiled code packed and published to the registry.
But the next problem is that the version in the source code libs/foo/package.json
gets updated via nx release version
subcommand, but the dist/libs/foo/package.json
does not get updated. So the publish fails because the dist version is stale and that version will already exist on the registry.
If i understand the comment above, you can workaround this issue by configuring release version to update the dist folder? But then the source folder is not matching still and that could lead to confusion as well? You'd just be relying on git tags to know what version your pkgs are on?
Yeah i also played around a bit more with this setup and i'm running into the same issue as @acc-nicholas is mentioning. The package.json
in the dist gets updated, however isn't updated from a commit in the source folder. The command even fails because of this behavior, as it doesn't detect any git changes.
Also in the last comment of @thdk it's important to note that the configuration option is packageRoot
instead of projectRoot
:
"release": {
"git": {
"commit": true,
"tag": true
},
"projectsRelationship": "independent",
"version": {
"generatorOptions": {
"specifierSource": "conventional-commits",
"currentVersionResolver": "git-tag",
"fallbackCurrentVersionResolver": "disk",
"packageRoot": "dist/packages/{projectName}"
}
},
}
i tried adding dependsOn
to my nx.json
, but that also didn't work.
"nx-release-publish": {
"dependsOn": ["build"],
"options": {
"packageRoot": "dist/libs/{projectName}"
}
},
If i understand the comment above, you can workaround this issue by configuring release version to update the dist folder? But then the source folder is not matching still and that could lead to confusion as well? You'd just be relying on git tags to know what version your pkgs are on?
Sorry for the type in my previous comment (and linked repo). I have fixed that for future readers.
But you are right that, also as far as I know at the moment, with this setup git tags would become your source of truth. The version property of your package.json
in libs/{projectName}
would never be touched.
This now becomes the same issue as my raised issue here: https://github.com/nrwl/nx/issues/20936
@fahslaj could you perhaps provide some insight in the recommended approach here?
It feels like the nx release
feature currently doesn't support a pretty standard publishing workflow: determining the next version of your package(s), compiling the code and publishing the compiled output with the corresponding package.json
. Besides that the package.json
in the package source code including the CHANGELOG.md
should be updated.
Currently it's possible to set the packageRoot
option, however that will point both publishing- and git commit logic to the dist
folder. It feels like we're missing an option where we can:
changelog
and version
creation (including creating a commit)publish
I would actually love to contribute here, it would be great if we can either support this workflow from the logic or write documentation on how to achieve this. Up until now i've had a pretty rough time discovering all the configuration options and the resulting output, so i feel like there's room for improvement.
I have the same problem and wonder how many cases we can publish the source code directly.
@JamesHenry, could you help us out here a little bit? Thanks!
@fahslaj could you perhaps provide some insight in the recommended approach here?
It feels like the
nx release
feature currently doesn't support a pretty standard publishing workflow: determining the next version of your package(s), compiling the code and publishing the compiled output with the correspondingpackage.json
. Besides that thepackage.json
in the package source code including theCHANGELOG.md
should be updated.Currently it's possible to set the
packageRoot
option, however that will point both publishing- and git commit logic to thedist
folder. It feels like we're missing an option where we can:
- Specify the package source path for
changelog
andversion
creation (including creating a commit)- Specify the package output path exclusively for
publish
I would actually love to contribute here, it would be great if we can either support this workflow from the logic or write documentation on how to achieve this. Up until now i've had a pretty rough time discovering all the configuration options and the resulting output, so i feel like there's room for improvement.
Thank you for the clear explanation. I agree that this flow should be supported better. For now, as a workaround, you can create a release script that uses the programmatic api. It would call releaseVersion(...)
, then perform your build command, then releasePublish(...)
.
However, I do think this should be supported entirely from the CLI. I'll bring this up with the team and follow up when I have updates.
@fahslaj thanks for getting back to us!
That's indeed what i ended up doing, however it did take some extra effort to only build the specific packages that are actually about to be released.
I've created a gist with the current publish.ts
script i've created: https://gist.github.com/lorenzodejong/acde50902b55ff2b72fffeeef2c35f9d.
For anyone interested on how to actually utilise it: you can use @swc/node
to execute this script programatically:
$ node -r @swc-node/register tools/scripts/publish.ts
I've also included a specific flow in which a prerelease only creates a Git tag for version tracking purposes, whereas the latest
release also includes a changelog and version bump in the package.json
. This allows for the following CI/CD workflow:
1.0.0-PR123.0
, 1.0.0-PR123.1
, etc)1.0.0
) gets released, creating a commit with the version bump and changelog and pushing itFor my team this is the ideal workflow, this allows you to test prerelease changes easily without being conflicted with version release commits on the PR branch.
One thing i've noticed however with this workflow is that an incorrect version gets released when creating the prerelease using the conventional-commits
specifier source. Lets say the previous version is 0.1.0
and i create the following commit: feat!: this should become 1.0.0
it actually bumps it to 0.1.1-PR123.0
instead of 1.0.0-PR123.0
.
@fahslaj i'm not sure if this bug has been reported already? Otherwise i'd be happy to create another more specific report for this with reproduction steps.
@lorenzodejong I am glad you were able to make it work with that script! As for the potential bug, if you could create a specific report with repro steps that would be greatly appreciated!
Hi @fahslaj, thanks for the prompt answer but I think that this should become top priority.
At the moment, nx release feels like a smartphone than can do everything except making phone calls... you know what I mean π
@rainerhahnekamp letβs try and keep the discourse productive. typescript-eslint and angular-eslint are significant repos that are publishing their source files just fine with nx release, as are many others
It's actually awesome work you guys have put into this new feature, i can already see it's a very useful tool for our team.
Even though our workflow isn't fully supported yet from the default configuration, i do feel like it's close to being supported. That was also the reason for me to open this issue. If there's anything i can contribute to i'd be more than happy to take a shot at it.
I've opened an issue for the specific issue regarding the prerelease version resolver: https://github.com/nrwl/nx/issues/22150
i tried adding
dependsOn
to mynx.json
, but that also didn't work."nx-release-publish": { "dependsOn": ["build"], "options": { "packageRoot": "dist/libs/{projectName}" } },
I tried this as well and it actually solves the issue of an outdated dist folder. In my pipeline I run nx release version
which updated the package.json and changelogs of my independent packages. I then go ahead and push the commit + tags to my remote. With the next step I run nx release publish
which first builds all the affected packages and then publishes them. Here my nx.json:
{
"targetDefaults": {
"nx-release-publish": {
"dependsOn": ["build"],
"options": {
"packageRoot": "{workspaceRoot}/dist/libs/{projectName}"
}
}
},
"release": {
"projectsRelationship": "independent",
"projects": ["libs/*", "!libs/workspace-extensions"],
"releaseTagPattern": "{projectName}-{version}",
"git": {
"commitMessage": "chore({projectName}): release version {version} [skip-ci]"
},
"version": {
"conventionalCommits": true
},
"changelog": {
"projectChangelogs": {
"createRelease": "github"
}
}
},
}
@raketeFlo does the nx release publish
command actually run the build or do you have to do that manually? I noticed you still have the dependsOn in place, but you also mentioned it didn't work. Just clarifying.
@ryan-mcginty-alation nx release publish
runs the build automatically because of the dependsOn. In my GitHub workflow I only run nx release version
then I push the changes and tags and then I run nx release publish
which builds my libs with the updated version and then publishes them.
The only thing I don't like about this approach is that the build could fail and then I already versioned and pushed the changes...
@rainerhahnekamp letβs try and keep the discourse productive. typescript-eslint and angular-eslint are significant repos that are publishing their source files just fine with nx release, as are many others
@JamesHenry thanks for this comment, I've started looking at typescript-eslint as a reference in trying to implement nx release
in my project. I notice that it only uses the programmatic API to achieve its release workflow, rather than the CLI.
Interestingly, it also uses tsc --build
with project references rather than @nx/js:tsc
with a root tsconfig with path mappings, and compiles dist folder side-by-side with source, rather than under a root dist/packages/{package-name}
. These changes all seem to sidestep one of the issues discussed here, where the source-code package.json gets updated with release, but the dist package.json doesn't. However, these are all non-standard approaches if one has just scaffolded an nx repository by using the CLI defaults from the various generators.
Are these tweaks all necessary to get an nx release
experience that requires minimal custom code? I.e. is it primarily suited for package-based monorepos rather than integrated monorepos that follow nx's more opinionated structure?
I've been trying to solve this problem for the last couple of days (for a react vite library), and finally settled on the following:
src
.eslintrc.json
project.json
tsconfig.json
tsconfig.lib.json
vite.config.ts
{
"name": "@spwntch/react-vite-sandbox",
"version": "0.0.11",
"main": "./dist/index.js",
"module": "./dist/index.mjs",
"types": "./dist/index.d.ts",
"keywords": [
"react",
"vite",
"sandbox"
],
"author": "spwntch",
"license": "MIT"
}
On publishing. this resulted in the following package, which I can consume:
π¦ @spwntch/react-vite-sandbox@0.0.11
=== Tarball Contents ===
194B README.md
42B dist/index.d.ts
13.5kB dist/index.js
20.8kB dist/index.mjs
189B dist/lib/react-vite-sandbox.d.ts
263B package.json
=== Tarball Details ===
name: @spwntch/react-vite-sandbox
version: 0.0.11
filename: spwntch-react-vite-sandbox-0.0.11.tgz
package size: 9.9 kB
unpacked size: 34.9 kB
shasum: 532dbf531be7b2e3c95f80e7b1b88c329bc992c8
integrity: sha512-xKdurjlp8+ej9[...]+7vLh8+gfKcLg==
total files: 6
Published to https://registry.npmjs.org/ with tag "latest"
Given the intent of project crystal, is this not a better approch than trying to mess with the nx release process?
"targetDefaults": {
"nx-release-publish": {
"dependsOn": [
"build"
],
"options": {
"packageRoot": "dist/{projectRoot}"
}
}
}
This repaired it for me. I want to say its just documentation issue rather than a actual bug
@fahslaj i see this is closed as completed, is this solution the recommended/correct approach? https://github.com/nrwl/nx/issues/21855#issuecomment-2041030949
@fahslaj i see this is closed as completed, is this solution the recommended/correct approach? #21855 (comment)
https://github.com/nrwl/nx/issues/21855#issuecomment-2041030949
My comment above seems to me the solution. It's missing documentation maybe worth a PR to fix
This eventually also fixed the issue for me, however the only minor difference in my solution is that i targeted the packageRoot
option to the source folder, e.g. {workspaceRoot}/packages/{projectName}
.
The reason for this being that if you want to commit the updated CHANGELOG.md
and version in the package.json
using the automated commit feature from nx release
, you want this to happen in the source folder of your code instead of the build output folder.
I still don't have a fix for this. Is my only option to spend time figuring out my own custom release script like @lorenzodejong has done? Not sure why this is closed.
The recommended fix of doing dependsOn: ["build"]
doesn't even work as designed based on this comment
Reopening this, as I do think more documentation and clarification is needed. Depending on your workspace structure, @Jordan-Hall or @lorenzodejong 's solutions may work for you. There definitely needs to be more documentation on how Nx release handles the packageRoot
and how to configure it according to your workspace setup.
In the meantime, this comment from James on another thread might shed some light on the matter: https://github.com/nrwl/nx/issues/20978#issuecomment-2122372518
Update: Additional documentation on the supported cases can be found on the docs site here.
@fahslaj I' still wondering does the problem solved by NX or we should rely on workarounds? I've tried to use built-in functionality of NX for build and publishing libraries in mono repo for creating change log and bumping version automatically, but as others mentioned with referencing "packageRoot" to corresponding dist folder, version of published package does not bump.
@HosseinSalmanian If you need the version of the package within source control to be updated, but also need to publish a different packageRoot
than the source, then you will need to use the nx release programmatic API along with your own scripting logic. This case is not currently supported by nx release CLI.
"targetDefaults": { "nx-release-publish": { "dependsOn": [ "build" ], "options": { "packageRoot": "dist/{projectRoot}" } } }
This repaired it for me. I want to say its just documentation issue rather than a actual bug
Normally, during CI we have a step for build
, prior to release
. In those cases, "dependsOn": ["build"]
might not be required.
"targetDefaults": { "nx-release-publish": { "dependsOn": [ "build" ], "options": { "packageRoot": "dist/{projectRoot}" } } }
This repaired it for me. I want to say its just documentation issue rather than a actual bug
Normally, during CI we have a step for
build
, prior torelease
. In those cases,"dependsOn": ["build"]
might not be required.
While true this enforces it to run. Nx cache will just use the cache build
We are using Nx with an Angular monorepo for the first time, but otherwise extremely experienced with Angular projects. Our new Nx monorep project has a few shared libraries, plus 2 apps and we are now trying to release/publish the apps into our own repository, but noticed that the publish was adding the 'src' files, rather than the 'dist' files.
So, as above, we followed the Nx docs to point at the workspace root 'dist' folder for each app build, so this files can be included in the tarball being published. However, as soon as we add the "nx-release-publish" > "options > "packageRoot": "dist/apps/{projectName}"
into the nx.json, there is a build error running npx nx release publish --projects=app1 --dry-run --verbose
.
i.e.
Error: ENOENT: no such file or directory, open '/projects/my-app/dist/apps/app1/package.json' at readFileSync (node:fs:448:20) at readJsonFile (/projects/my-app/node_modules/nx/src/utils/fileutils.js:17:43)
However, if the 'nx-release-publish' task is removed from the nx.json, then it builds and only includes the package.json π i.e.
nx run app1:nx-release-publish
π¦ @myOrg/app1@X.X.X-dry-run
=== Tarball Contents ===
310B package.json
=== Tarball Details ===
name: @myOrg/app1
version: X.X.X-dry-run
filename: app1-X.X.X-dry-run.tgz
package size: 266 B
unpacked size: 310 B
shasum: 85e557bb963dc1715a07f6b90d7451cdefa85522
integrity: sha512-FP5eUbS8qXQ1K[...]lQbQS3EoU4RMQ==
total files: 1
Would publish to https://my-org.repo.com/repository/npm-applications/ with tag "latest", but [dry-run] was set
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
NX Successfully ran target nx-release-publish for project app1
Our project is using the following Nx version, along with Angular 18:
Node : 22.3.0 OS : darwin-arm64 npm : 10.8.2
nx : 18.0.4 @nx/js : 18.0.4 @nx/linter : 18.0.4 @nx/eslint : 18.0.4 @nx/workspace : 18.0.4 @nx/devkit : 18.0.4 @nx/eslint-plugin : 18.0.4 @nrwl/tao : 18.0.4 typescript : 5.3.3
@thdk, I have cloned your repo, I get the same error as I am seeing locally when I run npx nx release publish --dry-run --verbose
, where the package.json cannot be found.
i.e.
Error: ENOENT: no such file or directory, open '/projects/nx-release-ts-packages/dist/packages/my-lib/package.json' at readFileSync (node:fs:448:20) at readJsonFile (/projects/nx-release-ts-packages/node_modules/nx/src/utils/fileutils.js:17:43)
Any help would be greatly appreciated.
To my opinion it should work, but probably this code is the reason:
So basically there is a plugin which overwrites target defaults with dependsOn: ['^nx-release-publish']. Because of this it doesn't make sense what you put into target defaults - it's just ignored...
I commented this line in my local node_modules and everything worked well, release now works based on configured graph, not overwritten one: https://github.com/nrwl/nx/blame/c4c8b0150d827657632596656d92040108a338aa/packages/nx/src/utils/package-json.ts#L191
Current Behavior
Currently setting up a default (integrated) monorepo preset yields incorrect behavior when using
nx release
. By default, packages generated from@nx/js
with thepublishable
option will generate aproject.json
which outputs changes to the /dist folder on the root of the monorepo. However thepackage.json
in the src folder points to./src/package.json
.Running the
nx release
command seems to be directly publishing the package from the source directory, causing the actual TypeScript source files to be published instead of the resolved build output in thedist
folder.It seems like the current implementation of
nx release
is only intended to be used in package-based repositories, where you specify your package.json to point to adist
folder present in the actual source folder.The documentation also seems to be incorrect on this regard. If you look at the example of the published tarball you'll see that actual
.ts
files are getting published:Expected Behavior
I expect
nx release
to be directly compatible with projects generated from the default provided Nx generators, without having to change the behavior of thepackage.json
andproject.json
.GitHub Repo
No response
Steps to Reproduce
npx create-nx-workspace release-example-workspace
, select to create anIntegrated Monorepo
when asked for it.npx nx g @nx/js:lib mylib
npx nx release --first-release --dry-run
npx nx release publish --dry-run
Observed behavior:
.ts
files are present in the tarball.Nx Report
Failure Logs
No response
Package Manager Version
No response
Operating System
Additional Information
No response