search

npm / rfcs

Public change requests/proposals & ideation
Other
727 stars 238 forks source link

[RRFC] `npm publish --if-needed` #466

Open ljharb opened 2 years ago

ljharb commented 2 years ago

Motivation ("The Why")

My company uses CD to publish our internal npm packages. However, the publish job, if ran twice, will error the second time, because the package is already published.

If we make the job never fail, then we might be hiding other issues.

Currently, our planned workaround is similar to:

  1. npm pack
  2. npm show @scope/package dist.shasum
  3. if there is a version, confirm the tarball's shasum matches the published one and succeed, else fail
  4. else, publish (and fail if the publish fails)

Desired Behaviour

It would be great if we could do npm publish --repeatable (however it's spelled) that would not fail if i'm attempting to publish the same package contents over a matching version.

Thoughts?

isaacs commented 2 years ago

Maybe we should just compare the integrity values of the built artifact and the one on the registry, and if they're the same, always say it worked? If they differ, sure, failure. But I can't imagine why you would not want npm publish to be idempotent and repeatable for an identical artifact.

We could also potentially implement this on the registry, so if we get an artifact for a version we already have, and it's identical integrity, go "yep, we got it, thanks".

ljharb commented 2 years ago

Doing what I'm asking by default would be even better but i assumed it'd be simpler to start by asking for an option :-)

Doing it on the registry is fine too, but wouldn't be as repeatable for private registries, such as the one my company uses, so it might be ideal to do it in both places.

azu commented 2 years ago

I created can-npm-publish for working similar workflow to this RFC. It is hard to maintance this tool because npm v6 and v7 output different format.

📝 I've used npm command instead of Public Registry API for supporting private registry like GitHub Packages.

I think it is very good that npm publish --if-needed will be supported.

ljharb commented 2 years ago

During the RFC call, it seems there was strong support for making the CLI do this by default, and anyone who wishes to contribute is invited to make a PR.

everett1992 commented 2 years ago

This issue describes packing, then comparing the packed tarball's checksum against the version in the registry. The tarball checksum includes file metadata, so even changing files' timestamps will create a different checksum. If the package includes lifecycle scripts that generate code on pack it's almost guaranteed to have a different checksum.

Then this proposal would publish the package with a different checksum and the registry will either error with EPUBLISHCONFLICT or replace the package version that's already published.

Instead I recommend calling publish and exiting 0 if the error is EPUBLISHCONFLICT. The downside here is your package can change significantly (in every way except pacakge.json's name and version fields) without being published or publish throwing an error. You have to be aware that you must update the package version for npm publish --if-needed to have an effect.

Alternatively npm could introduce more deterministic checksums that only change when file content changes. Then as long as the package has deterministic builds the described workflow could work.

isaacs commented 2 years ago

Alternatively npm could introduce more deterministic checksums that only change when file content changes. Then as long as the package has deterministic builds the described workflow could work.

That is currently the case. If the file contents are identical, you should always get an identical tarball, because we strip (or exclude) any platform, time, or OS specific data from the artifact, and explicitly specify all of the compression parameters to known values.

luwes commented 3 months ago

This would be really useful, especially with workspaces.

I'm in need for this since using release-please instead of lerna for releasing. Current workaround is this local script https://github.com/muxinc/media-elements/blob/main/scripts/publish-packages/publish-packages.js

pnpm does this by default, https://pnpm.io/cli/publish#--force