[rush] Support rush publish alternatives to NPM publish

[elliot-nelson]

Summary

The rush publish command today can be used to:

1. Apply the recently bumped versions of packages to the package.json file
2. Update changelog jsons
3. Npm pack/publish the new versions of packages to NPM (or private registry)
4. Check in the changed git files, deleted change log files, etc., and push them up to the remote.
5. Tag the commit and push the tags.

I'd like to be able to take advantage of most of this wrapping (steps 1, 2, 4, 5), but to replace step 3 with something different.

Details

It's possible to have packages in your monorepo that are not NPM packages, but something else.

For example, if it's a Docusaurus website, there's already a command (rushx deploy) that can deploy the project. You could rename each Docusaurus command to rushx deploy-websites, and make it a Rush bulk action, and deploy all your websites with one command... But this doesn't give you any of the automatic version checking, bumping, and git tagging that rush publish does.

If you could run a rush publish, that worked like publishing but replaced "publish to NPM" with "run my bulk action", that would be pretty nice.

Perhaps alternately, the guts of publishing to NPM could be turned into a plugin, and then more complex plugins could be created as replacements (with BulkActionPublisher being just one simple example of a plugin, that hands off the details to a bulk action defined on the individual projects).

It's also possible that the rush publish command is doing too much and should instead be broken up into separate commands with simpler APIs... but selecting the projects impacted with version changes is a big part of its usefulness, and I wouldn't want to see that have to happen in a series of different commands.

Standard questions

Please answer these questions to help us investigate your issue more quickly:

Question	Answer
@microsoft/rush globally installed version?	4.56.0
rushVersion from rush.json?	4.56.0
useWorkspaces from rush.json?	Yes
Operating system?	Mac
Would you consider contributing a PR?	Maybe!
Node.js version (node -v)?	14.18.1

[dmichon-msft]

My thoughts on this: 1) We want a command that does only the changelog related stuff: rush version --bump (this being a separate task is essential to large-scale monorepo to avoid merge conflicts when writing the versions 2) After committing the above changes, the affected projects can be selected via --only git:HEAD~1 3) Packing can be handled most cleanly as phased command that builds, then packs; this may be a custom plugin to more easily facilitate defaulting to "pnpm pack" as the pack phase script. Generating the mapping of git tag to commit (as a JSON file) can be hooked into the plugin at this phase. 4) Writing Git tags is best handled as an altogether separate command, this is because Azure DevOps Git (and others) support REST APIs that allow setting git tags in bulk with a single API call, and gracefully handle the case where the target tag already exists. 5) Publishing the packed tarballs to the npm feed is again best handled as a separate step (we do this at MS) in order to support retry of transient errors and similar.

[elliot-nelson]

So maybe today's publishing steps would look something like this:

# Consume change files, update change logs and versions, and commit/push tags(?)
rush version --bump --version-policy NONE --target-branch origin/main

# Build and pack
rush build --to git:HEAD~1
rush pack --only git:HEAD~1

# Git tags... today handled by "rush publish" I think, but perhaps could be a separate command?
noop

# Publish
rush publish --packfiles --only git:HEAD~1

The HEAD~0 / HEAD~1 trick makes sense but I think I'd feel a little more comfortable if I could point right at a list of files -- maybe a JSON file that is an array of project names?

Something like this:

rush version --bump --version-policy NONE --target-branch origin/main --project-list common/temp/projects.json

rush pack --only json:common/temp/projects.json
rush publish --publish --only json:common/temp/projects.json

Maybe even better than that would be if rush version could produce something like a "publish.json", which contains a list of projects to be published, the new version of each, and the tag name to be applied. Then each following command could point at this file to do its work.

# for example...
rush pack --only publish:common/temp/publish.json
rush publish --only publish:common/temp/publish.json
rush push-tags --publish-file common/temp/publish.json
# etc.

Lastly, the only thing odd about splitting up pushing tags and pushing tarballs to NPM is that I thought these were considered "atomic" (as much as possible)... i.e., let's say I ran rush publish, it is pushing each tag once the publish step passes, and if there are some projects that weren't published (for example, it determined v2.0.1 was already in Artifactory for some reason), then it wouldn't publish or push a tag. (I could be incorrect about that.)

[kkazala]

I had exactly the same requirements and although I hope it will become part of the product soon, I didn't want to wait and wrote my own extension. In case someone is interested: https://dev.to/kkazala/custom-rush-publishing-flow-15ld