Migrate monorepo from Lerna to Nx

[michaelbromley]

This monorepo is built on Lerna, a tool for managing monorepos which handles linking sibling packages and also makes the versioning & publishing of multiple packages much easier. Lerna has served us very well for the past ~4 years, but unfortunately has been unmaintained for a while and recently officially announced the fact.

Therefore we should move to another tool to handle these concerns. Recently I've been using Nx in some other projects and it is very, very good. It is not primarily aimed at the exact same use-case that we've been using Lerna for - specifically it does not handle all the publishing stuff automatically. But the monorepo management aspect is much more sophisticated. They even have a guide for migrating from Lerna to Nx.

This repo has, over the years, accumulated quite a lot of custom scripts and checks to ensure a smooth release cycle. This will all take some verification and no doubt quite a bit of manual modification to work in Nx.

So the plan is to start with that migration guide, and then go though each of the custom scripts (doc gen, code gen, release checks, version bumps etc) and get each working.

This will be done on the master branch and then merge up into minor and major.

[michaelbromley]

In the mean time, Nrwl took over maintenance of Lerna and started releasing new versions!

https://twitter.com/victorsavkin/status/1529493314621145090

So for the short-term I'll probably stick with Lerna and keep Nx migration open as a future project when there's a bit more time for non-critical tasks.

[cwerner1]

Hi @michaelbromley

I added a basic configuration for the project. So the nx features like the dependency graph is available.

I`ve also tested the new lerna version 5 where the nx integrationcan be used and takes advantage of the nx dependency management.

https://github.com/vendure-ecommerce/vendure/pull/1850

[michaelbromley]

Hey @cwerner1, thanks! What are your thoughts on moving to Lerna v6 vs Nx?

The thing I get the most value from in Lerna is the way it automatically manages dependency version bumps on release. I'm not sure whether that's something that is well supported in Nx or whether we'd need a custom plugin for that.

I was under the impression that migrating from Lerna v4 to the new Nx-powered v6 would be a good first step.

[asonnleitner]

What about rushjs or plain pnpm workspaces? @michaelbromley definitely recommend taking a look into those. Would be open to help with the setup and configuration. I would also consider migrating from the long deprecated tslint to eslint.

[michaelbromley]

Hi @asonnleitner, the main benefit I get from Lerna right now is the tooling around publishing multiple packages and keeping the versioning in sync between the monorepo packages. That's something that even Nx does not specialize in, which is why I more recently investigated just upgrading to Lerna v6 (see #1902). Do you have any insight as to whether rushjs or pnpm handle that aspect like Lerna does?

I definitely want to migrate away from tslint, it has just been low priority up till now because it still works fine currently. I would be happy to accept help on this task though! Main thing I have missed in other eslint-powered projects is the rule that splits import groups like we have in tslint. Maybe there is a preset for this somewhere I didn't find yet though.

[asonnleitner]

@michaelbromley If it is really just about publishing packages and keeping the dependencies in sync, then there would be 3 options.

1. use pnpm workspaces, and write your own version bump scripts, here are some examples:

   • nuxt
   • vue

2. use a tool like changesets to manage version bumps and publishing, here are some mono repositories using changesets:

   • pnpm
   • shopify hydrogen

3. and a more enterprise ready solution to use would be rush, which is used by microsoft. Some examples:

   • rush stack

[michaelbromley]

@asonnleitner thanks for those references!

Are you able to give a summary of potential benefits of using pnpm or rush over Lerna/Nx?

[asonnleitner]

@michaelbromley sorry for the late reply, was sick the last few days. But to sum it up.

In the case of using pnpm as package manager, the main benefit is that it is much faster than npm/yarn. This is especially noticeable when working with a large monorepo. The other benefit is that it is more strict about dependency versioning, which can help to avoid dependency conflicts. And is more efficient with disk space, as it uses a shared store for all packages, and only installs the packages that are not already present in the store.

Docs Benchmark

About rush, it is a monorepo management tool, which was specifically designed for large monorepos, and offers full control over every aspect of the monorepo. It is also very fast, and has a lot of features that are not available in other tools. Just to name a few, it has a built-in build cache, which can be used to speed up the build process, and it has a built-in dependency validator, which can be used to detect dependency cycles. It also has a built-in change detection system, which can be used to determine which packages need to be rebuilt when a change is made to a package. It also has a built-in command line tool, which can be used to run commands in parallel across all packages in the monorepo.

Rush offers a lot, but it is also very complex, and requires some configuration.

Docs Demo

[asonnleitner]

@michaelbromley the State of JS survey results were yesterday published, and pnpm is the leading monorepo tool!

https://2022.stateofjs.com/en-US/libraries/monorepo-tools/

[michaelbromley]

Major branch (v2) is now on Lerna v6.