Versioning only for applications based on library changes

[chasevida]

A few colleagues were looking for a solution to bump our application version based not only changes in the application code but also any libraries that it is dependent on. Is that possible with this module or will it only ever bump the version of the individual library or app when changed?

This sounds like a strange request. What we're looking to achieve is a way to kinda semantically version our overall application but most of the functionality of these is actually maintained in seperate libraries.

Thanks for this lib, was easy to get up and running and works like a charm.

[edbzn]

@chasevida The sync mode is made for this scenario where you have multiple projects combined in one single product. It would tie all the projects versions together so you can work only on some part of the product and it will bump all other parts.

[lasota-piotr]

It would be great to make the plugin work similar to Lerna. Apps have their package.json files only for storing an independent version. The main difference which I saw is in "Independent mode". lerna version bumps version of a changed package (for example minor change 1.1.0 -> 1.2.0) and all packages (apps) that import the package with a fix (for example 1.1.0 -> 1.1.1). Example in Lerna: The dependency graph is: app -> lib (app imports lib)

When we make a change in the lib Changelog of the lib would be like: Changelog of the app which imports the lib: .

The problem with "sync mode" in @jscutlery/semver is that, when only one lib is updated, all other libs also must be published to npm even they aren't updated.

Example: dependency graph is: app -> lib1 -> lib2 (app imports lib1, lib1 imports lib2)

In sync mode when we update lib1 and bump version, lib2 also has to be built, and published to npm, because package.json of lib1 imports the new version of lib2, even lib2 hasn't changed, and don't have to be updated.

[edbzn]

Ok I get it, it's a totally valid use case, we should discuss it to shape it correctly.

• Should it be automatically handled with independent mode? If yes it's a breaking change.
• Should it be a per project flag?
• Should it be a global builder option?

This feature reminds me kind of --with-deps Nx option to build packages depending from each others.

@lasota-piotr You told that Lerna automatically handle this in independent mode?

[lasota-piotr]

Yes, Lerna handles this. Example repo: https://github.com/lasota-piotr/lerna-repo

The problem with Lerna is that I assume, it uses depencencies, devDependencies, and peerDependencies of package.json to figure out what packages are dependent on each other. I assume, Nx figures out dependencies by searching for import statements in code, so it's hard to connect these two libraries to handle versioning.

[cwmrowe]

First of all really love where this project is heading. However I having trouble understanding how to use it in a CI environment, specifically when it comes to deploying apps, and it seems related to this issue.

Let's say I have the following application.

apps
-- bike-store
-- scooter-store
libs
-- bike-store
---- bike-detail
-- shared
---- ui

I don't really want to use syncMode as I want the bike-store and scooter-store apps to have separate version numbers. However if syncMode can do grouping #98 then maybe grouping the bike-store libs with the bike-store app would work.

However, even with grouping if I make a change to the shared ui lib, neither app's version number is going to get bumped.

I think what I would like, is a solution that whenever an app is affected it version gets bumped, even if there are not commits to it. Any help, would be greatly appreciated 😄

[edbzn]

Thanks for your feedback folks! I like the Lerna approach with using depencencies, devDependencies, and peerDependencies to figure out which package depend on each other. IMO this in combination with independent versioning bring exactly what you need.

At the moment sync groups are not well defined as we're discussing about it, and as @lasota-piotr mentioned with sync mode all packages get updated even if they are not directly affected, groups will not change this behavior.

[chasevida]

Thank you @edbzn good to know sync mode works like this. Our use case is more in line with what @lasota-piotr outlined (far better than I originally did - thank you). We have multiple apps and in our CI we build and containerize only those affected. If we use sync mode we would end up building everything even for changes that are not relevant. This just gets a bit slow and potentially costly with our CI.

As I'm not overly familiar with the internals I'll step back and watch this feature conversation and contribute when appropriate. Cheers.

[smasala]

We've got a similar issue. We'd liked to bump the app version because a library has some changes made to it and need to build and redeploy the application with those changes based on the new version (not to mention record the changelogs). The app itself has Nothing changed since last release. even though the library may have breaking changes.

[stefancplace]

We are having an nx workspace with 20+ apps and 40+ libraries. Would be really happy if versioning only for applications based on library changes was possible. This would massively decrease the build times on new release of e.g. only one specific library. Hope to see this feature soon :)

[lasota-piotr]

I made a proof of concept of versioning apps and libraries in NX monorepo using Lerna. Besides, it creates CHANGELOG.md files for apps and libraries.

I created package.json files for every app and not publishable library for storing their versions. Example of package.json file of app or a not-publishable library:

{
  "name": "next-app",
  "version": "0.1.18",
  "private": true
}

I created lerna.json file:

{
  "packages": ["libs/*", "apps/*"],
  "version": "independent",
  "command": {
    "publish": {
      "conventionalCommits": true
    },
    "version": {
      "message": "chore(release): publish"
    }
  }
}

I created a script that bumps the version of affected libs and apps. This script is needed to pass from NX affected to Lerna information what libs and apps changed. Lerna doesn't know it. Normally Lerna uses package.json dependencies to know how packages are connected. package.json files don't have dependencies so we get that information from NX.

const util = require('util');
const childProcess = require('child_process');
const exec = util.promisify(childProcess.exec);
const workspace = require('../workspace.json');

async function bumpVersionAffected() {
  const args = process.argv.slice(2);
  const argsString = args.join(' ');
  const printAffectedCommand = `npx nx print-affected ${argsString}`;
  console.log(printAffectedCommand);

  const printAffectedResult = (await exec(printAffectedCommand)).stdout.trim();
  const affectedProjects = JSON.parse(printAffectedResult).projects;

  console.log({ affectedProjects });
  const packageJsonNames = affectedProjects
    .filter((projectName) => !projectName.endsWith('-e2e'))
    .map((projectName) => {
      const packageJsonPath = `../${workspace.projects[projectName].root}/package.json`;
      const packageJson = require(packageJsonPath);

      return packageJson.name;
    });

  if (affectedProjects.length <= 0) {
    console.log('No changed packages found');
    return;
  }
  const packageJsonNamesString = packageJsonNames.join(',');
  const bumpVersionCommand = `npx lerna version --yes --force-publish=${packageJsonNamesString}`;

  console.log(bumpVersionCommand);

  // execSync with stdio: 'inherit' to show colored output in console
  childProcess.execSync(bumpVersionCommand, { stdio: 'inherit' });
}

bumpVersionAffected().catch((e) => {
  console.error(e);
  process.exit(1);
});

[slaven3kopic]

We have the same issue. Any news about it?

[edbzn]

Hi there, we already planned our next milestone and we cannot afford this for the moment, furthermore, we use semver for publishable libraries and not for applications, that's why it's not something we need in the short term.

However, I heard that some of you need it, so if the community (you) wants to take it on, we can open a discussion to shape the feature together before going into the implementation. We will guide you but we will not take it entirely, at least not in the near future.

PR & discussions open.

[MartaGalve]

Hi there, we already planned our next milestone and we cannot afford this for the moment, furthermore, we use semver for publishable libraries and not for applications, that's why it's not something we need in the short term.

However, I heard that some of you need it, so if the community (you) wants to take it on, we can open a discussion to shape the feature together before going into the implementation. We will guide you but we will not take it entirely, at least not in the near future.

PR & discussions open.

I think this would definitely be useful, not only for apps but for libraries too. You can have libs depending on other libs, so if lib1 imports lib2, when lib2 changes you can bump lib1 too. That must be a common enough use case.

[chasevida]

Thank you @edbzn appreciate you outlining that and thanks again for your work on this project.

[CaffeinatedCodeMonkey]

@edbzn I, too, find I need this feature. I've spent a bit of time and it looks like I have a solid solution in place on my own branch, and would like to make it available. What steps should I take to make sure that it meets your all's standards and works the way you'd expect, so that it might get merged into the project?

[edbzn]

@CaffeinatedCodeMonkey great, I'm excited to see your work! You should open a pull request and we will continue the discussion there. Note that I'm in vacations so I will not be very reactive, I also would like to discuss and review with @yjaaidi because I think it could be a breaking change so we should establish a release plan. Thanks for your contribution guys.

[CaffeinatedCodeMonkey]

@edbzn @yjaaidi Thanks! I've made the following PR: https://github.com/jscutlery/semver/pull/278.

[yjaaidi]

Hey everyone and special thanks to @CaffeinatedCodeMonkey for the PR. I added a large comment there https://github.com/jscutlery/semver/pull/278#issuecomment-938752939 but let's continue the discussion here.

[CaffeinatedCodeMonkey]

@yjaaidi No problem! Happy to contribute!

I've addressed the first two points you requested:

1. The option has been changed to --track-deps, which I completely agree is a better name.
2. Changes in a dependency can only ever trigger a patch bump.

As for the Changelog, are the changes you're describing something you want in my PR, or is it just discussion for the enhancement?

[edbzn]

@CaffeinatedCodeMonkey We will merge your changes without the changelog, a refactoring work is needed for that and your changes are very useful.

[yjaaidi]

Thx @CaffeinatedCodeMonkey for the quick fixes. Good job!

Issue

Since my last comment, I was thinking about this feature and the following issue: Given the following deps tree A -> B -> C, what if A has --track-deps on while B has it off? Changing C would bump A & C but not B. That would be very surprising, isn't it?

Solution

This is why, I thought about the following quick fix:

Instead of deep checking the dependencies, let's just check the direct dependencies.

The command should then be used like this: nx run-many --all --target version --with-deps (affected should work too) (--with-deps is important here as it will make sure that projects are versioned in the right order)

What will happen here is that Nx will run version on the projects in the following order: C, B then A.

1. this will bump C
2. trying to bump B will detect that its direct dep C has changed, so it will bump B and update B's local changelog and commit the changes.
3. trying to bump C will detect that its direct dep B has changed, so it will bump C and update its changelog & commit.

If B didn't enable --track-deps then this will happen:

1. this will bump C
2. trying to bump B will be a no-op
3. trying to bump C will be a no-op

Consequences

This should have the following consequences:

• code simplification
• better control over the change impact (if B doesn't have --track-deps on, changing C won't affect A)
• simplify changelog implementation

Tasks

• [ ] check direct deps only
• [ ] document that the option should be used with nx's --with-deps

Concerning the changelog, let's open another issue. In fact, the thing is that chores like deps update don't update the changelog by default so the issue will probably be related to how to generate changelogs with chores included.

@CaffeinatedCodeMonkey, @edbzn, everyone, what do you think?

[CaffeinatedCodeMonkey]

@yjaaidi Thanks for this idea! I had not considered this approach, but it brings up some interesting questions, and it really depends on the implementation in NX:

1. Does the --with-deps flag scope the command to only those projects that contain the specified target? (I'm not sure how it could not.)
2. If so, does that mean that each project in the dependency graph would need to have the version target in order for it to properly take into account its dependencies.

Scenario

App A has the following dependency graph: A -> B -> C -> D. In this scenario, only A, B, and D are set up for versioning. So:

1. When when we call the command, will D get versioned first? Or will it not catch it since it's not a direct descendant of B?
2. Since we would only check one dependency deep, will B execute first, and pick up changes in C, but not realize that it needs to include D, so if the change only occurs in D, will the change not get picked up, even though it ends up part of a build compilation? If so, this could make us get new code with an unchanged version.
3. (Continuation of 2) Even if D does get version ran on it by use of --with-deps, will its execution contribute to the resulting version of A?

If this is the case, would using --track-deps require that all libraries be versionable?

[yjaaidi]

Hi @CaffeinatedCodeMonkey! These are very relevant questions!

1. Does the --with-deps flag scope the command to only those projects that contain the specified target? (I'm not sure how it could not.)

Yes

2. If so, does that mean that each project in the dependency graph would need to have the version target in order for it to properly take into account its dependencies.

Yes

1. When when we call the command, will D get versioned first? Or will it not catch it since it's not a direct descendant of B?

D will get versioned first.

2. Since we would only check one dependency deep, will B execute first, and pick up changes in C, but not realize that it needs to include D, so if the change only occurs in D, will the change not get picked up, even though it ends up part of a build compilation? If so, this could make us get new code with an unchanged version.
3. (Continuation of 2) Even if D does get version ran on it by use of --with-deps, will its execution contribute to the resulting version of A?

In fact, D won't affect B and A. D will get versioned then version will run on B and while checking if B or C have changes, this won't bump anything.

If this is the case, would using --track-deps require that all libraries be versionable?

Yes

Otherwise, one could simply use --affected with version and --releaseAs=patch option. This will bump as patch all affected projects but I don't think that it's the proper solution.

IMO, there are 3 main use cases semver:

1. open-source library with independent publishable packages: in this case, we don't need --track-deps.
2. multiple apps depending on shared libraries: in this case, every library should be versionable. In fact, if libraries are not versionable the changelog will be nosy. e.g. apps A and B depend on library C, a change on C will pollute both apps A and B changelogs especially if the commits are something like docs(c): update C docs. On the other hand, if C is versioned then A and B changelog might contain something like chore: update dependency c from 1.0.1 to 1.0.2
3. groups of libraries or multiple apps depending on groups of libraries: e.g. when splitting an app into feature libs, it could be annoying to have to version every feature lib (even though I personally think it's a good idea) so one might want to group them. In this case, if we implement grouping, one can group libs and sync their versions.

TL;DR

I am in favor of versioning everything. If versioning everything is challenging, then we can improve schematics and implement lint rules to make sure that everything is versioned. The main reason for this approach is that I wouldn't want the same change to affect multiple bumps and changelogs as it would lead to lots of duplication and confusion.

What do you think?

[CaffeinatedCodeMonkey]

@yjaaidi That makes a lot of sense. I will get started on it, and also add documentation to properly explain the dependency on versioning being enabled on all libraries in the dependency graph, if they're to be tracked.

Thanks!

[yjaaidi]

Awesome! Thank you very much for your time and contributions @CaffeinatedCodeMonkey!

[CaffeinatedCodeMonkey]

@yjaaidi You're very welcome, and thanks for the guidance! I've finished up the changes.

Note: I didn't make a test specifically for this kind of cascade of versioning, because it's not something coded for, but rather a result of a sequence of workspace executions. But, I did test it in one of my own repos to make sure that it behaved as expected.

The one pitfall I found was if the user (me) used a hardcoded tag prefix, then there was no fixed point in time that the commits for dependencies could be tested against. As an example, if I used the tag prefix v, then it would result in the following:

For a change in C in a monorepo with the dependency graph of A -> B -> C, then a change in C would result in a tag v0.0.2. When B is versioned, it would run against the tag from the version execution on C: v0.0.2, from which point there were no new commits. This stopped the version cascade.

I do not feel that this is an error, but rather the logical result of certain configurations. Thoughts?

[CaffeinatedCodeMonkey]

@edbzn I noticed when migrating one of my projects to NX 13, that an error is thrown when using --trackDeps, due to the createProjectGraphAsync function being called with version 3.0, which is no longer supported. I also realized that there was not a test to catch this, since the project graph had to be mocked. So I made a PR with a version bump, and an e2e test that can catch if that function call is made with an unsupported version. https://github.com/jscutlery/semver/pull/369/files

[yjaaidi]

The one pitfall I found was if the user (me) used a hardcoded tag prefix, then there was no fixed point in time that the commits for dependencies could be tested against. As an example, if I used the tag prefix v, then it would result in the following:

For a change in C in a monorepo with the dependency graph of A -> B -> C, then a change in C would result in a tag v0.0.2. When B is versioned, it would run against the tag from the version execution on C: v0.0.2, from which point there were no new commits. This stopped the version cascade.

I do not feel that this is an error, but rather the logical result of certain configurations. Thoughts?

@CaffeinatedCodeMonkey, in fact, that's the intended result given that all projects share the same tag and that semver is based on tags.

[Michsior14]

Is there any possibility to have the lerna fixed mode behavior using this package? I mean that non-major changes bumps only the package itself and it's dependants?