search

dxos / gravity

System testing frameworks.
GNU Affero General Public License v3.0
1 stars 1 forks source link

Versioning requirements #69

Open dmaretskyi opened 3 years ago

dmaretskyi commented 3 years ago

Requirements

  1. Dependency ranges should not stretch across releases with breaking changes. This will ensure that existing releases on NPM are not broken by a future breaking change in their dependency graph.
  2. We should use version ranges to deduplicate compatible packages. The reasoning is, that widely used utility packages such as crypto or async do not frequently receive breaking changes but multiple installations of the same package might be incompatible with each other. Having an efficient and automatic method to deduplicate them is essential for developer experience.
  3. Bumping major version should only happen on big releases (1-2 year intervals) and should not be dictated by changes in code.

Solution

Versioning scheme

In light of those requirements it's important to define a versioning scheme that allows us to differentiate between breaking and non-breaking changes and set those filters in our dependency ranges. Semver is widely used standard that is consistent with requirements 1 and 2, but unfortunately contradicts requirements 3, as all breaking changes in semver must bump MAJOR.

I propose a versioning scheme that is similar to semver with a single modification that MAJOR stays constant, and MINOR is bumped on breaking change. PATCH is used for non-breaking releases.

To specify dependencies we can use tilde ranges (~). ~1.2.3 would match anything from >=1.2.3 up to <1.3.0. This would allow non-breaking updates to be installed automatically. But would prevent new releases with breaking changes from breaking already published releases.

Using ranges in this way also limits our use of preids for alpha and beta releases. Specifying them as postfixes as we do now will interfere with range resolution. So instead alpha releases from main branch will be published under alpha tag on NPM, and beta releases from beta branch will be published under beta tag. Packages published from release branch will go under latest tag.

dmaretskyi commented 3 years ago

Steps to implement this proposal

dmaretskyi commented 3 years ago

Concerns

telackey commented 3 years ago

I made the requested changes to our forks of release-please and release-please-action and published a new version. To use it, switch to v0.0.105 in the workflow and specify bump-minor-on-breaking in the config. For example:

      - uses: dxos/release-please-action@v0.0.105
        id: release
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          release-type: node
          package-name: release-playground
          bump-minor-on-breaking: true
telackey commented 3 years ago

I tested some package versioning in verdaccio.

I created three packages, and published one of them at 1.0.0@latest and 1.0.1@alpha. In the second package, I ran yarn add pkg1@alpha. It pulled 1.0.1. The package.json value was '^1.0.1'.

I published 1.0.2@alpha and tried running yarn upgrade and also recreating yarn.lock and running yarn install. In both cases it updated to 1.0.2.

In the third package I ran simple yarn add pkg1 and got 1.0.0 from latest. I then tried the yarn upgrade and yarn.lock experiments. It remained at version 1.0.0, I did not get 1.0.2. The package.json value was '^1.0.0'.

So it did not draw from the alpha channel in the second case, but did in the first case.

I then tried tagging version 1.0.1 to ‘latest’. After that the second package stopped drawing in 1.0.2@alpha with yarn upgrade and recreating the yarn.lock. Both pkg2 and pkg3 drew 1.0.1 from 'latest'.

So far I have not found this documented, but the search logic would appear to be something like this:

If the version requirement can be satisfied with a package from 'latest', use the highest match there and stop looking; else search over all the other tags and pick the highest match among them.

(The latter case does seem to search among all the tags, as I published a yet higher version tagged as beta, and it chose that over an adequate, but lower, match in alpha.)

The experiment still be be performed is how this works out for dependencies of dependencies (eg, if I this package depends on 1.0.0 from latest, but one of my dependencies uses 1.0.2 from alpha, and we are both using either '~' or '^' ranges, do we end up with both 1.0.0 and 1.0.2 (not good) or de-deduplicate automatically to just 1.0.2 (good))?

telackey commented 3 years ago

It does not appear to deduplicate the results; it lets each stand as it would have done independently. That makes sense, but is inconvenient for our some of our purposes.

In this example pk2 depends on ^1.0.3 of pkg1 (1.0.3 is an alpha release).

The current package depends on '^1.0.0' of pkg1 and '^1.0.0' of pk2.

The result is both 1.0.2 (latest) of pkg1 and 1.0.3 (alpha) of pkg1.

❯ yarn why '@telackey/pkg1'
yarn why v1.22.4
[1/4] 🤔  Why do we have the module "@telackey/pkg1"...?
[2/4] 🚚  Initialising dependency graph...
[3/4] 🔍  Finding dependency...
[4/4] 🚡  Calculating file sizes...
=> Found "@telackey/pkg1@1.0.2"
info Has been hoisted to "@telackey/pkg1"
info This module exists because it's specified in "dependencies".
info Disk size without dependencies: "8KB"
info Disk size with unique dependencies: "8KB"
info Disk size with transitive dependencies: "8KB"
info Number of shared dependencies: 0
=> Found "@telackey/pk2#@telackey/pkg1@1.0.3"
info This module exists because "@telackey#pk2" depends on it.
info Disk size without dependencies: "8KB"
info Disk size with unique dependencies: "8KB"
info Disk size with transitive dependencies: "8KB"
info Number of shared dependencies: 0
✨  Done in 0.05s.
telackey commented 3 years ago

If the version requirement can be satisfied with a package from 'latest', use the highest match there and stop looking; else search over all the other tags and pick the highest match among them.

This can be confirmed:

https://github.com/yarnpkg/yarn/blob/a4708b29ac74df97bac45365cba4f1d62537ceb7/src/resolvers/registries/npm-resolver.js#L51

https://github.com/yarnpkg/yarn/issues/3560#issuecomment-339672654

alexwykoff commented 3 years ago

Proposal from Thomas: Continue publishing main to alpha, but with no pre-id. Make a new release action that will tag the indicated release as latest. Then step back into main and bump the significant version so the next release will be in a disparate range.

dmaretskyi commented 3 years ago

It does not appear to deduplicate the results; it lets each stand as it would have done independently. That makes sense, but > is inconvenient for our some of our purposes.

Does this also happen in reverse situation?

a@1.0.0
+  b@~1.0.2 (alpha)
|    \ - c@~1.0.0 (latest) 
|
\ - c@~1.0.2 (alpha)

Would we get one or two version of c in this case?

dmaretskyi commented 3 years ago

@alexwykoff I'm leaning towards that idea. For now I'll be only integrating publishing from main to alpha with new semantics, and lets discuss beta/release strategy on the call today