search

zephyrproject-rtos / sdk-ng

Zephyr SDK (Toolchains, Development Tools)
Apache License 2.0
169 stars 124 forks source link

Semantic versioning for Zephyr SDK #608

Open stephanosio opened 1 year ago

stephanosio commented 1 year ago

Preface

The versioning scheme used by Zephyr SDK is currently more or less arbitrary in the sense that there are no clearly defined rules dictating how the version number is incremented, which leads to various problems such as:

Adopt semantic versioning scheme that clearly defines what each version field means and when each version field is allowed to be incremented.

Implementation Plan

Given a version number MAJOR.MINOR.PATCH, increment the:

  1. MAJOR version when you make major incompatible change(s) to multiple components that affect the overall user experience.
  2. MINOR version when you make minor incompatible change(s) to one or more components that do not affect the overall user experience.
  3. PATCH version when you make backwards compatible enhancements and bug fixes.

(loosely based on Semantic Version 2.0.0)

Examples of major incompatible changes

(to multiple components that affect the overall user experience)

Examples of minor incompatible changes

(to one or more components that do not affect the overall user experience)

Zephyr Build System Integration

Zephyr build system currently selects the latest version of SDK among the detected SDK installations, which may lead to various build issues if the selected version of SDK is not compatible with the version of Zephyr.

With the proposed semantic versioning scheme, the Zephyr build system can select for the highest PATCH version that is under a specific MAJOR.MINOR version to avoid such compatibility issues, since any PATCH versions within a specific MAJOR.MINOR version are guaranteed to be backwards compatible.

Backports and Maintenance Releases

Zephyr releases often use the latest SDK version available and, with the proposed semantic versioning scheme and the subsequent Zephyr build system integration strategy, they will be tied to the latest MAJOR.MINOR version of SDK available at the time of their release.

Since a Zephyr release may be supported for up to two years and it is not feasible to introduce breaking changes to a dot (aka. PATCH) release of Zephyr (i.e. you cannot bump the required SDK version for dot releases of Zephyr), it is necessary to backport any relevant SDK bug fixes and provide maintenance PATCH releases for use with the previous Zephyr releases.

This will be implemented by creating the release branches for every MAJOR.MINOR release, backporting patches to them, and releasing maintenance MAJOR.MINOR.PATCH releases from the release branches.

stephanosio commented 1 year ago

cc @tejlmand @nashif

tejlmand commented 1 year ago
  • e.g. SDK 0.15.2 and 0.16.0 may be used interchangeably while 0.14.2 and 0.15.0 may not.

Which is perfectly legal for semantic versioning, and one reason why Zephyr SDK is using major version 0. Ref: https://semver.org/

  1. Major version zero (0.y.z) is for initial development. Anything MAY change at any time. The public API SHOULD NOT be considered stable.

Adopt semantic versioning scheme that clearly defines what each version field means and when each version field is allowed to be incremented.

Actually we are following semantic versioning, and this has been discussed long time back and exactly one of the reasons why we are still at major version 0.

So probably this issue should not be about adopting semantic versioning cause we are already following that, see bullet 4. in semver. This issue should be about what are the requirements to release a major version 1.

stephanosio commented 1 year ago

@tejlmand Did you read the proposal? Because, based on your response, it does not seem like you did.

p.s. also note that this proposal is loosely based on Semantic Versioning 2.0.0, and the MAJOR and MINOR version fields have different semantic.

tejlmand commented 1 year ago

@tejlmand Did you read the proposal?

I did, and to a large extent I agree to the path forward in order to define when to have a first major revision 1.

What I oppose is the indication in the preface that we are not following semantic versioning (or something which resembles it), when in fact we are currently conforming to the scheme because our major version is 0, which means there are no rules and we can basically do as we please (Note I strongly agree that we should improve on this and hence aim at having a major revision >=1).

stephanosio commented 1 year ago

What I oppose is the indication in the preface that we are not following semantic versioning (or something which resembles it), when in fact we are currently conforming to the scheme because our major version is 0, which means there are no rules and we can basically do as we please

That is one way to look at it; but, to say we are following the Semantic Versioning 2.0.0 specifications is stretching it a bit too far because, for all practical purposes, we are not.

According to Semantic Versioning 2.0.0, "Major version zero (0.y.z) is for initial development." Zephyr SDK is well beyond the "initial development" phase and we are currently using it in production; in that sense, we are NOT following the Semantic Versioning 2.0.0.

Also Semantic Versioning 2.0.0 is not the only form of "semantic versioning" out there and we do not intend to follow it as is, as described in this proposal, mainly because Zephyr SDK is, in many ways, dissimilar to conventional software libraries, which the Semantic Versioning 2.0.0 mainly targets.

tejlmand commented 1 year ago

According to Semantic Versioning 2.0.0, "Major version zero (0.y.z) is for initial development." Zephyr SDK is well beyond the "initial development" phase and we are currently using it in production; in that sense, we are NOT following the Semantic Versioning 2.0.0.

From an older Toolchain WG, back before we had Windows and macOS support:

Anas: Zephyr SDK is a reference, not a product

Now that we have gotten support on all major platforms, then I agree we should be moving to a major release 1, but I still think that before doing that, then best would be to have host tools support on all platform.

Toolchain on all platforms (Windows, macOS, Linux) were introduced in Zephyr SDK 0.14, march 2022. Host tools are still only present on Linux release.

So imo it was not until 0.14 we could consider if we should move out of "initial development". Whether or not host tools are a requirement for moving out of initial development is open for discussion, but I don't believe you can say: Zephyr SDK is well beyond the "initial development" phase

I guess it's for a project to decide what is considered initial development, not semantic versioning. But I guess we are getting into semantics now :wink: my point was mainly that I disagreed with the indication that we just randomly chose a version number, when in fact there was a reason for being at major version 0.

tejlmand commented 1 year ago

Extra comment:

major incompatible change(s) minor incompatible change(s)

how can an incompatible change be major or minor, if it's incompatible then it's incompatible, so I suggest removing major and minor in those sentences and instead make precise when we consider an incompatible change should warrant a major bump.

For example:

MAJOR version when you make incompatible change(s) to components in the toolchain or changes to host tools which affects the overall user experience. MINOR version when you make incompatible change(s) to one or more components in non-mandatory extra components not required for building a project.

Example where a major bump is required are incompatible changes to gcc, ld, objcopy, and similar tools that are used for build, link and postprocessing. Example where a minor bump is acceptable are incompatible changes to bossac, qemu, openocd.

That in my opinion remove the ambiguity on when an incompatible change is impacting enough users to warrant a major bump. It still has a slight opening for discussions on when a change in for example qemu is allowed to bump major, but it makes it clearer when major must be bumped. That is when something related to building changes.

stephanosio commented 1 year ago

But I guess we are getting into semantics now my point was mainly that I disagreed with the indication that we just randomly chose a version number, when in fact there was a reason for being at major version 0.

Sure, all the nits aside, at least for the major version 0, I agree that was a conscious decision.

What this proposal mostly concerns, however, is to bring the "semantic versioning" to even 0.y.z versions, of which the y and z parts are currently more or less random, because we have been seeing various user experience-related problems related to this, as described in this issue, and we do need a solution, which this issue proposes.

Another option would be to wait until what we consider 1.y.z. (i.e. with host tools) comes and we start enforcing either the proposed or the de-facto standard semantic versioning scheme; but, that will take some time and we do not want the user experience issues to persist until then.

stephanosio commented 1 year ago

how can an incompatible change be major or minor, if it's incompatible then it's incompatible, so I suggest removing major and minor in those sentences and instead make precise when we consider an incompatible change should warrant a major bump.

I agree the current wording involving the "major" and "minor" incompatible changes for the conditions under which the MAJOR and MINOR version numbers must be bumped are rather vague and imprecise; but, the ambiguity is intentional -- the idea is to leave the details regarding the "major" version bump up to the discretion of the community and the SDK maintainer because:

In fact, we could very well do with just two version number fields -- i.e. MAJOR and MINOR in which a MAJOR bump indicates an incompatible change and a MINOR bump indicates a compatible change; but, the concern with that approach is that we could very well end up with the major version of 255 in the foreseeable future, which is not necessarily a bad thing but not very conventional and/or intuitive either.

Let us discuss this in the upcoming Toolchain WG meeting with a broader audience and try to find what works the best for us.

tejlmand commented 1 year ago

because we have been seeing various user experience-related problems related to this, as described in this issue, and we do need a solution, which this issue proposes.

and I agree very much to this, we should surely improve on this, irrespective of whether we are bumping to a 1.x release or sticking for some longer time to 0.x. Note, nothing in semantic versioning forbids a 0.x release from adhering some self-imposed restrictions / rules so we are free to setup own guide-lines / rules.

but, the ambiguity is intentional -- the idea is to leave the details regarding the "major" version bump up to the discretion of the community and the SDK maintainer because

I prefer to be a bit more precise to avoid long discussions on whether or not a specific change is important enough / impacting sufficient amount of users to warrant a major bump. I'm concerned that being to vague here bears a risk of still experiencing various user-related problems.

Just because we start out with a more precise conditions on when to bump doesn't mean we can't adjust those based on later experiences and feedback / wishes of community and maintainer.

but, the concern with that approach is that we could very well end up with the major version of 255 in the foreseeable future

I'm not sharing that concern. Based on the major following the description I proposed above I don't see why we should suddenly start changing major version so frequently.

Let us discuss this in the upcoming Toolchain WG meeting with a broader audience and try to find what works the best for us.

Agreed, and thanks for opening this rfc. It's very valuable, and to me it mostly seems we are discussing details, meaning we share the intention on the overall proposal 👍

stephanosio commented 1 year ago

but, the concern with that approach is that we could very well end up with the major version of 255 in the foreseeable future

I'm not sharing that concern. Based on the major following the description I proposed above I don't see why we should suddenly start changing major version so frequently.

You forgot to quote the preceding condition: "In fact, we could very well do with just two version number fields" in which case the MAJOR bump would be done for both "major" and "minor" incompatible changes, which was an example I gave to point out that the x and y bump in the version number x.y.z based on the proposed versioning scheme are semantically equivalent and it was only the human perception of these numbers that motivated the x and y to be separate fields.

Example where a major bump is required are incompatible changes to gcc, ld, objcopy, and similar tools that are used for build, link and postprocessing. Example where a minor bump is acceptable are incompatible changes to bossac, qemu, openocd.

There are many problems with the above proposal (i.e. major bump for incompatible toolchain changes and minor bump for incompatible host tool changes):

In my opinion, the initial proposal does sufficiently lay out the basic and essential requirements for MAJOR and MINOR version bump without going into too much details so as to require the description of every permutation of the incompatible changes (which we would surely want to avoid):

Given a version number MAJOR.MINOR.PATCH, increment the:

  1. MAJOR version when you make major incompatible change(s) to multiple components that affect the overall user experience.
  2. MINOR version when you make minor incompatible change(s) to one or more components that do not affect the overall user experience.
tejlmand commented 1 year ago

Does an architecture-specific (e.g. only affecting the SPARC architecture) incompatible GCC change really warrant a MAJOR bump?

Yes, cause it has a major impact on users of sparc arch. I don't want to make some archs second class citizens, like saying an incompatible change in sparc is minor bump, but an incompatible change on arm is a major bump.

Does a QEMU update that breaks all supported architectures deserve only a MINOR bump?

If you read my proposal then it must be at least a minor bump, but we are allowed to make it a major bump in this example, hence the reason for this part: or changes to host tools which affects the overall user experience.

I wanted to let users know that if incompatibilities in the toolchain itself are introduced then we promise to bump major. Basically be clear on when we must bump major, but don't remove the option of bumping major if there are circumstances that we believe require us to do so.

For instance, a major change in the distribution format and/or the setup script that completely changes the OOB user experience?

If the extracted toolchain / setup script is still compatible with the latest Zephyr release, then why should the format warrant a major bump ?

What is important is that if Zephyr code base has:

find_package(Zephyr-sdk 1.0)

will Zephyr-sdk 1.0 in format X and Zephyr-sdk 1.3 in format Y both work ? If yes, then I don't see a reason for bumping major in this case. You could even distribute Zephyr-sdk 1.3 in both format X and Y.

If you change the final installation in such a way that find_package(Zephyr-sdk 1.0) will fail to find or use the new Zephyr SDK, then yes we need to bump Zephyr SDK major to 2.0, cause the change introduced has broken the toolchain seen from Zephyr build system, hence the toolchain has incompatible changes and first rule applies.

without going into too much details so as to require the description of every permutation of the incompatible changes

We will need to go into details at some point, we can not avoid that, and therefore I much prefer to take the discussion up-front and then adjust / clarify the description as needed, instead of having to go through such discussions on each release.

But that's my preference, but ymmv.

stephanosio commented 1 year ago

Yes, cause it has a major impact on users of sparc arch. I don't want to make some archs second class citizens, like saying an incompatible change in sparc is minor bump, but an incompatible change on arm is a major bump.

So let me confirm this again, what you propose is to bump Zephyr SDK version from, let's say, 1.0.0 to 2.0.0 when we, for instance, change the name of an obscure architecture-specific GCC flag such as -mtp-regno to -mtp-regnum, is that correct? Because that does not sound very reasonable or intuitive to me (but at that point, this is approaching the personal preference territory and there is no single "right" answer; after all, there is nothing technically wrong with having a MAJOR version number of 255).

I wanted to let users know that if incompatibilities in the toolchain itself are introduced then we promise to bump major.

I do not agree with that. A toolchain change can be "major" or "minor" depending on the nature of the change. A host tool change can be "major" or "minor" depending on the nature of the change.

I do not see any reason why a toolchain change should, by default and unconditionally, warrant a MAJOR version bump. I would at least want to leave the option of allowing an incompatible toolchain change to be a MINOR version bump depending on the nature of the change (e.g. if a toolchain change affects all architectures, MAJOR bump; if it affects only a small number of architectures, then MINOR bump).

Basically be clear on when we must bump major, but don't remove the option of bumping major if there are circumstances that we believe require us to do so.

Maybe the attempt to classify "major" and "minor" changes and trying to set the version number based on that is where I got it wrong to begin with -- that will almost certainly result in heated discussion every time we make a new SDK release even if we set some ground rules, because what should be considered "major" or "minor" is highly subject to interpretation and what is at stake for each individual. After some thought, I cannot see how such an imprecise versioning scheme can work in favour of anyone in general.

An alternate proposal I have in mind is to simply set the first version field chronologically. So, in the version number format X.Y.Z:

This way, we can get rid of any ambiguity in the differentiation of "major" and "minor" changes.

The proposed Zephyr Build System Integration and Backports and Maintenance Releases strategy above will still work without any changes to them.

tejlmand commented 1 year ago

change the name of an obscure architecture-specific GCC flag such as -mtp-regno to -mtp-regnum, is that correct?

is gcc blindly removing it, or is gcc properly deprecating the old flag when introducing the new flag ? afaik gcc tends to deprecate old flags, which is not an incompatible change, cause the old flags are still present.

Is the feature / flag considered unstable / experimental ? In such case a change in itself is not considered a breaking / experimental change.

I do not see any reason why a toolchain change should, by default and unconditionally, warrant a MAJOR version bump.

I didn't say any toolchain change warrants this, I agree on the incompatible change. Incompatible change would be a non-experimental flag / feature suddenly being removed without deprecation. Removal of unstable / experimental features are always allowed without a major bump, that's the whole point of parts of a sw product being marked unstable / experimental.

But your making a good point, maybe incompatible changes are not the best, but description could be updated to reflect better that this applies for changes to / removal of stable api / flags / features on toolchains. (which is also my impression on when gcc bumps major number, but there could be exceptions)