Are major version changes for YANG Semver allowed for BC changes?

[rgwilton]

Is it acceptable to bump the major version number even if the module does not contain any NBC changes?

A corollary to this is: Is it acceptable to bump the major version number if there are no changes at all? E.g., bump the API version when a new major product version is being released?

A second corollary is: Does the same logic apply to minor version changes. E.g., can the minor version number be bumped if only editorial changes are made.

Somewhat related to https://github.com/semver/semver/issues/475

[jwdonahue]

What follows does not in any way represent the views of the SemVer maintainers. I am not a SemVer maintainer. I've simply got some skin in the game as a tool builder and long-time interested party. I also have some experience developing and testing network protocol stacks and components at various layers of the internet protocols.

I would like to address this question, in the context of YANG/NETCONF:

Is it acceptable to bump the major version number even if the module does not contain any NBC changes?

And its corollary:

Is it acceptable to bump the major version number if there are no changes at all?

The terminology around YANG versioning is unfortunate. NBC? Non-Breaking-Change or Non-Backwards-Compatible? The SemVer terminology is Breaking Change and Non-Breaking Change or Backwards Compatible Change. I'll simply use Breaking Change and Compatible Change here.

For the purposes of this post, I'd like to rephrase those two questions as:

1. Is it acceptable to bump the YANG version number when there have been no changes?
2. Is it acceptable to bump the YANG major version number when there has been no Breaking Change?

I believe these are distinct and overlapping concerns, and I have been answering one form or another of them, over and over for years.

There is much YANG/NETCONF detail that I am likely unaware of, so I'll state some assumptions before continuing:

1. Network devices tend to dial-home to check availability of updates, based on their current version number.
2. Code packages are pulled from a variety of services like NPM, Nuget, Crates, etc.
   • These are published by developers with variable development, test and release practices, some of whom are unaware of NETCONF/YANG.
3. Network API's and data structures MAY be versioned independently of any packages of implementations.
   • These are used for device status and control purposes.

Starting with assumption 1, I think it is safe to say that any frivolous version bump is unacceptable, given the bandwidth consumption, any device down-time during an update, and the associated risk to device integrity inherent in any update. Now consider common DevOps practices; where potentially thousands of builds occur every day, and it is common practice to bump the patch number and modify tags for each and every one of those builds. Such environments often have highly distributed and parallel builds, that may include some duplication where it is cheaper to build it from the local repo, than to pull it from an artifact store.

Now, build systems do tend to be gated at some point, to prevent too much chaos from leaking out of their build labs, but not everyone of them is capable, or willing to prevent all possible modes of extraneous version bumps. So my advice wrt to question 1 and assumption 1 is; no it's not acceptable, but it can't always be avoided, so any spec language on the subject; SHOULD advise against it, but can't possibly prohibit it.

That leaves us with Q2 wrt A1. Is a frivolous bump harmful in that case? I would argue that it probably is not. Any automated update scheme should prevent taking breaking changes without explicit human intervention, so the impacts are likely quite low. Again, we should consider common developer practices. It's not uncommon for a developer team to spin up vNext, by bumping the major version, making some tooling updates in preparation for the coming sprints, and then test that build system by generating an artifact that has a major version bump AND contains no product changes. They would then test that system by enabling their test devices to consume that package and verify that nothing was broken by the tool changes.

Again, best practices, requires a strong gate between the internal systems and consuming public, but not everyone follows best practices. I have worked on projects where we had large groups of early adopters at every step in the development chain. We would always do our best to prevent non-members of a specific ring from accidentally consuming our pre-releases, but when you have any kind of early release scheme, there is always some leakage.

So I don't believe it is desirable to prohibit frivolous version bumps wrt to assumptions 1 & 2. We should certainly discourage frivolous releases of minor and patch bumps, but allow and even expect them for pre-releases.

Moving on to assumption 3...

Here's where the details really come into play. The simplest setup is a single server, servicing a single API, call it ServiceX. A major version bump of ServiceX breaks all clients and requires them to update. This can be very costly, so we should avoid any frivolous major version bumps, but should we prohibit them entirely? What if I have thousands of clients running in the range 1.y.z, and an exploit is found that affects all but 1.yN.z where N might not be the highest possible Y? Let's just say that I have potential liabilities if I allow the update laggards to continue using their clients, and there is no change to the service code that could possibly fix the problem, but the service is an integral requirement for the exploit.

I can assure you that no MUST phrasing in a spec is going to stop me from bumping ServiceX to 2.0.0 and forcing my customers to update their clients in order or discontinue its use. I understand the need for tight specs and abhor unnecessarily vague statements, but it is really hard to write a tight spec that is also practical, when there are more than one stakeholder in the mix. In this case, it's really easy for me to say that YANG-SemVer and SemVer itself, MUST not ever attempt to prohibit a version bump for any reason whatsoever.

The SemVer spec requires version bumps under specific circumstances, but does not prohibit them for any other reasons. The main reason for the limited scope of the spec in this regard, is that it's just not possible to imagine all possible use cases and the bandwidth available in the SemVer string is extremely constrained.

It's been my experience that interacting sentient agents tend to punish or ban ecosystem members who do things that harm them. I can understand the distain for version bumps driven by marketing departments for no reason other than to rev their own propaganda. At the same time, I don't think there's many of those departments left, that haven't gotten the message from their customers, that frivolous churn in automated systems is dangerous and wasteful. I know there have been a few obnoxious cases here and there, but I haven't seen any evidence that it's a major ongoing problem.

[jwdonahue]

I will add that for testing and experimentation purposes, it may be necessary to bump all or any of the version fields. Consider infrastructure testing. If you write an absolute prohibition for some behavior into a spec, those generally find their way into tooling. In this case, you would be hard pressed to test your end-to-end infrastructure if there was a ban on version bumps in the absence of appropriate changes. If you're testing infrastructure, you don't want to have to jump through hoops to get the tooling to take your changes and you don't want to introduce payload changes, that could yield false positive test failures, having nothing to do with the infrastructure under test.

[rgwilton]

@jwdonahue, thanks for your comments and input. We also discussed this in our meeting yesterday and I believe that what you propose is inline with what we are thinking. A couple of comments inline ...

What follows does not in any way represent the views of the SemVer maintainers. I am not a SemVer maintainer. I've simply got some skin in the game as a tool builder and long-time interested party. I also have some experience developing and testing network protocol stacks and components at various layers of the internet protocols.

Okay. Ideally, it would be useful to know whether it is plausible that the Semver definition will change to be more strict. My current reading of semver/semver#475 is that is unlikely.

The terminology around YANG versioning is unfortunate. NBC? Non-Breaking-Change or Non-Backwards-Compatible? The SemVer terminology is Breaking Change and Non-Breaking Change or Backwards Compatible Change. I'll simply use Breaking Change and Compatible Change here.

I've raised #80 to see if it feasible to change this.

3. Network API's and data structures MAY be versioned independently of any packages of implementations.

These assets generally represent Network Management APIs, and hence really fit into your third assumption. I.e. they define the data module used between network devices and network management clients. These APIs may be defined by standards bodies (e.g., IETF or IEEE), where the API is versioned entirely independently from implementation, or they may be defined by vendors, where the API and implementation are likely to be more closely coupled - although even in that scenario it is likely that there will be multiple implementations relative to a single API version.

Your comments of assumption 3 make sense and seem like a pragmatic consideration. Thanks again for taking the time to provide input.

[jwdonahue]

You're welcome. It helps keep my head in the game. I've been side-lined due to CoVid and family matters for over a year.

Regarding whether the SemVer spec will tighten up or not, I have some additional thoughts for you to consider.

SemVer is an As-Built, not a true standard. The original author has stated explicitly that he was defining with words, what was already a common practice.

Current and past maintainers are/were far more interested in preventing drift in that document, that would cause any of them to have to modify their packaging tools. Their user base consists of package consumers.

If you are going to hitch your hoarse to their wagon, I recommend that you apply for membership as a maintainer (just file a new issue to ask for membership). But I really think that someone should consider proposing an IETF standard for YANG-SemVer that is forked from SemVer 2.0. Here's some things to consider:

1. Your use case seems to be pure API versioning, and you probably don't really care about how implementations are managed/distributed.
2. You have little control over the future of the SemVer spec, even if you were among the maintainers, who's interests don't necessarily align with your own. Making changes to the SemVer spec requires an act from a higher power than any I've ever seen, except that they do seem to be working on adding language around defining the limited commonalities; around how each of their various tools handle range specifications (most of which don't use well known mathematical interval/range notations).
3. The maintainers don't actually follow SemVer versioning of the specification itself! Any reference to SemVer 2.0.0 may have pointed to a much earlier version of it, and whatever it points to today will not be the same thing tomorrow. Doesn't the IETF have its own standards versioning scheme?
4. Tags in the current scheme, are either prerelease or build-meta tags, and the sorting rules for prerelease tags are non-semantic (Dev has higher precedents than Beta!). You could add something like an enumeration type tag where (Dev < Alpha < Beta < RC), or better yet, provide a means for developers to define their own. There's room here to cover something other than ASCII sort order, while preserving the SemVer style prerelease tag. Perhaps a quality tag allowing for a combination of flags for security reviewed, unit tested, human in the loop, etc.
5. It is not possible to differentiate between a SemVer 1.0.0 string, a SemVer 2.0.0 string, or any other random triple of dot separated numeric values.
6. Level of risk, wrt to selecting a higher version, is essentially defined by SemVer as "publisher thinks this release might or might not break something". Perhaps you could use something with a bit more precision?

[rgwilton]

You're welcome. It helps keep my head in the game. I've been side-lined due to CoVid and family matters for over a year.

I hope that you finally seeing the other side of that!

Regarding whether the SemVer spec will tighten up or not, I have some additional thoughts for you to consider.

SemVer is an As-Built, not a true standard. The original author has stated explicitly that he was defining with words, what was already a common practice.

Current and past maintainers are/were far more interested in preventing drift in that document, that would cause any of them to have to modify their packaging tools. Their user base consists of package consumers.

Yes, and as you say below, that is somewhat different to our use case.

If you are going to hitch your hoarse to their wagon, I recommend that you apply for membership as a maintainer (just file a new issue to ask for membership). But I really think that someone should consider proposing an IETF standard for YANG-SemVer that is forked from SemVer 2.0. Here's some things to consider:

I don't think that we are necessarily trying to hitch ourselves to semver.org in general, but just the specific 2.0.0 version.

This is effectively what we are trying to do. I.e. this specification, when it gets published, will formally define the versioning scheme that is used for any YANG modules that are labelled with rev:revision-label-scheme "yangver:yang-semver";

1. Your use case seems to be pure API versioning, and you probably don't really care about how implementations are managed/distributed.

That is basically correct. The implementations will be versioned separately, using whatever versioning scheme the vendor decides on. I believe that is often just a release based versioning scheme.

2. You have little control over the future of the SemVer spec, even if you were among the maintainers, who's interests don't necessarily align with your own. Making changes to the SemVer spec requires an act from a higher power than any I've ever seen, except that they do seem to be working on adding language around defining the limited commonalities; around how each of their various tools handle range specifications (most of which don't use well known mathematical interval/range notations).

I'm not sure that it too much of a problem for us, except for your comment below about Semver 2.0.0 not being stable! Our draft/RFC will end up being backwards compatible to the specific Semver 2.0.0 revision, and not necessarily other versions. If Semver 2.0.0 reference is not stable that could be an issue, otherwise, it probably doesn't matter too much, except for the potential to cause confusion in the future.

3. The maintainers don't actually follow SemVer versioning of the specification itself! Any reference to SemVer 2.0.0 may have pointed to a much earlier version of it, and whatever it points to today will not be the same thing tomorrow. Doesn't the IETF have its own standards versioning scheme?

I had assumed that the Semver 2.0.0 definition was static, so it is slightly alarming if the definition is changing, since that would become an unstable reference for us. Forking the Semver 2.0.0 repository definition might be an answer. Tracked with #90

Effectively, no IETF doesn't define it's own generic versioning scheme. Probably the YANG Semver draft is as close as it gets. Lots of protocols have their own bespoke versioning schemes that are tailored to their requirements.

4. Tags in the current scheme, are either prerelease or build-meta tags, and the sorting rules for prerelease tags are non-semantic (Dev has higher precedents than Beta!). You could add something like an enumeration type tag where (Dev < Alpha < Beta < RC), or better yet, provide a means for developers to define their own. There's room here to cover something other than ASCII sort order, while preserving the SemVer style prerelease tag. Perhaps a quality tag allowing for a combination of flags for security reviewed, unit tested, human in the loop, etc.

Okay, this might be something that we need to consider further. Tracked with #91.

5. It is not possible to differentiate between a SemVer 1.0.0 string, a SemVer 2.0.0 string, or any other random triple of dot separated numeric values.

For us, that isn't a problem, the YANG module will specify the versioning scheme in use with a tag: rev:revision-label-scheme "yangver:yang-semver"; If this tag exists in the module, then tools are expected to process the version number according to the YANG Semver document.

I.e., someone else (within or outside IETF) would be allowed to define a different versioning scheme for their YANG modules if they wanted, although that it obviously less interoperable if different folks are using different schemes.

6. Level of risk, wrt to selecting a higher version, is essentially defined by SemVer as "publisher thinks this release might or might not break something". Perhaps you could use something with a bit more precision?

Yes, we will probably end up having that.

There are programmatically readable data models after all, so tooling will be able to compare the data models and in many cases decide whether the changes may break clients or not. In some cases it is hard for tooling to do this (e.g. if the semantics are a data node changes via the text that defines/describes it's behaviour).

E.g. https://tools.ietf.org/html/draft-ietf-netmod-yang-solutions-01 describes the overall solution here, which comprises of 5 documents, with YANG Semver only being one of them.

All of this work is formally happening as part of the NETMOD WG in IETF. I will drop you an email to see if you are interested in participating in the IETF discussions more directly.

[xorrkaz]

This is believed to be addressed in rev -03.