What's the solution to versioning woes?

[jskeet]

I tried several times to think of a better title, but failed. I realize it's slightly hyperbolic at the moment. Versioning worries me a lot, and the point of this question is mostly to find out how much it worries you and others too. (I'm focusing on .NET here, but if other environments have good answers that would be applicable to .NET, I'd love to hear about them.)

SemVer sounds fine until it comes into contact with the real world, at which point it seems that really popular packages should never, ever release new major versions - at least with our current abilities.

Json.NET is currently on version 10.0.3, but it hasn't really had many significant breaking changes over the years, as far as I'm aware. Suppose version 11.0.0 was really breaking - like, "Most of the code that uses Json.NET would need to be changed, even if only a trivial way." (Imagine JsonConvert were renamed, for example.)

If an application tried to take dependencies on libraries X and Y, where X depended on 10.0.3 and Y depended on 11.0.0, you'd end up with X failing at execution time due to trying to use the old code removed in 11.0.0.

This isn't a new problem, but the more the ecosystem grows, the more acute it will become. Are you aware of any efforts to solve or mitigate this?

(I'm going to be blogging about these concerns in the future, just to give fair warning...)

[shanselman]

I'm starting to think we may need to have language features that go beyond type forwarding and make strongly type C# have opt-in duck typing. What do you think?

[jskeet]

I suspect it'll have to be a mixture of language features and runtime features... and that even then, it won't fix everything, but that would make whole classes of change possible that aren't possible right now.

I suspect we may want to revisit the idea - whose full name I've forgotten now - of "non-assembly-bound" interfaces, where if one package wants to be able to depend on an interface (e.g. for logging) without depending on the package that normally provides it, it can do so, and the types get unified at execution time.

I also like the idea of being able to provide "upgrade" NuGet packages (or tools within source packages) that use Roslyn to make changes to source code that depends on the library. That will make it at least somewhat easier for the whole ecosystem to keep closer to "current". The gap between "current" and "oldest that's still in use" is part of the problem.

[analogrelay]

I suspect we may want to revisit the idea - whose full name I've forgotten now - of "non-assembly-bound" interfaces, where if one package wants to be able to depend on an interface (e.g. for logging) without depending on the package that normally provides it, it can do so, and the types get unified at execution time.

Assembly-Neutral Interfaces (ANIs) :'(. I miss those days.

[MadsTorgersen]

I suspect both ANIs and Extension Everything would help. I am skeptical about automatically rewriting user code with Roslyn-based tools, but it could be useful if it comes as a refactoring-like experience, where the developer participates. I also like the idea of "compat pack" libraries that make removed/renamed members available to downlevel consumers as extension members on top of the new versions.

[jskeet]

@MadsTorgersen: Completely agree about the refactoring part being an interactive experience, when performed on code between strangers. (Large-scale automated refactoring on an internal codebase is a different matter.)

In terms of the compat-pack idea, for Noda Time we released a v1.4 as a "bridging" release - it made all the members not in 2.0 obsolete, and provided extension methods for the new names etc... at least as far as was feasible. It couldn't cover everything though. I haven't had feedback yet as to whether it actually helped people, mind you.

[MadsTorgersen]

@jskeet:

Interesting!

It couldn't cover everything though.

Was that mostly a result of the current limitations of extension methods (instance methods only), or was there a more fundamental shortcoming of the approach?

I haven't had feedback yet as to whether it actually helped people, mind you.

Yeah, this is part of the problem: it takes a long time before you realize whether versioning mechanisms actually help! It'd be great if something could be learned from NodaTime's approach over time. I'd love to hear any takeaways that emerge!

[jskeet]

@MadsTorgersen: The problems were mostly around type changes. For example, in LocalDate:

// 1.3
public IsoDayOfWeek IsoDayOfWeek => ...;
public int DayOfWeek => ...;

// 2.0
public IsoDayOfWeek DayOfWeek => ...;

So the type of the property has changed (for long-term simplicity, basically). It's easy enough for clients to change their code - if they were using the IsoDayOfWeek property they can just change to use DayOfWeek, and if they really want the integer, they can just cast to it. But there's no way of bridging that.

(There were a few places where we removed functionality with no replacement, which of course couldn't be bridged - in those cases, I just made those members obsolete and indicated that there was no replacement.)

I'm sure there was some situation that I couldn't quite handle that was more interesting, but I can't remember it right now. I may have a little look over the next few days.

One piece of experience that definitely could suggest more "proper" tooling is the ability to do API diffs. I built my own tooling for Noda Time which helped on this, so we can see:

• Changes from 1.3 to 1.4
• Changes from 1.4 to 2.0

Basically I kept on making things obsolete until the list of "newly obsoleted members" in 1.4 was as close as possible to the list of "removed members" in 2.0.

I'm hoping to present to the .NET TSG a list of "Tools I've written in open source projects that don't really belong in those projects" which would definitely include this :)

Will certainly pass along any feedback I do get around the Noda Time versioning.