search

eclipse-archived / ceylon-sdk

Standard platform modules belonging to the Ceylon SDK
http://www.ceylon-lang.org
Apache License 2.0
72 stars 60 forks source link

SDK module versioning #520

Open quintesse opened 8 years ago

quintesse commented 8 years ago

So we're getting to a point where I think it becomes important to decide what we are going to do about SDK versioning. Right now we just follow the Ceylon distribution versioning, but we're already running into situations where this is not such a good idea. See for example this discussion on the mailing list: https://groups.google.com/d/msg/ceylon-dev/dYOr7Y0wu9M/6a_Sjg-FCAAJ

The big advantage of the current system is that it's easy, it's easy to maintain and it's easy to use.

The big disadvantage is that the version number progression in no way relates to the progression of the separate modules (in fact there might be no progress at all).

We need to come up with some ideas and decide what to do.

quintesse commented 8 years ago

On the mailing list @jvasileff said:

In theory I really like the idea of sdk module versions being fully independent, but in practice this will make it much more difficult to add or upgrade dependencies. This can already be time consuming since versions must be listed in each module.ceylon, and would be much more so without being able to search&replace, and if you have to use Herd to determine the correct version for each dependency. Autocomplete in the IDE helps in some cases, but is not a general solution.

True, but isn't that true for every other module out there that is not part of the SDK?

And to me it might make things easier but it doesn't make it right

What do we want to do? Really call this completely rewritten module ceylon.html/1.2.3? Or worse, if we don't want to wait for Ceylon 1.2.3 we'll need to call it 1.2.2.1 !

Or we put the version number in the name: ceylon.html5/1.2.2 ... ok, that's weird. So how is a user going to decide now? Should they use ceylon.html or ceylon.html5? Make ceylon.html deprecated? So what happens with the next rewrite ceylon.html6?

It also means that the number gives no indication whatsoever to it's "maturity", everything is just 1.2.2, even when some modules have seen an enormous amount of work while others haven't been updated since the day they were added. It's paints a completely incorrect picture.

I say we bite the bullet, allow SDK modules to have individual versions and have the them evolve on their own.

lucaswerkmeister commented 8 years ago

(EDIT: this was written when Tako’s comment had not yet been posted)

The other end of the spectrum: every module is released independently of each other, not simultaneously, and with independent version numbers. (In particular, we would stop releasing new versions of modules that had no changes whatsoever – unless we continue breaking binary compatibility regularly.) “The Ceylon SDK” would then mean little more than a collection of modules that share a repository, the ceylon. prefix (though not exclusively – see formatter, AST, converter, Casey’s parser, …), and “sponsoring” (ditto – formatter and converter ship with the IDE).

Big disadvantage: it’s no longer obvious what the most recent version of any given SDK module is. But with some tooling (IDE, Herd, ceylon-sdk README) I think this should be fine.

quintesse commented 8 years ago

Btw, input needed from all involved: @gavinking @FroMage @tombentley @chochos @davidfestal @thradec etc

chochos commented 8 years ago

why not simply a fourth number? ceylon.collection/1.2.1.0, then 1.2.1.1, 1.2.1.2, etc. All those versions belong with Ceylon 1.2.1. We reset them with each Ceylon release: 1.2.2.0 but then any module can have a new release after that, 1.2.2.1, 1.2.2.2, etc.

DiegoCoronel commented 8 years ago

As a user of frameworks/libs my opinion is that its more natural to expect a 2.0.0 version because its the most common case for libraries upgrades. I dont care about the compiler version that was used, really, I am from java world and since version 1.5 i dont care about that.. Will be a big surprise if I have to look for libraries and worry about that.. i know that some times is necessary to break binary compatibility, but its not the common case... so, i dont believe we should make those patterns based on compiler versions.. this way we are saying that break compatibility IS the common case and everyone should worry about that.

quintesse commented 8 years ago

why not simply a fourth number?

Because for a user the difference between a 1.2.2.1 and a 1.2.2.2 is normally completely unimportant, a super minor relase, probably some uninteresting typo in an error message or something. Not a complete f-ing rewrite where you have to port your application to an entirely new API !

Like I said, it gives users a completely wrong idea about what versions mean. In the current system modules that never change forcibly get new versions all the time. And modules that get completely rewritten have the same version as all the others, in no way showing all the progress that has been made on them.

DiegoCoronel commented 8 years ago

Agree with @quintesse , +1

LisiLisenok commented 8 years ago

May I add a couple words? Some tool which checks compatibility and suggests the best suit versions may be usefull. For example, in your project you use compiler 1.2.1 and ceylon.collection module, the tool looks all available versions and suggests the latest one which compatible with 1.2.1 compiler. Lets it is ceylon.collection 1.2.3. But you use, say, ceylon.json which is only 1.2.2 compatible with 1.2.1 compiler and is not compatible with ceylon.collection 1.2.3, so the tool must sugget ceylon.collection 1.2.2. Sorry for messy. I think with such tool it is doesn't matter exact module versionning system and it must be easy to upgrade to new modules / compiler versions.

gavinking commented 8 years ago

I personally think completely independent version numbers would be hell. I prefer the scheme proposed by @chochos which is essentially what we've been doing until now.

If necessary, we could have a two-part version number of form <platform-version>-<module-version>, for example, 1.2.1-2.1.

quintesse commented 8 years ago

If necessary, we could have a two-part version number of form -module-version, for example, 1.2.1-2.1.

What advantage would that have and how would it be less of a "hell"?

And you didn't address anything I mentioned, for example about our versions not being related to actual code change. SDK version numbers would basically have no meaning at all. People could upgrade to what they think is just a minor update only to find out that nothing compiles anymore.

tombentley commented 8 years ago

Like I said, it gives users a completely wrong idea about what versions mean.

Well, what do versions mean? I know we kinda want more more "minor changes" to correlate to increments of more minor parts of a dotted decimal version number, but at what point does a minor change necessitate incrementing a more major part of a version number. It's all very wishy washy. And in the absence of tooling to support whatever rules you propose it's not going to work anyway because people make dumb mistakes. So you can never infer from comparing two version numbers how the changes will impact you. That's what release notes are for.

So sure, it would be surprising that ceylon.html/1.2.2 is not source compatible to 1.2.1, but so long as I can use ceylon.html/1.2.1 with ceylon.collection/1.2.2 and upgrade my dependency on ceylon.html in my own good time, I don't think it's a big deal.

quintesse commented 8 years ago

I don't think it's a big deal.

Ok, I'm not going to discuss this any further. We'll just go with 1.2.1.1 then. But I'm sure people are going to take us to task over this at some point. I think it's a big mistake.

tombentley commented 8 years ago

If necessary, we could have a two-part version number of form -module-version, for example, 1.2.1-2.1.

Wasn't that sort of thing a problem for OSGi?

davidfestal commented 8 years ago

yes, exactly, it's incompatible with the OSGI support : https://github.com/ceylon/ceylon/issues/6006

2016-03-02 13:28 GMT+01:00 Tom Bentley notifications@github.com:

If necessary, we could have a two-part version number of form -module-version, for example, 1.2.1-2.1.

Wasn't that sort of thing a problem for OSGi?

— Reply to this email directly or view it on GitHub https://github.com/ceylon/ceylon-sdk/issues/520#issuecomment-191220896.

lucaswerkmeister commented 8 years ago

Well, what do versions mean? […] It's all very wishy washy. […] So you can never infer from comparing two version numbers how the changes will impact you.

@tombentley perhaps I’m missing something, but aren’t you essentially asking for SemVer?

gavinking commented 8 years ago

What advantage would that have and how would it be less of a "hell"?

Well the point is that it should at least partially satisfy folks who are attached to this "semantic versioning" stuff. As you guys know, I'm not, and I think "semantic versioning" is mainly an emotional security blanket. OTOH, it's clear that we have two notions of "what version of the SDK module do I have here":

And those two numbers just aren't going to align going forward. We already have modules which are basically in maintenance mode, along with others whose APIs change quite often.

So, if the version number is supposed to be at all meaningful (semantic), then there just has to be two of them.

gavinking commented 8 years ago

yes, exactly, it's incompatible with the OSGI support :

That's rubbish. The OSGi module descriptor transpiler can easily chop off one of the versions.

gavinking commented 8 years ago

@tombentley perhaps I’m missing something, but aren’t you essentially asking for SemVer?

SemVer is a cargo cult. It solves no interesting issues in software engineering.

tombentley commented 8 years ago

Not really @lucaswerkmeister. I'm saying that while that seems attractive I'm not sure it really works very well in practice because it can be hard for people to reason correctly about what is a compatible change etc. Is that source or binary compatibility, for example? If it's BC then how doe that work in a language with multiple backends with their own evolving BC provisions? Is it's source compatibility then is something as "trivial" at adding/removing an annotation compatible? I guess it depends on the annotation. Presumably changing doc is OK?

While it's a nice statement of what we might like to be true, SemVer doesn't really give us much help in making it true.

gavinking commented 8 years ago

Note that my proposal assumes that the SDK release cycle remains coupled to the compiler release cycle. If that's not going to be the case in future, then the first version number isn't a ceylon version number, it's an SDK platform version number.

My proposal solves the following problems:

I don't see how one can satisfy those three requirements via any other versioning scheme.

davidfestal commented 8 years ago

The OSGi module descriptor transpiler can easily chop off one of the versions.

@gavinking : could you precise a bit what you mean here ? I don't see very well what you mean regarding the existing OSGI support implementation

gavinking commented 8 years ago

@davidfestal currently, we generate a manifest that looks like this:

Manifest-Version: 1.0
Bundle-SymbolicName: interop
Export-Package: interop;version=1.0.0
Bundle-Version: 1.0.0.v20160302-1224
Require-Bundle: ceylon.language;bundle-version=1.2.2;visibility:=reexp
 ort,com.redhat.ceylon.dist;bundle-version=1.2.2;visibility:=reexport
Bundle-ManifestVersion: 2
Bundle-ActivationPolicy: lazy;exclude:="*"
Require-Capability: osgi.ee;filter:="(&(osgi.ee=JavaSE)(version>=1.7))
 "

We code which generates this manifest can easily strip of the bit of the version before the -, in both Bundle-Version and Require-Bundle.

gavinking commented 8 years ago

The name of the .car is irrelevant, since for OSGi we have to rename it anyway.

davidfestal commented 8 years ago

Ah, OK, I see what you mean now. But do we still consider that the module version (without the platform version) is unique. That means the platform version in only informative. That there will only be one xxxx-2.1 version of a SDK module ? Or would it become possible to have both 1.2.0-2.1 and 1.2.1-2.1 ? Such a case would lead to problems IMO.

gavinking commented 8 years ago

do we still consider that the module version (without the platform version) is unique.

Yes.

That means the platform version in only informative.

Right.

Such a case would lead to problems IMO.

Sure, but I don't see how that could arise.

Alternatively, you could just replace the - with a ..

davidfestal commented 8 years ago

Alternatively, you could just replace the - with a ..

This would only make it more difficult to separate both parts.

gavinking commented 8 years ago

@davidfestal I mean "you" as in your MANIFEST.MF generator.

davidfestal commented 8 years ago

well, not really I think, because OSGI versions are ordered (lexical order), and changing the platform version number would disturb the ordering of true SDK versions, which would lead to problems for automatic update and provisioning.

gavinking commented 8 years ago

Hrm, in fact, one might argue that instead of 1.2.2-1.1, the version number should be of form 1.2-1.1.2.

That is, if we assume that we have BC between minor releases of the compiler/platform, then the .2 bit belongs to the build of the module, not to the build of the compiler/platform. Interesting.

gavinking commented 8 years ago

because OSGI versions are ordered (lexical order), and changing the platform version number would disturb the ordering of true SDK versions

I know that. But assuming that module version to platform version is a many-to-one association, there's no problem, AFAICT.

davidfestal commented 8 years ago

ah, OK, that might work if a n+1 SDK version is always associated to a platform version that is greater or equal to the platform version of the n SDK version

thradec commented 8 years ago

I understand the reasons behind it, but as a user, I would be really annoyed with versions composed from 5 numbers.

2016-03-02 14:35 GMT+01:00 David Festal notifications@github.com:

ah, OK, that might work if a n+1 SDK version is always associated to a platform version that is greater or equal to the platform version of the n SDK version

— Reply to this email directly or view it on GitHub https://github.com/ceylon/ceylon-sdk/issues/520#issuecomment-191239856.

gavinking commented 8 years ago

That's what I'm assuming. I'm assuming we're not backporting module API changes to previous revisions of the platform. Perhaps that's an invalid assumption, but it seems likely to me.

gavinking commented 8 years ago

I understand the reasons behind it, but as a user, I would be really annoyed with versions composed from 5 numbers.

Autocompletion.

thradec commented 8 years ago

I understand the reasons behind it, but as a user, I would be really annoyed with versions composed from 5 numbers.

Autocompletion.

autocompletation doesn't help, when you have to figure out, why it stopped working and have to track complex dependency graph with many nodes

FroMage commented 8 years ago

I propose we copy the wildly successful version schemes for Android/Google (Guava), since they're so successful:

FroMage commented 8 years ago

Joking aside, I think the bigger question is: do we want all modules in the SDK to have their own version, or increase separately. Considering that separate increment is only possible by groups, since bumping one version of a module requires bumping the version of all modules that import it too, transitively, due to a current compiler limitation that prevents the same module from being visible twice to the compiler with different versions.