Add support for GNU ELPA Devel versioning scheme

[AkibAzmain]

This PR adds support for GNU ELPA Devel's version scheme, as @minad proposed in melpa/melpa#8135. This adds a new user option package-build-elpa-versioning-scheme. When it is non-nil, the new versioning scheme is used.

The version never uses the first three component of version and always keeps at least a 0 between the tag version and timestamp version. This (hopefully) won't create any problem if the tag changes from 1.2.3 to 1.2.3.4 or from 1.2.3.4.5 to 1.2.3.4.5.6.

I have tested it with one of my packages, and it works fine.

[tarsius]

FWIW Next week I intend to finally implement the versioning scheme that I mentioned in that same thread.

This (hopefully) won't create any problem

This is fine if you use it for your own purposes, but a project like Melpa would have to take a more careful approach. I don't think we should merge this, but we could add a wiki page where contributors could post their approaches and advertise their advantages.

[tarsius]

You probably should build on top of the work branch. It likely won't be merged any time soon and is certain to be rebased several times until then, so use with caution. It does however contain changes that make it more feasible to implement alternative versioning schemes without having to make changes to the existing code, so maybe you can just use it but then have to wait for it to be merged.

[tarsius]

Add support for GNU ELPA Devel versioning scheme

This doesn't really implement the "GNU ELPA Devel versioning scheme". As far as the format of of the version strings go, this implementation does that, I believe. But [Non]GNU Elpa versions are based on the [Package-]Version header keyword, not Git tags. So this should be named something else.

[AkibAzmain]

This (hopefully) won't create any problem

This is fine if you use it for your own purposes, but a project like Melpa would have to take a more careful approach.

I said that because it needs more testing, and I don't have enough resources to pull all the MELPA packages to build with it.

[AkibAzmain]

Add support for GNU ELPA Devel versioning scheme

This doesn't really implement the "GNU ELPA Devel versioning scheme".

Any ideas what to call this new thing?

[AkibAzmain]

This is fine if you use it for your own purposes

I simply don't need package-build. I use Quelpa, and will probably use the new package-vc in the future. I just tried to do this because I really want melpa-devel to come to existence, because MELPA(-unstable)'s versioning the root cause of so many stupid dependency declarations in packages (for example, package x wants package y version 20211231.1421, which is incompatible with all the other archives).

[purcell]

This is all quite interesting, but I'm a bit sad that for seven years nobody has wanted the new version scheme enough to act on #2955, after all the discussion and careful design that led up to it. I'm a bit wary of suddenly merging a different scheme tbh.

[tarsius]

I am working on it right now and I will try to open a pull-request and a few issues for discussion later today.

I know I have said many times that I would get to this eventually and then I ended up working on seemingly unrelated stuff instead. In actuality I have been working on this for several years now, in the sense that all work I have put into package-build.el was to prepare for this change. This work has had other benefits of course, but the primary goal always was to make it more feasible to implement reasonable version scheme.

Granted a lot of that work is not strictly necessary to implement another version scheme, but it helps. For example, Melpa is extremely slow when it comes to pulling updated packages compared to the equivalent tooling used for the Emacsmirror. This meant that when I had to test a change to package-build on all packages, I had to either wait half a day (in the ideal case when nothing failed) for the pulls to complete, or use the state from a few months ago, when I last ran other test. On package-build's work branch there now is a change that allows me to use my local checkout of the Emacsmirror instead of checkouts managed by package-build itself. I can now update all packages in about half an hour using emir and then I can use that up-to-date data from package-build.

As you have told those who occasionally showed up to suggest that we switch to a better version scheme, implementing some algorithm to come up with a version string is only a tiny part of the work required. All the work I have already put in is also required. Only because I have already put in that work was it so easy for @AkibAzmain to hook up his algorithm.

The additional required work that you usually emphasized is the need for comprehensive testing. We also have to document the used algorithm. And then we have to announce and justify the change, using documentation and several blog/reddit/mailing-list posts to ensure as many users as possible are aware of the change and that they need to take action.

Most importantly, we have to get it right on the first try (and this is the reason why I have opted to do all those preparatory steps first). A lot of users and package maintainers will be very happy about this change, but there will also be people who will be upset that version numbers decrease and that they have to manually intervene. We have to make sure they only have to do that once.

(Ideally we would teach package.el to handle the special-case of updating from timestamp version strings to smaller real version strings. But that would be a further delay, and because requests to switch to a different version scheme are picking up in frequency lately, I am now of the opinion that we should skip that step. Also, merely implement this special-case in package.el is not enough, we would also have to get the emacs developers to merge it and to make package.el available on GNU Elpa so that users of older (released) emacsen could also benefit.)

This may have sounded a bit defensive. It wasn't meant as a criticism, for you not being fully aware that I am actually working on it. I have probably prioritized wrongly, including not communicating clearly enough that I am working on it and that I am very close now.

[AkibAzmain]

@tarsius: Thanks for working on this! Feel free close this PR if you wish, you definitely have more knowledge about MELPA internals than me.

[tarsius]

Even though I am closing this pull-request now, it did have a small positive effect; I have moved this from position two on my todo list to number one. :grinning:

[purcell]

I am working on it right now

I didn't mean I was sad that you hadn't done it yet, @tarsius! 😂

But thanks for digging in, doing the prep work and generally working through a plan. ❤️

[tarsius]

I figured, but wanted to make sure.