Require Semantic Versioning ("Semver")

[ozra]

Nimble should require packages to follow sem versioning.

This way all dependency version matching is much simpler and more reliable. (Pseudo:)

require:
    jester @ 1.*  # any version, new features too, as long as major doesn't change (breaking change)
    funkyMod @ 2.3.*   # Don't upgrade to new features, only patches, to avoid possible overload probs

[ozra]

@dom96, what's your stance on this? I think it would benefit the Nim eco system greatly if this was a requirement as early as possible (now ;) - makes reasoning on dep's so much easier for everyone.

http://semver.org

[dom96]

Why a requirement? I think this can be implemented as an optional feature.

[ozra]

Yeah, the thing is, it doesn't even have to be implemented really, it's more of a social norm thing of the package eco system. If it's the norm, then you know you can always rely on the major,minor,patch for version-matching. I don't see a good reason to explicitly use another versioning method. And an ad-hoc one is no good. So why not stick to one that is getting more and more widespread?

Creating a sort of, dependency harmony ;)

[Araq]

I don't understand this discussion. A tool cannot check that you actually stick to semantic versioning so all you can do is to document that you should do it. What am I missing?

[ozra]

Exactly. It's just a matter of official policy:

"How to publish your own modules"

1. Use semantic versioning (http://semver.org) when publishing modules in the nimble eco system - everyone will be happier (you to!).
2. ...

So, that's all I'm hoping for.

[dom96]

It's not just a matter of policy. Nimble needs to be aware of semantic versioning.

[ozra]

Ah, yes damn it, you're right - for the matching and ranges.

For simple version expressions "modname @ 1.2". Any excluded version parts are implicitly * (any) (this example would include "1.2.999999" for instance, and could be written explicitly as "1.2.*").

When it comes to lt/gt comparisons etc. the important thing is also there that omitted version parts are implicitly accounted for. So "modname <= 1" means "<= 1.." (for example "1.9999.99999" is included in that semantic range).

[dom96]

Perhaps a ~> would be good: http://yehudakatz.com/2010/08/21/using-considered-harmful-or-whats-wrong-with/

[Araq]

I prefer to change the semantics of >= so that >= means ~>. Good defaults are the way to go. For strict "greater than or equals" we can introduce >=! But I don't mind either way since I don't believe in version checks really. Nim has compiles and declared that should be used IMHO. Test for features, not for versions.

[Araq]

Hrm, I think I changed my mind, and people should just use pkg >= 2.0.2 && < 3.0. And in fact, > should be removed cause it makes no sense.

[dom96]

Why should > be removed? Why doesn't it make sense in Nimble but it does in Nim?

[dom96]

Something else we could support: https://twitter.com/rachelcolby11/status/676843156957061121

[endragor]

Why isn't there still a policy regarding how people should version their packages in Nimble? Semver couldn't be verified, of course, but at least it should be mentioned in the README that following semver is expected. Right now the README mentions this, which doesn't make much sense:

Similarly you should not specify an upper-bound [for a version requirement]

If you don't specify an upper bound, a breaking change in the dependency will break your package, so it rather should be an exception to NOT specify an upper-bound.

Versioning policy is about culture - it doesn't require too much from implementation standpoint, but it lets everyone know what version number really means for packages managed by the package manager.

Formally, >= and < operators already allow to follow semver in dependency requirements, but since a requirement like "somePkg >= 1.1.0 & < 2.0.0" is a common one, a shortcut would be nice. In other systems (Rust's Cargo or Node's NPM) there are ^ and ~ prefixes that work as such shortcut. More details here: http://doc.crates.io/specifying-dependencies.html (~> in Ruby gems is equivalent for tilde in those). In such systems people usually specify dependency versions with caret prefix, which means "I'm OK with any version of the package starting from this one which doesn't have breaking changes". A more strict/safe way is to use ~, which only allows minor version updates, which usually stand for bugfixes.

[dom96]

Similarly you should not specify an upper-bound [for a version requirement]

If you don't specify an upper bound, a breaking change in the dependency will break your package, so it rather should be an exception to NOT specify an upper-bound.

There is a reason for this. Specifying an upper bound limits the number of versions of the dependent package that can be used. Consider two packages, package Foo which depends on "jester >= 1.1.0 & < 2.0.0" and package Bar which depends on "jester >= 2.1.0 & < 3.0.0". Your application depends on Foo and Bar, satisfying these dependencies is impossible because you cannot import two different versions of Jester.

[endragor]

@dom96 I understand that, but it still doesn't justify the suggestion. If the dependency tree has conflicting requirements, let the package author resolve them, there is no other way around that. Failing fast is better than getting a package that doesn't work at certain point of time without its author knowing (when a new dependency version got breaking change).

[dom96]

I disagree. Failing because a library cannot compile due to a change in API is better than failing because the versions don't match. In the latter case the packages may still work together.

I also don't think that resolving the issue I described is easy. How do you resolve it? Tell the maintainer of package Foo to update their package's dependencies? what if they can't?

[endragor]

Failing because a library cannot compile due to a change in API is better

The problem is that it fails at undefined point of time, it's uncontrolled by the author of the package. When it gets broken, anyone using the package will be blocked and it's going to be impossible to resolve. Imagine that a package used by hundreds of other packages and applications suddenly gets broken, and there is no way to fix that for anyone (except to stop using that package and perhaps Nimble altogether, because it's going to be common for Nimble if it keeps such policy).

I also don't think that resolving the issue I described is easy.

It doesn't matter too much here if it's easy or not to resolve that issue, the important thing is that resolving it is in the hands of the package author and they know the issue prior to publishing the package. In the worst case, depending on the license, they may temporarily copy-paste the conflicting package and make it use the newer/older version.

My main point is that when the same version of an already published package stops working, it is the most serious issue that can occur when dealing with package managers, and the versioning/dependency policy should not promote that.

I took a quick glance at Rust/Cargo discussions and it seems to allow conflicts in certain cases - when there is no actual type/proc sharing happens outside of packages that create conflicts. Not entirely sure how it does that (seems Rust compiler does some verification, not Cargo), but such approach would resolve many issues with dependency conflicts. Perhaps the package author itself could define whether the dependency is internal or exposed - if it's internal, it should be fine to duplicate it and let different packages use different versions of it.

Java's Maven and its derivatives allow to override dependency version of your own dependency, so it also gives some control to the package author for resolving dependency conflicts.

In general, dependency conflicts is one of the core issues of any package/dependency manager, and it requires deep thinking, but letting published packages break easily is definitely not a solution.

[endragor]

By the way, Facebook announced Yarn - a new package manager for JS a couple of days ago, and it got a lot of hype. One of the main selling points of Yarn is this:

From the get-go, the Yarn lockfile guarantees that repeatedly running yarn on the same repository results in the same packages.

(Cargo has this feature, too)

So you can imagine how people get annoyed because of unreliable builds, if they are ready to make a switch from an established package manager with the largest community in the world and years of work put in. And that makes sense - you want your application to be exactly the same when built, regardless of when or where you build it. If one thing gets built on dev machine, and another on CI agent, then you get unexpected behaviour. Even worse if CI agent builds one thing today, and another thing tomorrow (or fails to build tomorrow) without any changes in the app's repository.

[dom96]

https://github.com/nim-lang/nimble/issues/127

On Thursday, 13 October 2016, Ruslan Mustakov notifications@github.com wrote:

By the way, Facebook announced Yarn https://code.facebook.com/posts/1840075619545360/yarn-a-new-package-manager-for-javascript/

• a new package manager for JS a couple of days ago, and it got a lot of hype. One of the main selling points of Yarn is this:

From the get-go, the Yarn lockfile guarantees that repeatedly running yarn on the same repository results in the same packages.

(Cargo has this feature, too, by the way)

So you can imagine how people get pissed of because of unreliable builds, if they are ready to make a switch from an established package manager with the largest community in the world and years of work put in. And that makes sense - you want your application to be exactly the same when built, regardless of when or where you build it. If one thing gets built on dev machine, and another on CI agent, then you get unexpected behaviour. Even worse if CI agent builds one thing today, and another thing tomorrow (or fails to build tomorrow) without any changes in the app's repository.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nim-lang/nimble/issues/130#issuecomment-253411836, or mute the thread https://github.com/notifications/unsubscribe-auth/AAPDe1SMTQ3Kgy3LFpX0XtzEZwMRo415ks5qzbO4gaJpZM4EsQrw .

[dom96]

There is a good point made here: https://news.ycombinator.com/item?id=13378637

1. Software using Semantic Versioning MUST declare a public API.

We need to keep this in mind. Not all Nim packages declare a public API.

[endragor]

Could you elaborate? I think it's usually applications that don't declare API and semver doesn't make much sense for them, but libraries intended to be used by others by definition do declare public APIs. And packages in a package manager are usually libraries and frameworks.

[dom96]

Yes, my point is that not all packages should be forced to use semver.

[binaryben]

I'm slightly horrified semversioning isn't enforced. Especially in the context that I came to report nimble check is providing the following error to me: "Error: Version may only consist of numbers and the '.' character but found '-'."

A tool cannot check that you actually stick to semantic versioning

So clearly, this line of reasoning is wrong. A tool is indeed checking the version...

Yes, my point is that not all packages should be forced to use semver.

I disagree mostly, although I understand some may want to use release date or similar as a version. Even then though, that can be incorporated into semver. Examples in the wild of requiring SemVer:

• As already mentioned, Node requires it
• Although a version appears to be optional, PIP uses semver when a version is provided
• Docker does almost, but separates out the tag

Obviously there are others and this is becoming more common 7 years after the OP simply because it works in terms of managing dependencies.

If someone wants to call their package the-night-rabbit-feasts though ... who am I to judge? 🤷🏼‍♂️

---

Anyway, rant aside. In the spirit of not enforcing any particular versioning scheme, can semver at least be an option? 1.20221026.123-beta and 1.0.0-x.7.z.92 are valid semver versions and should pass the check. Happy to open a new ticket instead if needed.