PB SemVer: Tanks, and Another Link Too

Hi @aziascreations,

thanks for publishing this project, and with such a permissive license. I was looking at your SemanticVersioning.pb and I'm very interested in it.

I didn't know that there was already some PB code to deal with SemVer, so I ended up drafting a version of my own for a project I was working on (but currently stalled due to lack of time and loss of motivation). But I'm surely going to need to look at your SemVer version soon.

I though you might like to know about the version I created:

https://github.com/tajmone/pb-archives-test/blob/source/_butler_/butler-src/butler-mod_dependencies.pbi

It comes as a module, and might need just a few tweaks to work anywhere else (see comments about cross-module Enums).

The module also validates a given SemVer against a constrained version, using some constrain operators:

; A module for checking that dependencies satisfy specific version ranges.
;
; Currently Supports:
; -- Numerical identifiers (only)
; -- Versions with two or more segments (ie: 1.0; 1.0.0; 1.0.0.0; etc.)
; -- Constraint operators:
;    -- "="  --- Strict:        Lock all segements (default)
;    -- "^"  --- Caret:         Lock MAJOR version
;    -- "~"  --- Tilde:         Lock MAJOR and MINOR version
;    -- "~>" --- Twiddle-Wakka: Lock all segements except rightmost

My module doesn't take in consideration pre-release versions when comparing, though (as this would have added to much complications for the time being).

The good thing is that version validation works also with non stricly SemVer version schemes (eg: 2.0.0.1) — it actually works with any number of segments as it will balance segments between the version to validate and the constrain.

I hope it might be useful to you also.

Hi @tajmone,

Thank you for your comments and suggestions, I'm glad you find my code interesting too.

I've looked at your code for a bit and it's actually quite nice, and the possiblity to "select" a specific range of versions with operators is really cool too.
I haven't implemented that in mine yet since it was mainly a quick test, which is clearly visible in some part of the module, and I think some procedure might be missing if my memory is correct.
I will try to rewrite a correct and more professional looking version as soon as I can.

By the way, I think the way you select/restrict version could be improved with operators like >/<, +/- and !. ;)
A good example for that would be @zafarkhaja's jsemver project.

I will try to implement some of them when I rewrite it too, it might be usefull to you if you end up using it.

Regarding version validation with non-semver versions (eg: 1.0.0.0), I would personally tend to reject them outright in the module since they can cause some confusion for devs/end users or cause some weird errors in some cases, but I can try to add some flags like #SEMVER_AUTOCONVERT/LAXMODE in procedures parameters to handle/process them if you think that would be more appropriate.

I hope my comment was also useful for you.

Indeed, I agree. I don't remember exactly how ended up making the version module, but I remember that initially I wanted to create it strictly for SemVer 2.0, but then for the project at hand I needed more flexibility for checking the version of a dependency that used another scheme. The idea is that it has different procedures by which you can interface with the various sub-procedures, and the SemVer procedures do (or did) validate for SemVer criteria first.

So, basically some procedures where left open to be reusable for other dependencies.

The original SemVer module I was working on (intended for SemVer only) is resting somewhere on my harddrive waiting to be polished, but some time has gone by and (at the time) I gave precedence to the project I was working on at the time and then didn't catch up with it.

Anyhow, I've published as Gists the original SemVer module and its test file:

https://gist.github.com/tajmone/f32b4c905f679c16205fecf0973f5af9

... but I'm not really sure how stable it was (it should be well commented though). The RegExs were thoroughly tested, both for loose and strict validation.

I hope the Gist files might be useful.

I'm definitely going to need to improve the SemVer module sometime in the future, and I'll happily look into your source for ispiration.

The tough part is going to be implementing the way SemVer decides pre-releases precedence (just many rules that need to be implemented there). Also, implementing support for range operators, or cumulative constraints (eg a range followed by an exception, etc.) is going to take quite some time (and testing).

The problem is that in the PB echosphere SemVer is not used a lot, so it really comes handy only when checking external dependencies (mainly binary, at the current moment). But there's definitely a need for a decent SemVer module/library for PB.

I have looked around a bit yesterday and I found @vdurmont's semver4j project and in it, there is the possibility to specify a version type which could be used to easily indicate how to parse and compare them.

For example we could have something like this in the module:

EnumerationBinary
    #TYPE_STRICT ; 1.0.0-abc+abc (nothing else)
    #TYPE_NPM ; 1.0.0-abc+abc (with range selectors)
    #TYPE_COCOAPODS ; Like the one you explained

    #TYPE_LOOSE_CONVERSION ; 1.2.3.4 -> 1.2.3
    ; Even more specific conversion types if needed.
EndEnumeration

; And you could compare them like that (The code would be a bit more complex with inits and version structures)
; > AreCompaptible(Ver1.s, Type1.i, Ver2.s, Type2.i)

AreCompaptible("1.0.0", #TYPE_STRICT, "1.0.5", #TYPE_STRICT) ; True
AreCompaptible("1.0.0", #TYPE_STRICT, "1.1.0", #TYPE_STRICT) ; False

AreCompaptible("1.0.0", #TYPE_STRICT, "1.0.5.0", #TYPE_STRICT) ; False (Would cause an error when initializing)
AreCompaptible("1.0.0", #TYPE_STRICT, "1.0.5.0", #TYPE_STRICT | TYPE_LOOSE_CONVERSION) ; True

AreCompaptible("1.0.0", #TYPE_STRICT, "1.*", #TYPE_STRICT) ; False (Would cause an error when initializing too)
AreCompaptible("1.0.0", #TYPE_STRICT, "1.*", #TYPE_NPM) ; True

It would still allow you to use the "industry standard" of %MAJOR%.%MINOR%.%BUILD%.%COMPILE% and compare them with Semver ones without having to do anything before.

The hardest part of this will be the implementation of range operators like you said, but I think it could actually be easier than it seems.
The more complex ones(eg:~1.3 | (1.4.* & !=1.4.5) | ~2) can easily be separated and checked independently which shouldn't add too much code and complexity in the end if it is done correctly.

I quickly looked at your gist too, and it seems a bit more complex than the code you previously linked, I will try to take a better look at it when I have some free time.
But by the looks of it, there could be some good ideas in there.

The hardest part of this will be the implementation of range operators like you said, but I think it could actually be easier than it seems.

The major problem here is also that SemVer 2.0 doesn't actually set any standards nor provide any guidelines when it comes to constraints operators and patterns, it only provides the rules of how precedence should be calculated (see below).

So, there are actually different standards into the wild when it comes to how version ranges and constraint operators are implemented. Most notably, the predominant stanards are set by Ruby, Node.js/NPM, Composer:

Ruby:
- http://guides.rubygems.org/patterns/#pessimistic-version-constraint
Node.js/NPM:
- https://github.com/npm/node-semver
Composer:
- https://getcomposer.org/doc/01-basic-usage.md#package-versions

This online tool is good for testing ranges and constraints, and also documents differences across the major implementations:

http://jubianchi.github.io/semver-check/

At the end of day, some choices have to be made on how to handle these. At the time I wrote the module, I was a bit in a hurry to include it in my project, so I ended up skipping the full monty and just adapt it to my strict needs. But I realized that some thoughts need to go into this before choosing which approach to take in ranges and operators. For one thing, PureBasic has more freedom of choice here, unlike other languages or packages which have some pre-existing standards to which they have to adapt any new feature they take on (PB is currently so detached from the world of standards that there's basically unlimited freedom of choice here).

Constraint operators are just syntactic sugar for ranges, but if one starts to create complex declaration (for example, allowing to specify a range and then exclude a set of specific versions from it) then all syntaxes variations matter.

NPM might be a good starting point; their system is well documented and even provides a BNF grammar for it:

range-set  ::= range ( logical-or range ) *
logical-or ::= ( ' ' ) * '||' ( ' ' ) *
range      ::= hyphen | simple ( ' ' simple ) * | ''
hyphen     ::= partial ' - ' partial
simple     ::= primitive | partial | tilde | caret
primitive  ::= ( '<' | '>' | '>=' | '<=' | '=' | ) partial
partial    ::= xr ( '.' xr ( '.' xr qualifier ? )? )?
xr         ::= 'x' | 'X' | '*' | nr
nr         ::= '0' | ['1'-'9'] ( ['0'-'9'] ) *
tilde      ::= '~' partial
caret      ::= '^' partial
qualifier  ::= ( '-' pre )? ( '+' build )?
pre        ::= parts
build      ::= parts
parts      ::= part ( '.' part ) *
part       ::= nr | [-0-9A-Za-z]+

Anyhow, I don't think is going to be so quick to implement it, test it, and document it. But it's well worthy doing it. In my opinion, I'd create a module that handles both SemVer and other version schemes too (eg: vX.Y.W.Z and X.ZZ, as in v1.0.3.4 and 5.60), offering some SemVer specific procedures and other more general (and customizalbe) version comparison procedures. The same code can be reused in different systems, and it would be easier to maintain this way. Also, when working with external dependencies, SemVer is mostly used for API, while binary tools will use one of the other schemes, or variants thereof.

Precedence Rules Problems

I also have a vague recollection that there were some problems with implementing SemVer 2.0 precedence rules. Some time has passed, so my memory is not fresh on the issue (I might have jotted down a few notes somewere), but if I remember correctly it had to do with ASCII sorting non-numeric identifiers.

I must have done some tests with the SemVer online tools and come to the conclusion that PB's native string comparison and sorting functionality doesn't actually fit the requirements. As I sayd, I'm not 100% sure on this, but there was some problem with implementing precedence rules that forced me to postpone its completion to a future time where I could dedicate more energy into it.

Unfortunately there is often confusion between alhphabetic sorting and Ascii sorting, and one only has to look how the various DIR and ls commands will produce differently sorted results for the same folder to realize that ASCII sorting implementations are often opinionated and differ in details. My understanding is that in SemVer non-numerical sorting has to follow strictly the ASCII precedence, with shorter strings comming first — maybe PB handles it different, not enforcing the string length in comparison.

I don't remember, but it has to be cross checked against some officially blessed SemVer tool to make sure it's done right.

The Precedence Rules

https://semver.org/#spec-item-11

Precedence refers to how versions are compared to each other when ordered.

Precedence MUST be calculated by separating the version into major, minor, patch and pre-release identifiers in that order (Build metadata does not figure into precedence).
Precedence is determined by the first difference when comparing each of these identifiers from left to right as follows:
- Major, minor, and patch versions are always compared numerically. Example:
```
1.0.0 < 2.0.0 < 2.1.0 < 2.1.1
```
- When major, minor, and patch are equal, a pre-release version has lower precedence than a normal version. Example:
```
1.0.0-alpha < 1.0.0
```
- Precedence for two pre-release versions with the same major, minor, and patch version MUST be determined by comparing each dot separated identifier from left to right until a difference is found as follows:
  - identifiers consisting of only digits are compared numerically and identifiers with letters or hyphens are compared lexically in ASCII sort order.
  - Numeric identifiers always have lower precedence than non-numeric identifiers.
  - A larger set of pre-release fields has a higher precedence than a smaller set, if all of the preceding identifiers are equal. Example:
```
1.0.0-alpha < 1.0.0-alpha.1 < 1.0.0-alpha.beta < 1.0.0-beta
< 1.0.0-beta.2 < 1.0.0-beta.11 < 1.0.0-rc.1 < 1.0.0
```

Hey, I'm sorry for the late response, I've been a little busy and I wanted to take the time to read what you linked to avoid making mistakes.

I looked at the problem with precedence that you talked about, and I think it is possible to organize it in a way that would make the implementation a bit easier to work with and modular:

In a first module, you could put the versioning structure and every basic function that would let you calculate precedence. And in a second module, that requires the first one, you could implement a specific standard and all of it's related procedures (Ruby, npm, ...). I will post an example soon so you can understand more easily.

It would also allow you to easily support other versioning standards without bloating the source code and leaving a ton of unused code in the executable since according to some sources, the compiler seems to leave them in there.^[1]

Regarding the problem with string comparaison in purebasic, I wasn't even aware that this could be a problem, I will try to play around with that when I have some free time to try and pinpoint some of it's quirks that could cause problems down the line.

Appart from these problems and probably a few others later, it will take time to implement everything correctly but it is definitively possible.

By the way, I wanted to thank you for the amount of information you provided, I was not expecting that much and was a bit overwhelmed to be honest, but it will really come in handy when we will work on it.

Hey, I'm sorry for the late response

... no time problems on this one. As I said, it's something I had worked on in the past, but didn't manage to finish it because of time constraints. But I definitely would like to see a good version validation module available in the future. Ideally, I'd like to include it in this project:

https://github.com/SicroAtGit/PureBasic-CodeArchiv-Rebirth

PBCompiler Optimization of Unused Code

according to some sources, the compiler seems to leave them in there.

I've read that too in the past, but when I actually put it to the test I've seen that unused code doesn't get compiled — but I didn't test with modules, though. I've done the tests the easy way: using /COMMENTED compiler option to generate the commented Assembly source file you can then look into it and see what was actually compiled and what was left out. Left out parts will be visible in the commented PB source, but will have no ASM code associated to it.

It's always worth checking the actual output, but in general PB does some basic optimizations now, and unused code shoul be left out from the final executable. I'm not sure if modules behave the same, it needs to be checked — but this is worth checking for with each individual project anyhow. This IDE tool simplifies the task of converting PB source to ASM:

https://github.com/tajmone/PBasmUI

Keeping This Thread Open

Appart from these problems and probably a few others later, it will take time to implement everything correctly but it is definitively possible.

Let's keep each other up to date on the progress via this thread. Probably, the first one of us who needs to use SemVer in a project is the one going to take on the task first. Whichever way, if we notify each other of the progresses, than we can join forces and eventually achieve the goal.

Right now I'm buried in coding with three projects I'm struggling to move forward. But SemVer is definitely in my wishlist and I'd like to contribute to it sooner or later.

Create a PB-SemVer Playgroun on GitHub?

We might even think of opening a dedicated repository for this project — just an Alpah-Dev testing ground where we can gather all the code snippets we've both worked on, and use the Wiki to annotate the various problems, set a roadmap to decided all the minimum features the module will have to cover to be declared stable release, and add links to external resources. No obbligations, no deadlines, just a joint effort in sharing resources. What you think? I could set the repo up and add you as collaborator, with full access rights.

I think it's a good idea, I've used this approach in the past and the good side of it is that you receive notification when other collaborators update something, so it's also a reminder and incentive. Also, on GitHub you can create all the alpha dev repositories you want, and even if you abbandon the project you can just archive it and leave it there, maybe others could find it useful and carry on from where you left.

By the way, I wanted to thank you for the amount of information you provided, I was not expecting that much and was a bit overwhelmed to be honest, but it will really come in handy when we will work on it.

It's a pleasure, when I saw that you created this nice repository for sharing with other your code I felt that the minimum I could do was to share with you similar code which is either out there in the wild or sitting on my hard drive. The PB community is not all that huge, and its presence on GitHub is even smaller, so I think is good to keep an eye on each other's works and interests and try to communicate more and build a network of contacts and collaboration.

aziascreations / PB-Utils