unascribed / FlexVer

A SemVer-compatible intuitive comparator for free-form versioning strings as seen in the wild, designed to sort versions like people do.
Creative Commons Zero v1.0 Universal
65 stars 10 forks source link

Test vectors #3

Closed unascribed closed 1 year ago

unascribed commented 2 years ago

It would be useful for testing new implementations and verifying consistency between implementations to have a large series of test vectors, that any implementation can read and verify. For this purpose, I suggest a trivial plain text format, like the following, called something like test_vectors.txt in the root of the repo:

# Basic numeric ordering (lexical string sort fails these)
10 > 2
100 > 10

# Trivial common numerics
1.0 < 1.1
1.0 < 1.0.1
1.1 > 1.0.1

# SemVer compatibility
1.5 > 1.5-pre1
1.5 = 1.5+foobar

# SemVer incompatibility
1.5 < 1.5-2
1.5-pre10 > 1.5-pre2

# Optional features
## Too large for a 64-bit integer or double, checks if codepoint-wise or integer-parse is being used
36893488147419103232 <? 36893488147419103233

# ...and so on

We would also want weird Unicode edge cases in here, and tests for things like the leading-zero pitfall with integer-parse. (Open question: Does there exist a Unicode codepoint that sorts differently as UTF-16 and as UTF-32, when compared as outlined in the spec?)

It would likely be good to start this by moving the current test vectors out of being hardcoded in each language implementation.

A series of test vectors for decompositions may also be useful, such as:

1.0.1-pre2+foobar5 n1 t. n0 t. n1 p-pre n2 a+foobar n5