Crate versioning

[anderslanglands]

Currently we're just tracking the OpenEXR major/minor version and assuming we'll bump patch as necessary. This is going to be pretty terrible because it means we can't iterate on the safe API - we'd have to wait for OpenEXR 4 to be released to make any changes.

One kinda gnarly solution to this would be to multiply the openexr version by 10, then use the units for the crate version. This is a horrible abuse of semver, but I think it would actually work.

e.g. at the moment we're tracking OpenEXR 3.0.1, which in this scheme would make our crate version 30.00.10. We then decide we need to make changes to the safe API, so we publish a new crate with version 31.00.10. Then say we make a coupe of minor changes, it becomes 31.02.10. Then when we add a publish for OpenEXR 3.1.0, our crate version becomes 31.12.00.

It does require a bit of stopping and doing a bit of math to figure out the version parts... but it does mean we can make API changes without breaking semver (and I haven't thought of another way yet).

[scott-wilson]

I was thinking about this a bit. One direction is we can say that the major and minor version will sync with OpenEXR, but the patch may not.

Either that, or don't make any guarantee that the version will match, and make it very easy to tell which version of the crate is compatible with which version of OpenEXR. Such as in the readme, and etc.

[anderslanglands]

The first option is what we’re doing currently, which is not great because it means we can’t make API changes.

The second could get incredibly confusing - we’d need a big table in the readme that people would need to refer to.

On Mon, 21 Jun 2021 at 03:15, Scott Wilson @.***> wrote:

I was thinking about this a bit. One direction is we can say that the major and minor version will sync with OpenEXR, but the patch may not.

Either that, or don't make any guarantee that the version will match, and make it very easy to tell which version of the crate is compatible with which version of OpenEXR. Such as in the readme, and etc.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/vfx-rs/openexr-bind/issues/36#issuecomment-864569591, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOYQXJO6QNX3KTOR3ZKXKDTTYAYLANCNFSM4675FODQ .

[anderslanglands]

I've added a bit about this to the readme on the docs branch: https://github.com/vfx-rs/openexr-bind/blob/eb108030ed6b6073ce7e1531151bd333e590e3fd/openexr-rs/src/lib.rs#L85-L101

[virtualritz]

I saw at least one crate that used x.y.z-a.b.c. Where x.y.z is the crate version and a.b.c is the lib version the crate wraps. Can't recall which crate this was though.

[anderslanglands]

Yes that puts it into the name nicely but unfortunately everything after X.Y.Z is ignored by cargo for version resolution.

On Wed, 30 Jun 2021 at 04:00, Moritz Mœller @.***> wrote:

I saw at least one crate that used X.Y.Z-A.B.C. Where X.Y.Z is the crate version and A.B.C is the lib version the crate wraps. Can't recall which crate this was though.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/vfx-rs/openexr-bind/issues/36#issuecomment-870723574, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOYQXKHB4QT3ZKVSUZMVL3TVHUZZANCNFSM4675FODQ .

[virtualritz]

Ok, but I don't see how that is an issue? Maybe I miss something?

[scott-wilson]

I realized this morning that we might not want to commit to a version number above 0 for now.

Reason is if the OpenEXR project takes ownership of this, then we'd probably want to make sure the version number is in sync with C++ releases. Or, they might have an opinion that we'd need to support.

[anderslanglands]

Ok, but I don't see how that is an issue? Maybe I miss something?

I'm trying to avoid crate versions being arbitrarily numbered. Say we've got 1.0.0-3.0.0 published, then OpenEXR-4 is released. Now we've got multiple versions of the crate leap-frogging each other for version numbers. Does the OpenEXR-3 crate get to be version 2, (2.0.0-3.0.0 and 3.0.0-4.0.0) or does the OpenEXR-4 crate (2.0.0-4.0.0 and 3.0.0-3.0.0)?

[anderslanglands]

I realized this morning that we might not want to commit to a version number above 0 for now.

Reason is if the OpenEXR project takes ownership of this, then we'd probably want to make sure the version number is in sync with C++ releases. Or, they might have an opinion that we'd need to support.

We have to have numbers above 0 or we won't be able to support multiple major versions of a library, like not supporting OpenEXR 2 and 3 at the same time.

[scott-wilson]

I realized this morning that we might not want to commit to a version number above 0 for now. Reason is if the OpenEXR project takes ownership of this, then we'd probably want to make sure the version number is in sync with C++ releases. Or, they might have an opinion that we'd need to support.

We have to have numbers above 0 or we won't be able to support multiple major versions of a library, like not supporting OpenEXR 2 and 3 at the same time.

That's true, but my argument is that we might not be the ones to pick the version number in the end. So, a version number will be picked in the future, but that may be in complete lock step with the OpenEXR project, or not. I think we need to talk with the project maintainers before we make a decision.

[virtualritz]

I'm trying to avoid crate versions being arbitrarily numbered. Say we've got 1.0.0-3.0.0 published, then OpenEXR-4 is released. Now we've got multiple versions of the crate leap-frogging each other for version numbers. Does the OpenEXR-3 crate get to be version 2, (2.0.0-3.0.0 and 3.0.0-4.0.0) or does the OpenEXR-4 crate (2.0.0-4.0.0 and 3.0.0-3.0.0)?

I do not get the problem. What is the issue with making a rule and sticking to it? I.e. crate 1.x will bind to OpenEXR 3.x. 2.x binds to OpenEXR 4.x. Basta.

Leap frogging or not leap frogging is a decision of whoever sets the version number. Simply do not leap frog.

[anderslanglands]

I'm trying to avoid crate versions being arbitrarily numbered. Say we've got 1.0.0-3.0.0 published, then OpenEXR-4 is released. Now we've got multiple versions of the crate leap-frogging each other for version numbers. Does the OpenEXR-3 crate get to be version 2, (2.0.0-3.0.0 and 3.0.0-4.0.0) or does the OpenEXR-4 crate (2.0.0-4.0.0 and 3.0.0-3.0.0)?

I do not get the problem. What is the issue with making a rule and sticking to it? I.e. crate 1.x will bind to OpenEXR 3.x. 2.x binds to OpenEXR 4.x. Basta.

Leap frogging or not leap frogging is a decision of whoever sets the version number. Simply do not leap frog.

You have to leap-frog. If we make a series of API changes to the Rust bindings then we'll have:

1.x -> 3.x
2.x -> 4.x
3.x -> 3.x
4.x -> 4.x
5.x -> 3.x
6.x -> 4.x

It gets worse if we make API changes on API that's only there in a specific major version as well...

1.x -> 3.x
2.x -> 4.x
3.x -> 4.x (4-only Rust API change)
4.x -> 4.x (4-only Rust API change)
5.x -> 3.x
6.x -> 4.x

[anderslanglands]

@virtualritz thinking about it some more I guess it doesn't really matter that much. Might be easier to just stick to X.Y.Z-A.B.C for now at least.

@scott-wilson yeah I agree we should consider that. The trouble is we can't promise to match major versions. If there's a soundness bug that requires an API change, we have to bump the major version regardless of what anyone prefers the version numbers to be and we can't wait for a new major version of the library to do that.

[scott-wilson]

@virtualritz thinking about it some more I guess it doesn't really matter that much. Might be easier to just stick to X.Y.Z-A.B.C for now at least.

@scott-wilson yeah I agree we should consider that. The trouble is we can't promise to match major versions. If there's a soundness bug that requires an API change, we have to bump the major version regardless of what anyone prefers the version numbers to be and we can't wait for a new major version of the library to do that.

Yeah, that's fair. Probably should come up with a list of topics for OpenEXR and invite them to chat

[anderslanglands]

Just to summarise what our options are for the OpenEXR group.

It's important for context to remember that the Rust bindings aren't just a straight translation of the C++ API - Rust imposes extra rules that should be accounted for. In particular there are several APIs in OpenEXR that should really be marked unsafe in Rust (e.g. Slice), and we'll want to iterate on these to find the best safe solution, which means making API-breaking changes faster than OpenEXR itself is likely to bump its major version. Moreover semver is supposed to be a strong promise in Rust as it's used by Cargo, the package manager, for resolving dependency trees.

In the below, X.Y.Z is the C++ library version, and A.B.C is the Rust wrapper version.

Versioning Scheme	Pros	Cons
X.Y.C Match OpenEXR major/minor, allow patch to increment independently e.g. 3.1.3 would be C++ version 3.1.0 and some Rust version.	Major/minor always matches C++	Have to wait for C++ library to bump version to make API changes, which could be a very long time.
A.B.C-X.Y.Z Encode the C++ version in the pre-release part of the version e.g. 1.2.3-3.1.0 would be C++ version 3.1.0 and Rust version 1.2.3	Separates the C++ and Rust versions cleanly	Breaks cargo's dependency resolution as pre-releases can never be automatically selected, you can only pin to a specific version cargo add and cargo update won't work Impossible to find the latest crate version you want to match a specific version of OpenEXR without looking at the list of published versions
XA.YB.ZC Encode the C++ version and Rust version together by multiplying the C++ version by 10 and adding the Rust version as unit e.g. 31.12.03 would be C++ version 3.1.0 and Rust version 1.2.3	Cargo dependency resolution and cargo add and cargo update works as expected Can derive both parts' versions just by looking at the number	Still have to look up major version (is it 30, 31, 32 etc?) A bit weird at first glance

[davvid]

Have to wait for C++ library to bump version to make API changes, which could be a very long time

Since API removals are what strictly require major bumps then this pain is only slightly diminished if it's feasible to maintain older APIs around alongside the iterative additions. That may be a very big if, though, and we already know we want to iterate on certain APIs so this con may have a lot of weight.

Still have to look up major version (is it 30, 31, 32 etc?) A bit weird at first glance

Based on the table this seems like the one w/ the least number of cons IMO. 👍

An alternative that prevents clashes for when we get to patch or minor > 10 would be to multiply by 100. 312.1434.1556 for c++ version 3.14.15 and rust version 12.34.56.

Slightly more complex, but maybe convenient for the C++-centric users ~ XYA.B.ZC 31412.34.1556 for c++ version 3.14.15 and rust version 12.34.56. Promote the C++ minor into the major version, let rust fully own the minor version, and multiply the concatenated c++ minor+major and patch by 100.

314xx tells us that it's C++ version 3.14. xxx12.13 in the remainder of the major and minor tells us that it's rust 12.13. The patch is nice and segmented as well.

Concatenated major+minor or not, 100 instead of 10 might be good to allow for more than 9 before overlapping.

[tiago-lqt]

I understand the appeal of having a flexible versioning method and also understand the apprehension of picking a method that will limit how we implement and iterate on the Rust side, but it seems its being done at the expense of other factors that will become more relevant once this project development pace meshes with the OpenEXR project itself.

In an ideal world we'd launch, say, version 4.1.0 a couple days or weeks after OpenEXR version 4.1.0 is released. And crate consumers could easily check in crates.io, or other related Rust sites, such as docs.rs, that theres a 4.1.0 crate available. Or even go to this repo and see that theres a 4.1.0 tag. All this to say that with semver theres an implicit expectation that Rust crate versions and the versions of the project itself will match to some degree. Otherwise we risk to both create unnecessary friction with package consumers and also burden ourselves with maintaining this mapping from "uuid" version to Rust + CPP versions. Important to say that package consumers won't be just us with knowledge of the whole development pipeline, or the vfx studios with people dedicated to package management, but everyone that the ASWF is trying to reach and many more, who likely don't need the added complexity of decoding the right version. Of course we're not in an ideal world, but I'm still not 100% convinced we need to take an alternative approach to well established semver.

It's important for context to remember that the Rust bindings aren't just a straight translation of the C++ API - Rust imposes extra rules that should be accounted for. In particular there are several APIs in OpenEXR that should really be marked unsafe in Rust (e.g. Slice), and we'll want to iterate on these to find the best safe solution, which means making API-breaking changes faster than OpenEXR itself is likely to bump its major version.

I agree that we're on the start of the journey, and there's still lots of iteration to be done to make this crate a first class one, but after we settle on a good solution, is it really beneficial to keep iterating? Seems to me there will be a point where stability is more valuable than improvements to the Rust API.

Taking the example above, say we do a good job with 4.0.x with a solid Rust API and OpenEXR launches 4.1.0. Even following semver we can iterate on 4.1.0-beta.x packages before releasing a 4.1.x package. Once the 4.1.0 Rust package is released do we really want to iterate again and tell consumers "hey, we changed the API again!"?

[anderslanglands]

Hi Tiago, thanks for your comments, you make a strong argument.

I think I worded my original post poorly. My concern is less about freedom to iterate and more about what if we discover an unsoundness issue that requires an API change to fix?

Our options are either don’t fix it till the next major release, or force a breaking change into a patch release, neither of which are particularly appealing.

On Tue, 6 Jul 2021 at 15:52, Tiago Carvalho @.***> wrote:

I understand the appeal of having a flexible versioning method and also understand the apprehension of picking a method that will limit how we implement and iterate on the Rust side, but it seems its being done at the expense of other factors that will become more relevant once this project development pace meshes with the OpenEXR project itself.

In an ideal world we'd launch, say, version 4.1.0 a couple days or weeks after OpenEXR version 4.1.0 is released. And crate consumers could easily check in crates.io, or other related Rust sites, such as docs.rs, that theres a 4.1.0 crate available. Or even go to this repo and see that theres a 4.1.0 tag. All this to say that with semver theres an implicit expectation that Rust crate versions and the versions of the project itself will match to some degree. Otherwise we risk to both create unnecessary friction with package consumers and also burden ourselves with maintaining this mapping from "uuid" version to Rust + CPP versions. Important to say that package consumers won't be just us with knowledge of the whole development pipeline, or the vfx studios with people dedicated to package management, but everyone that the ASWF is trying to reach and many more, who likely don't need the added complexity of decoding the right version. Of course we're not in an ideal world, but I'm still not 100% convinced we need to take an alternative approach to well established semver.

It's important for context to remember that the Rust bindings aren't just a straight translation of the C++ API - Rust imposes extra rules that should be accounted for. In particular there are several APIs in OpenEXR that should really be marked unsafe in Rust (e.g. Slice), and we'll want to iterate on these to find the best safe solution, which means making API-breaking changes faster than OpenEXR itself is likely to bump its major version.

I agree that we're on the start of the journey, and there's still lots of iteration to be done to make this crate a first class one, but after we settle on a good solution, is it really beneficial to keep iterating? Seems to me there will be a point where stability is more valuable than improvements to the Rust API.

Taking the example above, say we do a good job with 4.0.x with a solid Rust API and OpenEXR launches 4.1.0. Even following semver we can iterate on 4.1.0-beta.x packages before releasing a 4.1.x package. Once the 4.1.0 Rust package is released do we really want to iterate again and tell consumers "hey, we changed the API again!"?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/vfx-rs/openexr-bind/issues/36#issuecomment-874438540, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOYQXORWBSFERCPPDJNJ2DTWJ4YVANCNFSM4675FODQ .