Proposal: Improve Cargo dependency format

[wllenyj]

The crates in rust-vmm use >= comparison-requirements to specify specific dependencies. This places a significant maintenance burden on the user who uses it.

Problem

We use vm-memory as an example because the structure of vm-memory runs through the vmm ecosystem and cannot use two different versions. See version-incompatibility-hazards for reasons.

1. When I develop a library my-lib 0.1.0, it depends on linux-loader and vm-memory, and linux-loader depends on vm-memory >= 0.6. The version of my-lib that depends on vm-memory requires:
   • If use default Caret requirements, then you can only use the latest 0.8.0, not older versions.
   • Also use >=, Finally, it will also use the latest 0.8.0.
2. When an incompatible version of vm-memory 0.9.0 is released, it will be a disaster. Since cargo's strategy is to use the greatest version. Then, all libraries that depend on vm-memory >= must be fixed and released a new version. And the previous my-lib 0.1.0 will not work. This break is silent.
3. If we don't want to do an update. Can lock the old version for binary crates. But for libraries, we need to use some hacks. e.g. cargo update -p vm-memory:0.9.0 --precise 0.8.0. And we have to lock the linux-loader as well. But when my-lib is released, it still can't work.

Solution

According to cargo recommendations:

Use caret requirements for dependencies, such as 1.2.3, for most situations. This ensures that the resolver can be maximally flexible in choosing a version while maintaining build compatibility.

And most open sources also use the SemVer version requirement(x.y.z), which is equivalent to Caret requirements(^x.y.z). Mentioned in document specifying-dependencies:

It actually specifies a range of versions and allows SemVer compatible updates. An update is allowed if the new version number does not modify the left-most non-zero digit in the major, minor, patch grouping.

The SemVer version requirement can solve the problems mentioned earlier.

1. We can specify any version, as they all have a corresponding compatibility range. e.g. linux-loader 0.5.x corresponds to vm-memory 0.9.x, x upgrade does not break compatibility either.
2. Incompatible versions are released, and will not affect other crates.
3. If you do not upgrade the dependencies, it will also compile normally.

NOTE: An incompatible version means the left-most non-zero digit in x.y.z grouping is different.

Drawbacks

1. When an incompatible version is released, the rust-vmm crate that depends on this crate needs to provide a new version. So that users can choose when they need to upgrade.
2. When an incompatible version is released, it may be necessary to create a corresponding branch, in order to prepare for a patch release later. You can also not create it, then the incompatible version needs to be released immediately after the API incompatible change is merged.

Required Work

1. Version releases need to be strictly follow SemVer Compatibility to determine API compatibility, and release the corresponding version number.
2. Modify all rust-vmm crate dependencies to SemVer version requirements. and release a new incompatible version.
3. [Option] Create branches for incompatible versions.

Progress

• [ ] update vmm-sys-util.
• [ ] update vm-memory, kvm-bindings.
• [ ] update linux-loader, virtio-queue, kvm-ioctls.
• [ ] update others.

Thanks.

[andreeaflorescu]

This proposal sounds sane to me. We can enable dependabot updates for rust-vmm crates as well so that we can merge the dependency updates easier.

CC: @rbradford @sboeuf @bonzini @jiangliu what do you think?

[jiangliu]

Looks good to me:)

[sboeuf]

Sounds good to me

[slp]

I agree that switching to = x.y.z is a good idea, but that would only start fixing our problems once crates start switching from zero versions (0.y.z) to stable versions (1.y.z), as otherwise each increase of y would mean an incompatible version. I think vm-memory and vmm-sys-util are good candidates for becoming stable.

I've started packaging rust-vmm components in Fedora, and dealing with 0.y.z versions is making it really painful.

[wllenyj]

but that would only start fixing our problems once crates start switching from zero versions (0.y.z) to stable versions (1.y.z)

Mentioned in semver-compatibility:

Versions are considered compatible if their left-most non-zero major/minor/patch component is the same. 
For example, 1.0.3 and 1.1.0 are considered compatible, and thus it should be safe to update from 
the older release to the newer one. However, an update from 1.1.0 to 2.0.0 would not be allowed to 
be made automatically. This convention also applies to versions with leading zeros. For example, 
0.1.0 and 0.1.2 are compatible, but 0.1.0 and 0.2.0 are not. Similarly, 0.0.1 and 0.0.2 are not compatible.

So we can increase y in the 0.y.z for incompatible version, otherwise increase z for bugfix version. and also increase x in the x.y.z for incompatible version, otherwise increase y or z for bugfix version.

[andreeaflorescu]

So we can increase y in the 0.y.z for incompatible version, otherwise increase z for bugfix version. and also increase x in the x.y.z for incompatible version, otherwise increase y or z for bugfix version.

This is our current strategy. This is why your proposal for using caret dependencies works even for crates that are not at a major version. I think we can separate the discussions:

1. Use your proposal for crates that are at 0.x.y. @wllenyj can you open a PR so we can merge this proposal? Just to clarify, the proposal is not to use =0.x.y, but ^0.x.y which means that crates update automatically to newer patch versions, but don't update to minor versions which might have compatibility issues.
2. Discuss about moving to 1 for crates that are stable. @slp do you want to open an issue in the repositories where you think that should be the case?

[andreeaflorescu]

This is not fixed for vm-memory, re-opening the issue.