rust-vmm / community

rust-vmm community content
477 stars 26 forks source link

Manual update dependencies in rust-vmm crates - and provide releases #136

Open andreeaflorescu opened 1 year ago

andreeaflorescu commented 1 year ago

This update is needed because otherwise there will be crates using vmm-sys-util 0.10.0 and others using 0.11.0. The reason behind this is the (wrong) way in which we specify dependencies in a few of the rust-vmm crates. Specifically, some crates will just pick up the latest available version of a crate even if it introduces breaking changes (i.e. picking up 0.11.0 because the dep is specified as >= 0.9.0). We're progressively fixing this by always having dependencies specified with caret, but this work was not yet completed for a few crates. As such, we need to:

For example, we first need to update kvm-bindings, publish it, and then update kvm-ioctls by specifying a new version for both vmm-sys-util and kvm-bindings.

This issue is tracking this one time effort. We're not tracking here a saner way to do the updates, as this is going to happen afterwards. Tracking issues: #135 & TBD (for releases).

roypat commented 1 year ago

I went through all of the rust-vmm crates and created the following table that keeps track of all their dependencies and whether they currently rely on (a) non-caret requirements and (b) outdated versions

Crate Version Uses caret deps? rust-vmm deps at fixed versions (outdated are bold) rust-vmm deps at non-caret versions
kvm-ioctl 0.11.0 yes vmm-sys-util@0.10.0 kvm-bindings@0.5.0 (MERGED)
vmm-sys-util 0.11.0 yes
kvm-bindings 0.5.0 yes vmm-sys-util@0.10.0 (MERGED)
event-manager 0.2.1 no vmm-sys-util>=0.8.0
vfio-ioctls 0.1.0 no kvm-bindings\~0, kvm-ioctls\~0, vm-memory>=0.6, vfio-bindings~0, vmm-sys-util>=0.8.0 (NO PR OPEN)
vfio-bindings 0.3.1 no vmm-sys-util>=0.8.0 (NO PR OPEN)
vhost-device-gpio 0.1.0 no virtio-queue@0.6. vhost-user-backend@0.7.0 vmm-sys-util=0.10.0. vm-memory>=0.8, virtio-binding>=0.1 (NO PR OPEN)
vhost-device-i2c 0.1.0 no vhost-user-backend@0.7.0, vhost@0.5, virtio-queue@0.6 vm-memory>=0.8, vmm-sys-util=0.10.0 (NO PR OPEN)
vhost-device-rng 0.1.0 no vhost@0.5, vhost-user-backend@0.7.0, virtio-queue@0.6 vm-memory>=0.8, vmm-sys-util=0.10.0, virtio-bindings>=0.1 (NO PR OPEN)
vhost-device-vsock 0.1.0 no vhost@0.5, vhost-user-backend@0.7.0, virtio-queue@0.6, viortio-vsock@0.1.0 vm-memory>=0.8, vmm-sys-util=0.10.0, virtio-bindings>=0.1 (NO PR OPEN)
mshv-ioctls 0.1.1 no vmm-sys-util>=0.8.0 (NO PR)
mshv-bindings 0.1.1 no vmm-sys-util>=0.2.1 (NO PR)
virtio-blk 0.1.0 no vmm-sys-util@0.10.0, vm-memory@0.9.0, virtio-device, virtio-bindings(MERGED)
virtio-console 0.1.0 no virtio-queue@0.6.0, virtio-bindings@0.1.0, vm-memory@0.9.0(MERGED)
virtio-vsock 0.1.0 no virtio-queue@0.6.0, virtio-bindings@0.1.0, vm-memory@0.9.0(MERGED)
virtio-queue 0.6.1 no vm-memory@0.9.0, vmm-sys-util@0.10.0, virtio-bindings@0.1.0(MERGED)
virtio-queue-ser 0.3.0 no virtio-memory@0.9.0, virtio-queue@0.6.0(MERGED)
virtio-device 0.1.0 no vm-memory@0.9.0, virtio-queue@0.6.0, virtio-bindings (MERGED)
virtio-bindings 0.1.0 no
vm-fdt 0.2.0 yes
vm-device 0.1.0 yes
vm-allocator 0.1.0 no
vm-memory 0.9.0 yes vmm-sys-util@0.10.0 (MERGED)
vhost 0.5.0 yes vmm-sys-util@0.10.0 (No PR), vm-memory@0.9.0
vhost-user-backend 0.7.0 yes vmm-sys-util@0.10.0 (No PR), vm-memory@0.9.0, vhost@0.5, virtio-queue@0.6, virtio-bindings@0.1
vm-superio 0.6.0 no vmm-sys-util>=0.7.0 (MERGED)
vm-superio-ser 0.2.0 no vm-superio@=0.6.0 vmm-sys-util>=0.7.0 (MERGED)
linux-loader 0.7.0 yes vm-memory@0.9.0 (MERGED)
roypat commented 1 year ago

Going through the dependency graph, updates need to be done in the following order (all crates on the same level can be updated in parallel):

  1. vm-superio, vm-memory and kvm-bindings, event-manager
  2. vhost, vm-superio-ser, virtio-queue, virtio-blk, kvm-ioctl
  3. vm-vhost-user-backend, the various vhost-*, vfio-* and mshv-* crates
andreeaflorescu commented 1 year ago

Based on @roypat's analysis, we need to go through 3 stages (corresponding to the number list above).

Stage 1

vm-superio - DONE

Changes Needed:

Blocks: vm-superio-ser

event-manager - DONE

Changes Needed:

Blocks: N/A

vm-memory - DONE

Changes Needed:

Blocks:

kvm-bindings - DONE

Changes Needed:

roypat commented 1 year ago

At this point, all stage 1 crates have PRs open fixing this issues. Next step is their crates.io release, so we can start working on the Step 2 crates.

aghecenco commented 1 year ago

kvm-bindings v0.7.0 is released :white_check_mark: event-manager CI is repeatedly failing with a strange permissions issue on a local dir (seemingly unrelated to these releases, more likely a Buildkite thing and one more problem to fix)

studychao commented 1 year ago

kvm-bindings v0.7.0 is released ✅ event-manager CI is repeatedly failing with a strange permissions issue on a local dir (seemingly unrelated to these releases, more likely a Buildkite thing and one more problem to fix)

Is that v0.6.0 or a newer version of kvm-bindings is planned to release?

andreeaflorescu commented 1 year ago

kvm-bindings v0.7.0 is released ✅ event-manager CI is repeatedly failing with a strange permissions issue on a local dir (seemingly unrelated to these releases, more likely a Buildkite thing and one more problem to fix)

Is that v0.6.0 or a newer version of kvm-bindings is planned to release?

It is 0.6.0.

andreeaflorescu commented 1 year ago

Stage 2

vm-superio-ser - DONE

kvm-ioctls - DONE

virtio-queue - DONE

linux-loader - DONE

wllenyj commented 1 year ago

image

rbradford commented 1 year ago

I know that a release is missing for (at least) vhost-user-backend.

rbradford commented 1 year ago

I know that a release is missing for (at least) vhost-user-backend.

This is all done now. Thanks to @stefano-garzarella. vfio-ioctls & vfio-bindings remain along with some of the virtio backend devices.

andreeaflorescu commented 1 year ago

I know that a release is missing for (at least) vhost-user-backend.

This is all done now. Thanks to @stefano-garzarella. vfio-ioctls & vfio-bindings remain along with some of the virtio backend devices.

@jiangliu @liuw @sameo @sboeuf can one of you take care of the two crates as well? What is important is properly setting the crate requirements to caret (if they have any that are not properly specified), and provide a release.