rust-vmm / rust-vmm-container

Container with all dependencies required for running rust-vmm crates integration tests.
Apache License 2.0
65 stars 32 forks source link

Initial riscv64 support for rust-vmm-container #91

Open endeneer opened 10 months ago

endeneer commented 10 months ago

This pull request is to introduce riscv64 Docker image for rust-vmm CI purpose.

Unlike x86/ARM CI test, we don't have native server for RISC-V. As a workaround, QEMU can be used so that we can run RISC-V CI on x86/ARM CI server.

The CI flow using QEMU looks like this:

  1. Pass the repo to be tested as a volume to Docker container.
  2. In the container, cross-compile the repo (I did try Rust RISC-V native toolchain, but it seems unstable at the moment).
  3. The Docker container proceed to launch a QEMU RISC-V guest and pass the cross-compiled repo to the guest using 9pfs.
  4. The guest boots, and during init, a custom systemd service (test.service) will mount the 9pfs, which then searches and runs all the cross-compiled test binaries in that 9pfs mount point.
  5. The guest writes to virt test MMIO device using devmem utility to trigger QEMU exit with custom exit code.
  6. The exit code is passed along the exit path from QEMU to Docker container and back to host CI server.

To better explain the CI flow, I also made a YouTube video: https://youtu.be/r2OhfmPq74U

andreeaflorescu commented 10 months ago

@endeneer thanks for adding this support in rust-vmm. I think this is pretty cool!

We had some problems with the space on the git runners because of running in separate RUN commands. That's why we added all required tools in a single RUN command. See https://github.com/rust-vmm/rust-vmm-container/pull/73/commits/510807353c7ca9a8af0863d940e02799e99095d1. It might be that you'll need to update the Docker file to have it similarly as in the other ones to not run in the space problem.

I also notice that the CI is not running for the newly added container. I guess this is because we need to update the .github/workflows as well to include the scripts related to building the RISC-V container. Can you update it and that way we'll also see if the build is successful?

endeneer commented 8 months ago

Thanks @andreeaflorescu, I have updated the Dockerfile as suggested. I have successfully tested build and push in my own repository:

The Buildkite CI pipeline generation is ready too (https://github.com/rust-vmm/rust-vmm-ci/pull/148).

PS 🎅 🎄 Merry Christmas everyone, looking forward to adding more riscv64 support for rust-vmm next year, but for now, just have fun! ☃️

TexasOct commented 4 months ago

I noticed that this pull request hasn't changed in a long time, and I'd like to do some work for it, what can I do for it?

If needed, I will do this work in a new pull request to taking it forward.

endeneer commented 3 months ago

Hi @TexasOct , I just updated this PR by resolving the suggestions from maintainer.
At the moment, before rust-vmm starts to accept riscv64-related commits, it's unsure whether we shall wait until Buildkite supports RISC-V agent platform, or use the workaround I provided in this PR (run CI in QEMU on x86 Buildkite agent platform). Maybe we shouldn't rush, at the same time keeping an eye on industry news regarding availability of RISC-V servers for CI/CD services.