Crate Addition Request: vmm-fdt

[MrXinWang]

Crate Name

vmm-fdt (or vmm-arm-fdt? or any other better name :))

Short Description

A crate that provides flattened device tree functionality (for arm).

There is already an example implementation in Firecracker. However it is also very nice to implement the device tree as a seperate crate so that users can decide whether they need this feature (as arm architechture users usually need fdt, while x86 users usually do not). Also a seperate crate would enable the further work for making fdt more generic to support different devices.

Why is this crate relevant to the rust-vmm project?

rust-vmm is a project aiming at providing building blocks to let users customize their own VMMs. It currently supports both x86 architecture and arm architecture. The core reason for the existence of device tree in Linux is to provide a way to describe various non-discoverable hardware, which is essential for arm architecture. With this fdt crate, users have more freedom to tailor their VMM for different devices.

[andreeaflorescu]

@MrXinWang It makes sense to have this as a crate, I just have a few comments:

• From what I know fdt can be used on other platforms as well, so there is no need to make it arm specific. Ideally we would have support for x86_64 even if there are other ways of discovering devices on other platforms.
• There is already a device-tree crate on crates.io so we maybe can work with the maintainer of that crate and join efforts? Here is the documentation of the crate: https://docs.rs/device_tree/1.1.0/device_tree/ and the GitHub repository: https://github.com/mbr/device_tree-rs
• I think this is very useful as a crate, and I would be waiting for a few other people to give their input before creating the repository. Before we actually write the code, can we also have some design discussions about it?

What do you think?

[MrXinWang]

Hi @andreeaflorescu ! Thanks for your comments :))

what I know fdt can be used on other platforms as well, so there is no need to make it arm specific.

I highly agree.

There is already a device-tree crate on crates.io so we maybe can work with the maintainer of that crate and join efforts.

Thanks very much for this information! We would love to work together :))) I will take a look at their source code first.

Before we actually write the code, can we also have some design discussions about it?

Yes of course. For now, I just used cargo new --lib vmm-fdt to create a project locally on my machine and referenced the Firecracker example implementation to finish the basic porting. Currently, this version passed the cargo build and cargo test locally. Would be awesome to get feedback from others (in fact that is the reason why I posted this issue :)) ).

Looking forward to getting more comments.

[MrXinWang]

There is already a device-tree crate on crates.io

Also we noticed that there is another version of fdt in: https://docs.rs/crate/fdt/0.0.1/source/

[andreeaflorescu]

There is already a device-tree crate on crates.io

Also we noticed that there is another version of fdt in: https://docs.rs/crate/fdt/0.0.1/source/

Nice! We should check that out as well.

[michael2012z]

Hi, regarding the requirement of the new FDT crate. In Firecracker, FDT crate is used to build ".dtb" file from created devices. Could this requirement be extended in Rust-VMM? Do we need to cover more functionality in the FDT crate?

[MrXinWang]

Hi @andreeaflorescu !

Had some discussions with my colleagues who has experience in Firecracker community today. We think it is nice to do some code refactoring from the Firecracker project to extract all fdt common code into a single file (or files) and keep the "arch specific" code in different directories, as the FDT in Firecracker only support aarch64 so probably the code there is a little bit messy for creating a single crate.

By doing such code refactoring, we can support different architectures easily, and also if needed, we can add other functionality such as compiling .dts files (mentioned by @michael2012z) in the common code.

I wonder if you are happy with this design or are there any other suggestions. For now I keep working on code and hopefully will finish the first version for reviewing. What do you think?

[andreeaflorescu]

Hey @michael2012z and @MrXinWang.

Let's take a step back and see what we actually want to achieve with this crate. I don't have a good answer here as fdt is not necessarily my strong point, but it looks like if we first define the requirements, the usage and the interface, we might be able to answer the above questions. I'll also try to find some time to talk with the more savvy people to get their input as well.

I would also like to avoid just pulling the code from Firecracker and merge it as is in rust-vmm. The code in Firecracker is really laser focused on our use case and we would need to make sure we define the appropriate interface so that multiple VMMs can use of it. For example, when we created the vm-memory crate we actually had to work a lot on the interface so that it can fit all the known usecase. While functionality wise, vm-memory offers all the things required by Firecracker, at the interface level is very different. I am expecting the same to happen for other crates as well (this one included).

To wrap it up, I would like us to:

• define the features of this crate (what do we want to support)
• define how we want this to be used
• define the interfaces

IMO these things should happen before we discuss at the code level.

If you would like to make some improvements at the code level in Firecracker that would be totally fine. I would suggest to open an issue in the Firecracker repository as well to give the maintainers some visibility into the proposed changes.

[MrXinWang]

Requirements:

Use case: From the perspective of VMM, the vmm-fdt crate is used to handle the generation of the device tree blob when starting a VM kernel. When creating a VM with extra devices, these devices are firstly created and handled by the vm-device crate. The vmm-fdt crate should take charge of creating the corresponding device tree and compiling the resulting device tree to a device tree blob file which will be mapped to the guest physical address space.

The vmm-fdt crate should also take charge of some optional features such as printing current devices in the device tree.

Required Features:

• Generate a device tree blob(.dtb) from a list of devices

• Compile .dts file to .dtb file

• Add/remove a single device node to/from the existing device tree

• List device(s) in current device tree

• Update nodes of the existing device tree

Optional Features:

• Device hotplug

Interfaces:

High level interfaces:

• Add device

• Remove device

• Update device

• List current devices in the device tree

• Print a single device in the device tree

• Compile to .dtb file

• Mapping to vm-memory (this function may not belong to this crate)

Low level helper functions, including read/write/parse properties, big/little endianness operations, string operations, etc. We can refer implementations in https://docs.rs/crate/fdt/0.0.1/source/src/ or https://github.com/mbr/device_tree-rs.

@andreeaflorescu Any comments are welcomed :)))

[danielverkamp]

I recently rewrote the crosvm FDT support in native Rust (previously it was a wrapper around libfdt), and I would be happy to help integrate it into a vmm-fdt crate if it looks useful:

https://chromium.googlesource.com/chromiumos/platform/crosvm/+/refs/heads/main/arch/src/fdt.rs

This currently only provides an API for writing a device tree from scratch (not reading or modifying in place), so it does not fulfill all the requirements from the previous comment, but maybe it could be useful as a starting point.

[andreeaflorescu]

Thanks @danielverkamp! Nit on the name, can we use vm-fdt to be in tone with the other rust-vmm components? I'll wait for a few more people to chime in, and then create the repository!

@MrXinWang I'm sorry for not answering this in a reaaaaally long time. It got de-prioritized, and we never got around to it.

[MrXinWang]

Thanks @danielverkamp, beautiful work!

No problem @andreeaflorescu. Good to see we can restart the progress here.

[andreeaflorescu]

@danielverkamp I've created the vm-fdt repo: https://github.com/rust-vmm/vm-fdt.

@sameo and @jiangliu had some concerns related to the name (i.e. it is not a VM-specific functionality), but we couldn't find a better name. We can later rename it (before publishing to crates.io).

Closing this issue, let's continue the design discussion in the repo directly.