Closed yisun-git closed 12 months ago
I am much in favour of this approach. Although I do have to say that we will probably not have the time to work on this any time soon. If you want to pick this up I can help along the way.
Do you have any design in mind? I am thinking about the following questions:
Anyway, I find this a interesting problem to solve!
Does this limit Firecracker to EC2 instances? If so, why such a limitation–is the emulated CPUID
required for the guest to understand what instructions are available?
I am much in favour of this approach. Although I do have to say that we will probably not have the time to work on this any time soon. If you want to pick this up I can help along the way.
Thank you! I'd like to have a try.
Do you have any design in mind?
Yes, I have got some ideas.
First, I want to start from refining FC's cpuid module to provide generic framework to handle x86 cpu model related resources, cpuid/msr/etc. (Other architectures are not considered yet.) Per my observation, I think FC's current cpuid module can be refined for below things.
So I plan to do below things.
Second, I want to extend above CPU model to handle CPU states, address space, interrupts, etc. The goal is also generic and hypervisor agnostic.
Last, I'd like to implement cpu hotplug, live migration capabilities.
I am thinking about the following questions:
- How does the user pass the CPU Template? Is it going to be a huge JSON sent as part of the machine-config API call? Or do you want to pass it as a configuration file?
Personally, I like a configuration file or a template in code. For template in code, I expect the template codes should be very simple to just add a few codes, e.g. a model string, the CPUID settings.
- How would a template look like? Whitelist vs blacklist?
I think there may be different ways. So far, I have two ideas.
Depending on the approach it might be a relatively complicated problem to solve. One thing to keep in mind is that we will most likely not have any emulated cpu features in Firecracker. So if one template is asking to enable a feature which is not supported by hardware, nor emulated by KVM the guest might report as supported an unsupported feature.
Yes, I see. There will be CPUID filter. We can compare user settings with host enabled features. Only the common features are reported to Guest.
Anyway, I find this a interesting problem to solve!
Does this limit Firecracker to EC2 instances? If so, why such a limitation–is the emulated
CPUID
required for the guest to understand what instructions are available?
Current FC's cpuid can only support EC2 C3 and EC2 T2 instance types. My plan is to remove such limitation. I guess the reason of this limitation is that FC should support AWS first. :)
Does this limit Firecracker to EC2 instances? If so, why such a limitation–is the emulated
CPUID
required for the guest to understand what instructions are available?
Firecracker can be run on any bare metal instance and even with nested virtualization. We only implemented two templates that are indeed specific to EC2 instances, but this doesn't mean that there is any limitation in running Firecracker on any machine that has access to /dev/kvm
. By default, Firecracker microVMs will report the CPUID of the host with minor changes like CPU topology, Cache topology and some other disabled features (like PMU).
The purpose of this issue as I see it is to enhance the Firecracker cpuid crate to support other custom templates. Ideally these templates are not hardcoded in the code, but rather can be passed as configuration files through Firecracker API.
To make cpu-model be hypervisor agnostic, we need do some works to make Firecracker be hypervisor agnostic. So, I proposed a hypervisor agnostic solution on rust-vmm as below.
https://github.com/rust-vmm/vmm-vcpu/issues/5
After this is done, we can implement other cpu-model functions described above.
Added this to the roadmap, under the "Researching" column.
Removing it from the roadmap because we do not have any task related to it in a foreseeable future.
Since 1.4, firecracker has support for CPUID customization. See https://github.com/firecracker-microvm/firecracker/blob/main/docs/cpu_templates/cpu-templates.md#custom-cpu-templates for documentation.
Although Firecracker has supported CPUID and provided templates, it only supports EC2 C3 and EC2 T2 instance types. Per discussion with Andreea, it seems not easy to add more templates especially customized templates.
But customized CPU model is important because of below reasons.
So I propose to provide a generic framework which has standard interfaces and flexible mechanism to support customized CPU models.