search

kubevirt / community

Community content
https://kubevirt.io
49 stars 103 forks source link

Generalize virtualization stack to different libvirt hypervisor-drivers #259

Open harshitgupta1337 opened 7 months ago

harshitgupta1337 commented 7 months ago

This KubeVirt design proposal discusses how KubeVirt can be used to create libvirt virtual machines that are backed by diverse hypervisor drivers, such as QEMU/KVM, Xen, VirtualBox, etc. The aim of this proposal is to enumerate the design and implementation choices for enabling this multi-hypervisor support in KubeVirt.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged): Fixes #

Special notes for your reviewer:

Checklist

This checklist is not enforcing, but it's a reminder of items that could be relevant to every PR. Approvers are expected to review this list.

Release note:

kubevirt-bot commented 7 months ago

Hi @harshitgupta1337. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
kubevirt-bot commented 7 months ago

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Once this PR has been reviewed and has the lgtm label, please assign fabiand for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files: - **[OWNERS](https://github.com/kubevirt/community/blob/main/OWNERS)** Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
alicefr commented 7 months ago

/cc

alicefr commented 7 months ago

@harshitgupta1337 I think what we need here it is an infrastructure for hypervisor plugins. Putting everything into kubevirt code would make the code base very large and hard to maintain. Do you think it would be possible to change the goal of this proposal and try to develop a plugin instead. Cloud Hypervisor could be the first hypervisor plugin.

alicefr commented 7 months ago

I think we need also a list of features supported by each specific hypervisor.

From the old proposal for Cloud Hypervisor integration, not all the features supported by QEMu are available in CH. We should have here also a similar section with the features as in the old proposal. Then, these options should be advertised in the new CRD if they are supported or not.

harshitgupta1337 commented 7 months ago

@harshitgupta1337 I think what we need here it is an infrastructure for hypervisor plugins. Putting everything into kubevirt code would make the code base very large and hard to maintain. Do you think it would be possible to change the goal of this proposal and try to develop a plugin instead. Cloud Hypervisor could be the first hypervisor plugin.

@alicefr You're referring to this feature in Golang, isn' it ? https://pkg.go.dev/plugin Then we would need 2 plugins, right? One for qemu for backwards compatibility and a new one for cloud-hypervisor?

alicefr commented 7 months ago

@harshitgupta1337 I think what we need here it is an infrastructure for hypervisor plugins. Putting everything into kubevirt code would make the code base very large and hard to maintain. Do you think it would be possible to change the goal of this proposal and try to develop a plugin instead. Cloud Hypervisor could be the first hypervisor plugin.

@alicefr You're referring to this feature in Golang, isn' it ? https://pkg.go.dev/plugin Then we would need 2 plugins, right? One for qemu for backwards compatibility and a new one for cloud-hypervisor?

No, I'm referring more to an API where we can plug different hypervisors, like CRI for the container runtimes

kubevirt-bot commented 4 months ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

/lifecycle stale

iholder101 commented 4 months ago

/cc I haven't looked at the proposal yet, but wanted to point out that this proposal reminds me of a similar design proposal from the past: https://github.com/kubevirt/community/pull/184.

This proposal is not focused specifically on cloud-hypervisor, which I think is a step forward. I think that reading the discussion there will be interesting and valuable, especially this comment. To summarize:

So we'd need a similar interface to CRI, but for hypervisor runtimes for kubevirt.

I think that it will definitely not be easy to design and implement something like that, especially since currently Kubevirt heavily relies on libvirt-specific features/APIs. Saying that, if you're willing to invest the effort to achieve it, I think it would be valuable and I will be happy to promote it.

stu-gott commented 4 months ago

/remove-lifecycle stale

kubevirt-bot commented 1 month ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

/lifecycle stale

iholder101 commented 1 month ago

@harshitgupta1337 are you still interested in this?

harshitgupta1337 commented 1 month ago

@iholder101 Yes I am working on addressing the suggestions mentioned earlier. I should be able to come up with an updated design by the end of this month.

iholder101 commented 1 month ago

@iholder101 Yes I am working on addressing the suggestions mentioned earlier. I should be able to come up with an updated design by the end of this month.

Great to hear that! Good luck, let me know if I can help in any way.

/remove-lifecycle stale

harshitgupta1337 commented 1 month ago

@harshitgupta1337 I think what we need here it is an infrastructure for hypervisor plugins. Putting everything into kubevirt code would make the code base very large and hard to maintain. Do you think it would be possible to change the goal of this proposal and try to develop a plugin instead. Cloud Hypervisor could be the first hypervisor plugin.

@alicefr You're referring to this feature in Golang, isn' it ? https://pkg.go.dev/plugin Then we would need 2 plugins, right? One for qemu for backwards compatibility and a new one for cloud-hypervisor?

No, I'm referring more to an API where we can plug different hypervisors, like CRI for the container runtimes

Hi @alicefr I am curious why a CRI-like interface is necessary in this scenario, given that libvirt provides a uniform abstraction for creating VMs?

xpivarc commented 4 weeks ago

@harshitgupta1337 In the code (virt-handler's one to be specific) you can find out that we directly configure qemu as well. So as such Libvirt does support multiple "runtimes" but since the libvirt is not privilege in Kubevirt, it can't do everything.

harshitgupta1337 commented 1 week ago

Open question: How to manage the features that are supported on one Virt Stack and not on another?

kubevirt-bot commented 5 days ago

Thanks for your pull request. Before we can look at it, you'll need to add a 'DCO signoff' to your commits.

:memo: Please follow instructions in the contributing guide to update your commits with the DCO

Full details of the Developer Certificate of Origin can be found at developercertificate.org.

The list of commits missing DCO signoff:

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
harshitgupta1337 commented 5 days ago

Hey @everyone, I have updated the design proposal with a couple of interfaces that I have added in my PoC that expose necessary functions for supporting both qemu-kvm and cloud-hypervisor. @vladikr