Whether nested virtualization possible with KVM hosts?

[naseemvk]

Hi Team

I am using cloudstack version 4.17.2.0

I want to know whether nested virtualization is possible if we are using using KVM hosts.

[boring-cyborg[bot]]

Thanks for opening your first issue here! Be sure to follow the issue template!

[weizhouapache]

@naseemvk just to clarify, you want to the VMs to support nested virtualization ?

[weizhouapache]

You need to configure the kvm host to support nested virtualization please refer to https://lists.apache.org/thread/zobt6yjwqbo22g8w0o5lt70py9d9pt6y

and also update agent.properties to add cpu flag "vmx" to vms https://lists.apache.org/thread/vzthx7hbdk4j184fpk37qzkjjfxnmvqg

[naseemvk]

Yes I want VMs to support nested virtualization

Exact requirement is I want to run Hyper V on Windows Server 2019 VM

My host processor is AMD EPYC

Just for clarification Whether I need to follow below steps since my host is already showing as nested virtualization enabled

Create the following file: /etc/modprobe.d/kvm-nested.conf and add the following details to the file: options kvm-amd nested=1

[weizhouapache]

Yes I want VMs to support nested virtualization

Exact requirement is I want to run Hyper V on Windows Server 2019 VM

My host processor is AMD EPYC

Just for clarification Whether I need to follow below steps since my host is already showing as nested virtualization enabled

Create the following file: /etc/modprobe.d/kvm-nested.conf and add the following details to the file: options kvm-amd nested=1

it looks good. nested is already enabled on the kvm host.

now you need to add a cpu flag to the vms. please refer to https://docs.cloudstack.apache.org/en/latest/installguide/hypervisor/kvm.html#configure-cpu-model-for-kvm-guest-optional

since the processor is AMD, you can try with a cpu feature "svm" ("vmx" is for intel processors)

[naseemvk]

I followed the same but didn't do the below step

_* Encode the text to be passed, as: '%3Ccpu%20mode%3D%27host-model%27%20check%3D%27partial%27%2F%3E'

• Stop and update the virtual machine, passing the encoded text as the extraconfig parameter: update virtualmachine extraconfig="%3Ccpu%20mode%3D%27host-model%27%20check%3D%27partial%27%2F%3E" id=_

Now when I checked inside my Hyper V server VM it is showing like this

agent.properties file content

cluster=2 pod=2 resource=com.cloud.hypervisor.kvm.resource.LibvirtComputingResource private.network.device=cloudbr0 domr.scripts.dir=scripts/network/domr/kvm guest.cpu.mode=host-model router.aggregation.command.each.timeout=600 guest.network.device=cloudbr1 keystore.passphrase=kmusdfrqCccYx5sfvy9YIsdfsdf hypervisor.type=kvm port=8250 zone=2 public.network.device=cloudbr1 local.storage.uuid=5afdsfce-3cb3-4985-ab1d-06esdfdf80cd host=192.168.0.200@static guid=2casdasd63-d67d-32d3-83eb-8adasdasdd8 LibvirtComputingResource.id=9201 workers=5 iscsi.session.cleanup.enabled=false vm.migrate.wait=3600 guest.cpu.features=svm

Super thanks for all your response and support

[weizhouapache]

I followed the same but didn't do the below step

_* Encode the text to be passed, as: '%3Ccpu%20mode%3D%27host-model%27%20check%3D%27partial%27%2F%3E'

• Stop and update the virtual machine, passing the encoded text as the extraconfig parameter: update virtualmachine extraconfig="%3Ccpu%20mode%3D%27host-model%27%20check%3D%27partial%27%2F%3E" id=_

Now when I checked inside my Hyper V server VM it is showing like this

agent.properties file content

cluster=2 pod=2 resource=com.cloud.hypervisor.kvm.resource.LibvirtComputingResource private.network.device=cloudbr0 domr.scripts.dir=scripts/network/domr/kvm guest.cpu.mode=host-model router.aggregation.command.each.timeout=600 guest.network.device=cloudbr1 keystore.passphrase=kmusdfrqCccYx5sfvy9YIsdfsdf hypervisor.type=kvm port=8250 zone=2 public.network.device=cloudbr1 local.storage.uuid=5afdsfce-3cb3-4985-ab1d-06esdfdf80cd host=192.168.0.200@static guid=2casdasd63-d67d-32d3-83eb-8adasdasdd8 LibvirtComputingResource.id=9201 workers=5 iscsi.session.cleanup.enabled=false vm.migrate.wait=3600 guest.cpu.features=svm

Super thanks for all your response and support

@naseemvk You are welcome. Can we close this issue?

[naseemvk]

Sorry @weizhouapache if my previous message caused any confusion.

What I intended to convey is NESTED VIRTUALIZATION IS NOT WORKING

After installing Hyper-V on my VM, it fails to boot after the restart.

Kindly let me know if anything further needs to be done. If anything I have missed.

So please keep this thread open. If we can achieve the results, it would be helpful for many others as well.

[weizhouapache]

Sorry @weizhouapache if my previous message caused any confusion.

What I intended to convey is NESTED VIRTUALIZATION IS NOT WORKING

After installing Hyper-V on my VM, it fails to boot after the restart.

Kindly let me know if anything further needs to be done. If anything I have missed.

So please keep this thread open. If we can achieve the results, it would be helpful for many others as well.

@naseemvk I did not use hyper-V before. As I remember, this should work if you use other hypervisors, e.g kvm or VMware. I suggest you to ask Microsoft for help. Maybe hyper-V has other requirements which are unknown to us.

[naseemvk]

Okay. after instaling Hyper-V in server 2019 VM, the VM fails to boot.

Also we tested in a Ubuntu VM by enabling KVM. When we run a VM(L2) in that the nested VM fails at booting

[weizhouapache]

In the case of ubuntu22, what's the error in cloudstack mgmt and agent when you start a VM on it?

[rohityadavcloud]

Yes @naseemvk if your CPU supports, you'll need to set guest.cpu.mode=host-passthrough in /etc/cloudstack/agent/agent.properties in case of the KVM host. Kindly use our users mailing list or Github discussions if you've further questions that aren't bug reports.