kubevirt / kubevirt

Kubernetes Virtualization API and runtime in order to define and manage virtual machines.
https://kubevirt.io
Apache License 2.0
5.63k stars 1.34k forks source link

Can't start VM on k3os #4992

Closed alexanderniebuhr closed 3 years ago

alexanderniebuhr commented 3 years ago

Is this a BUG REPORT or FEATURE REQUEST?:

Uncomment only one, leave it on its own line:

/kind bug /kind enhancement

What happened: Cannot start VM on k3os Cluster

status:
  activePods:
    4fd7782e-5787-44f8-964f-8936d4b1cfad: ""
  conditions:
  - lastProbeTime: null
    lastTransitionTime: "2021-02-09T13:05:19Z"
    message: '0/2 nodes are available: 2 Insufficient devices.kubevirt.io/vhost-net.'
    reason: Unschedulable
    status: "False"
    type: PodScheduled

What you expected to happen: Succesfully start VM

How to reproduce it (as minimally and precisely as possible): install cluster of k3os and try to start linux vm

Anything else we need to know?: logs of handler show that vhost-net is not present, shortly after that it is present, so the nodes should be available

2021-02-09T12:15:06.243859153Z {"component":"virt-handler","hostname":"balix","level":"info","pos":"virt-handler.go:174","timestamp":"2021-02-09T12:15:06.243769Z"}
2021-02-09T12:15:06.244835359Z W0209 12:15:06.244785 19241 client_config.go:614] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
2021-02-09T12:15:06.263789001Z {"component":"virt-handler","level":"info","msg":"Starting collector: node name=balix","pos":"prometheus.go:539","timestamp":"2021-02-09T12:15:06.263708Z"}
2021-02-09T12:15:06.263804300Z {"component":"virt-handler","level":"info","msg":"set verbosity to 2","pos":"virt-handler.go:383","timestamp":"2021-02-09T12:15:06.263708Z"}
2021-02-09T12:15:06.263813356Z {"component":"virt-handler","level":"info","msg":"STARTING informer kubeVirtInformer","pos":"virtinformers.go:254","timestamp":"2021-02-09T12:15:06.263745Z"}
2021-02-09T12:15:06.263815799Z {"component":"virt-handler","level":"info","msg":"STARTING informer extensionsKubeVirtCAConfigMapInformer","pos":"virtinformers.go:254","timestamp":"2021-02-09T12:15:06.263778Z"}
2021-02-09T12:15:06.263819048Z {"component":"virt-handler","level":"info","msg":"STARTING informer vmiInformer","pos":"virtinformers.go:254","timestamp":"2021-02-09T12:15:06.263788Z"}
2021-02-09T12:15:06.263821197Z {"component":"virt-handler","level":"info","msg":"STARTING informer configMapInformer","pos":"virtinformers.go:254","timestamp":"2021-02-09T12:15:06.263795Z"}
2021-02-09T12:15:06.263825922Z {"component":"virt-handler","level":"info","msg":"STARTING informer CRDInformer","pos":"virtinformers.go:254","timestamp":"2021-02-09T12:15:06.263803Z"}
2021-02-09T12:15:06.264252157Z {"component":"virt-handler","level":"info","msg":"certificate with common name 'kubevirt.io:system:client:virt-handler' retrieved.","pos":"cert-manager.go:198","timestamp":"2021-02-09T12:15:06.264202Z"}
2021-02-09T12:15:06.264330612Z {"component":"virt-handler","level":"info","msg":"certificate with common name 'kubevirt.io:system:node:virt-handler' retrieved.","pos":"cert-manager.go:198","timestamp":"2021-02-09T12:15:06.264296Z"}
2021-02-09T12:15:06.264812451Z {"component":"virt-handler","level":"info","msg":"metrics: max concurrent requests=3","pos":"virt-handler.go:392","timestamp":"2021-02-09T12:15:06.263801Z"}
2021-02-09T12:15:06.267863173Z {"component":"virt-handler","level":"info","msg":"Updating cluster config from KubeVirt to resource version '3324373'","pos":"config-map.go:515","timestamp":"2021-02-09T12:15:06.267784Z"}
2021-02-09T12:15:06.267879937Z {"component":"virt-handler","level":"info","msg":"set verbosity to 2","pos":"virt-handler.go:383","timestamp":"2021-02-09T12:15:06.267822Z"}
2021-02-09T12:15:06.282284117Z W0209 12:15:06.282224 19241 warnings.go:70] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
2021-02-09T12:15:06.294330105Z W0209 12:15:06.294239 19241 warnings.go:70] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
2021-02-09T12:15:06.367178081Z {"component":"virt-handler","level":"info","msg":"Starting virt-handler controller.","pos":"vm.go:1005","timestamp":"2021-02-09T12:15:06.367051Z"}
2021-02-09T12:15:06.467439890Z {"component":"virt-handler","level":"info","msg":"refreshed device plugins for permitted/forbidden host devices","pos":"device_controller.go:211","timestamp":"2021-02-09T12:15:06.467330Z"}
2021-02-09T12:15:06.502047834Z {"component":"virt-handler","level":"info","msg":"enabled device-pluings for: []","pos":"device_controller.go:212","timestamp":"2021-02-09T12:15:06.467366Z"}
2021-02-09T12:15:06.502053938Z {"component":"virt-handler","level":"info","msg":"disabled device-pluings for: []","pos":"device_controller.go:213","timestamp":"2021-02-09T12:15:06.467375Z"}
2021-02-09T12:15:06.502082699Z {"component":"virt-handler","level":"info","msg":"Starting a device plugin for device: tun","pos":"device_controller.go:88","timestamp":"2021-02-09T12:15:06.467366Z"}
2021-02-09T12:15:06.502085525Z {"component":"virt-handler","level":"info","msg":"refreshed device plugins for permitted/forbidden host devices","pos":"device_controller.go:211","timestamp":"2021-02-09T12:15:06.467390Z"}
2021-02-09T12:15:06.502106575Z {"component":"virt-handler","level":"info","msg":"enabled device-pluings for: []","pos":"device_controller.go:212","timestamp":"2021-02-09T12:15:06.467397Z"}
2021-02-09T12:15:06.502114689Z {"component":"virt-handler","level":"info","msg":"disabled device-pluings for: []","pos":"device_controller.go:213","timestamp":"2021-02-09T12:15:06.467408Z"}
2021-02-09T12:15:06.502117620Z {"component":"virt-handler","level":"info","msg":"Starting a device plugin for device: kvm","pos":"device_controller.go:88","timestamp":"2021-02-09T12:15:06.467420Z"}
2021-02-09T12:15:06.502123795Z {"component":"virt-handler","level":"info","msg":"Starting a device plugin for device: vhost-net","pos":"device_controller.go:88","timestamp":"2021-02-09T12:15:06.467466Z"}
2021-02-09T12:15:06.502127240Z {"component":"virt-handler","level":"info","msg":"set verbosity to 2","pos":"virt-handler.go:383","timestamp":"2021-02-09T12:15:06.467618Z"}
2021-02-09T12:15:06.505701263Z {"component":"virt-handler","level":"info","msg":"kvm device plugin started","pos":"generic_device.go:178","timestamp":"2021-02-09T12:15:06.505630Z"}
2021-02-09T12:15:06.505715935Z {"component":"virt-handler","level":"info","msg":"vhost-net device plugin started","pos":"generic_device.go:178","timestamp":"2021-02-09T12:15:06.505663Z"}
2021-02-09T12:15:06.505802828Z {"component":"virt-handler","level":"info","msg":"device '/dev/kvm' is present.","pos":"generic_device.go:318","timestamp":"2021-02-09T12:15:06.505722Z"}
2021-02-09T12:15:06.505807482Z {"component":"virt-handler","level":"warning","msg":"device '/dev/vhost-net' is not present, the device plugin can't expose it.","pos":"generic_device.go:315","timestamp":"2021-02-09T12:15:06.505721Z"}
2021-02-09T12:15:06.505810060Z {"component":"virt-handler","level":"info","msg":"tun device plugin started","pos":"generic_device.go:178","timestamp":"2021-02-09T12:15:06.505768Z"}
2021-02-09T12:15:06.505843211Z {"component":"virt-handler","level":"info","msg":"device '/dev/net/tun' is present.","pos":"generic_device.go:318","timestamp":"2021-02-09T12:15:06.505807Z"}
2021-02-09T12:15:06.514561180Z {"component":"virt-handler","level":"info","msg":"device '/dev/vhost-net' is present.","pos":"generic_device.go:318","timestamp":"2021-02-09T12:15:06.514489Z"}
2021-02-09T12:15:27.494684496Z {"component":"virt-handler","level":"info","msg":"Updating cluster config from KubeVirt to resource version '3324888'","pos":"config-map.go:515","timestamp":"2021-02-09T12:15:27.494610Z"}
2021-02-09T12:15:27.494765966Z {"component":"virt-handler","level":"info","msg":"refreshed device plugins for permitted/forbidden host devices","pos":"device_controller.go:211","timestamp":"2021-02-09T12:15:27.494676Z"}
2021-02-09T12:15:27.494773320Z {"component":"virt-handler","level":"info","msg":"enabled device-pluings for: []","pos":"device_controller.go:212","timestamp":"2021-02-09T12:15:27.494702Z"}
2021-02-09T12:15:27.494776250Z {"component":"virt-handler","level":"info","msg":"set verbosity to 2","pos":"virt-handler.go:383","timestamp":"2021-02-09T12:15:27.494709Z"}
2021-02-09T12:15:27.494778707Z {"component":"virt-handler","level":"info","msg":"disabled device-pluings for: []","pos":"device_controller.go:213","timestamp":"2021-02-09T12:15:27.494723Z"}
2021-02-09T12:20:06.367946656Z {"component":"virt-handler","level":"info","msg":"resyncing virt-launcher domains","pos":"cache.go:385","timestamp":"2021-02-09T12:20:06.367802Z"}
2021-02-09T12:23:32.295359312Z W0209 12:23:32.295269 19241 warnings.go:70] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
2021-02-09T12:25:06.367986047Z {"component":"virt-handler","level":"info","msg":"resyncing virt-launcher domains","pos":"cache.go:385","timestamp":"2021-02-09T12:25:06.367817Z"}
2021-02-09T12:30:06.367657754Z {"component":"virt-handler","level":"info","msg":"resyncing virt-launcher domains","pos":"cache.go:385","timestamp":"2021-02-09T12:30:06.367552Z"}
2021-02-09T12:30:20.295965266Z W0209 12:30:20.295899 19241 warnings.go:70] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition

Environment:

gitlawr commented 3 years ago

You need to load the vhost_net kernel module. Example snippet in k3os config:

k3os:
  modules:
  - kvm
  - vhost_net

For reference: https://github.com/rancher/k3os#k3osmodules

alexanderniebuhr commented 3 years ago

fixed it with the suggestion by @gitlawr