search

kubernetes / minikube

Run Kubernetes locally
https://minikube.sigs.k8s.io/
Apache License 2.0
29.35k stars 4.88k forks source link

kvm2 driver: check if user in "libvirt" group and exit if they are not #5617

Closed piccadil closed 3 years ago

piccadil commented 5 years ago

Please, add that user starting minikube should be in "libvirt" group. Because without adding user in "libvirt" group will be an error "authentification unavailable"

medyagh commented 5 years ago

preferably we should do that for the users. or tell them nicely that it should be like that

@aidkz do you mind sharing our OS Distro and version ?

piccadil commented 5 years ago
  • @josedonizetti what you think ?

preferably we should do that for the users. or tell them nicely that it should be like that

@aidkz do you mind sharing our OS Distro and version ? Linux localhost.localdomain 3.10.0-1062.1.2.el7.x86_64 CentOS Linux release 7.7.1908 (Core) Name : qemu-kvm Arch : x86_64 Epoch : 10 Version : 1.5.3 Release : 167.el7_7.1

afbjorklund commented 5 years ago

CentOS is supposed to provide some kind of authentication mechanism, like PolicyKit ?

Of course, that also means entering your password each time you want to access it...

afbjorklund commented 5 years ago

Please, add that user starting minikube should be in "libvirt" group.

Note that the name of the group might vary between Linux distributions.

For instance, on Ubuntu 16.04 the socket is owned by the "libvirtd" group...

Might be better to refer to external documentation, like https://libvirt.org ?

tstromberg commented 5 years ago

Running virt-host-validate should provide the correct checks here. Our documentation calls for it, but we should do so as well in our code.

piccadil commented 5 years ago

running virt-host-validate not showing checks for user/group permissions:

virt-host-validate
  QEMU: Checking for hardware virtualization                                 : PASS
  QEMU: Checking if device /dev/kvm exists                                   : PASS
  QEMU: Checking if device /dev/kvm is accessible                            : PASS
  QEMU: Checking if device /dev/vhost-net exists                             : PASS
  QEMU: Checking if device /dev/net/tun exists                               : PASS
  QEMU: Checking for cgroup 'memory' controller support                      : PASS
  QEMU: Checking for cgroup 'memory' controller mount-point                  : PASS
  QEMU: Checking for cgroup 'cpu' controller support                         : PASS
  QEMU: Checking for cgroup 'cpu' controller mount-point                     : PASS
  QEMU: Checking for cgroup 'cpuacct' controller support                     : PASS
  QEMU: Checking for cgroup 'cpuacct' controller mount-point                 : PASS
  QEMU: Checking for cgroup 'cpuset' controller support                      : PASS
  QEMU: Checking for cgroup 'cpuset' controller mount-point                  : PASS
  QEMU: Checking for cgroup 'devices' controller support                     : PASS
  QEMU: Checking for cgroup 'devices' controller mount-point                 : PASS
  QEMU: Checking for cgroup 'blkio' controller support                       : PASS
  QEMU: Checking for cgroup 'blkio' controller mount-point                   : PASS
  QEMU: Checking for device assignment IOMMU support                         : WARN (No ACPI DMAR table found, IOMMU either disabled in BIOS or not supported by this hardware platform)
medyagh commented 4 years ago

I would be happy to review any PR that would add this check !

nanikjava commented 4 years ago

/assign @nanikjava

nanikjava commented 4 years ago

The main code resides inside pkg/minikube/registry/devs/kvm2/kvm2.go

All drivers init() function initialize the driver using the registry.Registry(..)which accept registry.DriverDef(..)struct. The DriverDef(..)struct only have specific function for initialize the driver (Loader) and status checker (StatusChecker) nothing in between.

The 'cleanest' way is to add something like a 'precheck' function that will do other necessary steps after initializing the driver, however this will make the changed quite bit as there are codes (test and non-test) that will need to be refactored.

The alternative is to add the group checking inside the init function however the downside is that the error message will be bit confusing for the user.

tstromberg commented 4 years ago

@nanikjava - I suspect the current call to virsh domcapabilities will fail if the user is not in the correct group, but quite likely with a terribly confusing message.

One trick here is how to reliably discover what group the user should be in to launch a VM. You will probably have more luck finding a virsh command that reliably fails if the user is not in the correct group, and provide the appropriate advice if it fails.

Every distro I know of uses libvirt or libvirtd, but I'm sure there are some strange configurations out there.

nanikjava commented 4 years ago

Testing was done by removing local account from libvirt group. This is the full error after running

minikube start --vm-driver=kvm2 

I1117 20:24:12.412275    3016 translate.go:92] Setting Language to en-US ...
I1117 20:24:12.412826    3016 out.go:131] Setting OutFile to fd 1 ...
I1117 20:24:12.412845    3016 out.go:172] isatty.IsTerminal(1) = true
I1117 20:24:12.412853    3016 out.go:138] Setting ErrFile to fd 2...
I1117 20:24:12.412863    3016 out.go:172] isatty.IsTerminal(2) = true
I1117 20:24:12.412962    3016 root.go:284] Updating PATH: /home/nanik/.minikube/bin
I1117 20:24:12.413705    3016 notify.go:125] Checking for updates...
I1117 20:24:12.679333    3016 start.go:250] hostinfo: {"hostname":"pop-os","uptime":195333,"bootTime":1573787319,"procs":491,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"19.04","kernelVersion":"5.3.0-22-generic","virtualizationSystem":"kvm","virtualizationRole":"host","hostid":"c50e6dd4-924b-a886-a6a3-2b925dce0cdd"}
I1117 20:24:12.680457    3016 start.go:260] virtualization: kvm host
😄  minikube v1.6.0-beta.0 on Ubuntu 19.04
.......
.......
.......
.......

I1117 20:24:12.971718    3016 start.go:292] selected driver: kvm2
I1117 20:24:12.971728    3016 start.go:582] validating driver "kvm2" against <nil>
I1117 20:24:13.006292    3016 start.go:588] status for kvm2: {Installed:true Healthy:false Error:/usr/bin/virsh domcapabilities --virttype kvm failed:
error: failed to connect to the hypervisor
error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': Permission denied Fix:Follow your Linux distribution instructions for configuring KVM Doc:https://minikube.sigs.k8s.io/docs/reference/drivers/kvm2/}

⚠️  'kvm2' driver reported an issue: /usr/bin/virsh domcapabilities --virttype kvm failed:
error: failed to connect to the hypervisor
error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': Permission denied
💡  Suggestion: Follow your Linux distribution instructions for configuring KVM
📘  Documentation: https://minikube.sigs.k8s.io/docs/reference/drivers/kvm2/

I1117 20:24:13.006966    3016 install.go:102] Validating docker-machine-driver-kvm2, PATH=/home/nanik/.minikube/bin:/home/nanik/bin:/home/nanik/Downloads/node-v10.16.3-linux-x64/bin/:/home/nanik/Downloads/node-v10.16.3-linux-x64/bin/gradle-5.6.2-bin/bin:/home/nanik/Downloads/Postman/app:/home/nanik/bin/jdk-9.0.4/bin:/home/nanik/AndroidProjects/android-studio/bin:/home/nanik/Golang/go/bin:/home/nanik/AndroidProjects/android-sdk/platform-tools:/home/nanik/AndroidProjects/android-sdk/tools/bin:/home/nanik/AndroidProjects/android-sdk/emulator:/home/nanik/bin:/home/nanik/AndroidProjects/docker/docker:/home/nanik/AndroidProjects/android-sdk/emulator/qemu/linux-x86_64:/home/nanik/Music:/home/nanik/AndroidStudioProjects/ReverseEngineering/dex-tools-2.1-SNAPSHOT:/home/nanik/AndroidStudioProjects/ReverseEngineering/dex-tools-2.1-SNAPSHOT/bin:/home/nanik/AndroidStudioProjects/nanikjavaminikube/src/golang.org/x/tools/cmd/goimports:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
W1117 20:24:13.023961    3016 install.go:46] docker-machine-driver-kvm2: docker-machine-driver-kvm2 is version 1.5.2, want 1.6.0-beta.0
💾  Downloading driver docker-machine-driver-kvm2:
I1117 20:24:13.025739    3016 install.go:145] Downloading: &{Ctx:<nil> Src:https://github.com/kubernetes/minikube/releases/download/v1.6.0-beta.0/docker-machine-driver-kvm2?checksum=file:https://github.com/kubernetes/minikube/releases/download/v1.6.0-beta.0/docker-machine-driver-kvm2.sha256 Dst:/home/nanik/.minikube/bin/docker-machine-driver-kvm2 Pwd: Mode:2 Detectors:[] Decompressors:map[] Getters:map[] Dir:false ProgressListener:<nil> Options:[0x9a73b0]}
    > docker-machine-driver-kvm2.sha256: 65 B / 65 B [-------] 100.00% ? p/s 0s
    > docker-machine-driver-kvm2: 13.86 MiB / 13.86 MiB  100.00% 46.30 KiB p/s 
I1117 20:29:34.102657    3016 downloader.go:60] Not caching ISO, using https://storage.googleapis.com/minikube/iso/minikube-v1.5.1.iso
I1117 20:29:34.102854    3016 profile.go:89] Saving config to /home/nanik/.minikube/profiles/minikube/config.json ...
I1117 20:29:34.102906    3016 lock.go:41] attempting to write to file "/home/nanik/.minikube/profiles/minikube/config.json" with filemode -rw-------
I1117 20:29:34.110656    3016 cache_images.go:300] CacheImage: k8s.gcr.io/k8s-dns-kube-dns-amd64:1.14.13 -> /home/nanik/.minikube/cache/images/k8s.gcr.io/k8s-dns-kube-dns-amd64_1.14.13
I1117 20:29:34.110659    3016 cluster.go:97] Machine does not exist... provisioning new machine
I1117 20:29:34.110710    3016 cache_images.go:300] CacheImage: k8s.gcr.io/pause:3.1 -> /home/nanik/.minikube/cache/images/k8s.gcr.io/pause_3.1
I1117 20:29:34.110714    3016 cache_images.go:300] CacheImage: k8s.gcr.io/kube-addon-manager:v9.0 -> /home/nanik/.minikube/cache/images/k8s.gcr.io/kube-addon-manager_v9.0
I1117 20:29:34.110726    3016 cache_images.go:300] CacheImage: k8s.gcr.io/coredns:1.6.2 -> /home/nanik/.minikube/cache/images/k8s.gcr.io/coredns_1.6.2
I1117 20:29:34.110733    3016 cache_images.go:300] CacheImage: k8s.gcr.io/kube-proxy:v1.17.0-beta.1 -> /home/nanik/.minikube/cache/images/k8s.gcr.io/kube-proxy_v1.17.0-beta.1
I1117 20:29:34.110751    3016 cache_images.go:306] /home/nanik/.minikube/cache/images/k8s.gcr.io/coredns_1.6.2 exists
I1117 20:29:34.110744    3016 cache_images.go:306] /home/nanik/.minikube/cache/images/k8s.gcr.io/kube-addon-manager_v9.0 exists

.......
.......
.......
.......

💡  Tip: Use 'minikube start -p <name>' to create a new cluster, or 'minikube delete' to delete this one.
I1117 20:30:05.686432    3016 main.go:110] libmachine: (minikube) Calling .GetState
I1117 20:30:05.686731    3016 cluster.go:114] Machine state:  Error
🔄  Retriable failure: Error getting state for host: getting connection: getting domain: error connecting to libvirt socket.: virError(Code=38, Domain=7, Message='Failed to connect socket to '/var/run/libvirt/libvirt-sock': Permission denied')
I1117 20:30:05.686964    3016 main.go:110] libmachine: Found binary path at /home/nanik/.minikube/bin/docker-machine-driver-kvm2
I1117 20:30:05.687004    3016 main.go:110] libmachine: Launching plugin server for driver kvm2
I1117 20:30:05.698701    3016 main.go:110] libmachine: Plugin server listening at address 127.0.0.1:40435
I1117 20:30:05.698959    3016 main.go:110] libmachine: () Calling .GetVersion
I1117 20:30:05.699246    3016 main.go:110] libmachine: Using API Version  1
I1117 20:30:05.699255    3016 main.go:110] libmachine: () Calling .SetConfigRaw
I1117 20:30:05.699445    3016 main.go:110] libmachine: () Calling .GetMachineName
I1117 20:30:05.699736    3016 main.go:110] libmachine: Found binary path at /home/nanik/.minikube/bin/docker-machine-driver-kvm2
I1117 20:30:05.699756    3016 main.go:110] libmachine: Launching plugin server for driver kvm2
I1117 20:30:05.711294    3016 main.go:110] libmachine: Plugin server listening at address 127.0.0.1:46817
I1117 20:30:05.711619    3016 main.go:110] libmachine: () Calling .GetVersion
I1117 20:30:05.711988    3016 main.go:110] libmachine: Using API Version  1
I1117 20:30:05.711999    3016 main.go:110] libmachine: () Calling .SetConfigRaw
I1117 20:30:05.712186    3016 main.go:110] libmachine: () Calling .GetMachineName
I1117 20:30:05.712303    3016 main.go:110] libmachine: (minikube) Calling .GetState
⚠️  Unable to get the status of the minikube cluster.
W1117 20:30:05.712641    3016 start.go:1010] DeleteHost: Docker machine "minikube" does not exist. Use "docker-machine ls" to list machines. Use "docker-machine create" to add a new one.

💣  Unable to start VM
❌  Error: [KVM_CONNECTION_ERROR] Error getting state for host: getting connection: getting domain: error connecting to libvirt socket.: virError(Code=38, Domain=7, Message='Failed to connect socket to '/var/run/libvirt/libvirt-sock': Permission denied')
💡  Suggestion: Have you set up libvirt correctly?
📘  Documentation: https://minikube.sigs.k8s.io/docs/reference/drivers/kvm2/

The current behaviour is not good as it keeps on going even after the permission denied error shows up. 

Error:/usr/bin/virsh domcapabilities --virttype kvm failed:
error: failed to connect to the hypervisor
error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': Permission denied Fix:Follow your Linux distribution instructions for configuring KVM Doc:https://minikube.sigs.k8s.io/docs/reference/drivers/kvm2/}

Personally would like for minikube to directly terminate after it shows up with the error.

This tie in to my previous comment as the code does not treat fatal error such as this as a show stopper. The code must be refactored to take into account if any error does occur there should be some kind of flag to indicate that the code must stop.

nanikjava commented 4 years ago

Idea of how the checking and failing will work https://github.com/nanikjava/minikube-1/commit/3be173faf6e1606459f2ec1bbe3a7e148bda73b8

josedonizetti commented 4 years ago

@nanikjava can you create a PR with the idea? Mark is as WIP, easier to give feedback on the PR.

priyawadhwa commented 4 years ago

@nanikjava, are you still working on this issue?

nanikjava commented 4 years ago

@nanikjava, are you still working on this issue?

Yes I will work on this as I have created an early proto of the idea. Thanks

nanikjava commented 4 years ago

@nanikjava can you create a PR with the idea? Mark is as WIP, easier to give feedback on the PR.

@josedonizetti will do.

nanikjava commented 4 years ago

PR submitted https://github.com/kubernetes/minikube/pull/6120

fejta-bot commented 4 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

bushraKhan23 commented 4 years ago

❌ [KVM_CONNECTION_ERROR] Failed to start kvm2 VM. "minikube start" may fix it. driver start: creating network: getting libvirt connection: error connecting to libvirt socket.: virError(Code=38, Domain=7, Message='Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory')

medyagh commented 4 years ago

is anyone still interested to do this PR ?

@nanikjava

bushraKhan23 commented 4 years ago

yes. me trying

On Tue, May 12, 2020 at 4:07 AM Medya Ghazizadeh notifications@github.com wrote:

is anyone still interested to do this PR ?

@nanikjava https://github.com/nanikjava

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/kubernetes/minikube/issues/5617#issuecomment-627012293, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHZBLE74UZZRZHDJSXHAH5LRRCAKXANCNFSM4JANEG6A .

medyagh commented 4 years ago

@bushraKhan23 excellent. let me know if you needed any PR review

priyawadhwa commented 4 years ago

/remove-lifecycle stale

fejta-bot commented 4 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

fejta-bot commented 4 years ago

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle rotten

medyagh commented 4 years ago

@bushraKhan23 still intereted?

priyawadhwa commented 3 years ago

This issue is open for anyone who is interested in taking it on!

prezha commented 3 years ago

could be solved together with https://github.com/kubernetes/minikube/issues/10070

/assign

Undefeated22 commented 1 year ago

did'nt helped