Closed FlorianLudwig closed 3 years ago
Hi @FlorianLudwig !
Hmm, the current situation is a bit confusing and the above seems to be as well:
Your step 1) is about "none" driver and step 2) is about "docker" driver ?
Maybe this happened because of some misguided suggestions on minikube's part:
❗ 'none' driver reported an issue: the 'none' driver must be run as the root user 💡 Suggestion: For non-root usage, try the newer 'docker' driver
Currently running minikube requires root, this is (or should be) documented: #7963 (The situation on macOS and Windows is a bit different, I'm going to ignore that here.)
And yes this also has security implications - which should be clearly documented - like so:
https://docs.docker.com/engine/install/linux-postinstall/#manage-docker-as-a-non-root-user
If you don’t want to preface the docker command with sudo, create a Unix group called docker and add users to it. When the Docker daemon starts, it creates a Unix socket accessible by members of the docker group.
Warning
The docker group grants privileges equivalent to the root user. For details on how this impacts security in your system, see Docker Daemon Attack Surface.
The podman driver is a bit more straight-forward, in that it still uses sudo rather than a root group.
Using a VM also requires admin privileges to set up*, but then root is inside the VM. So this might be a better option, rather than running privileged containers (as root) ?
* the libvirt
group is also root-equivalent
The current check against not running as root (with sudo), is mostly for the config files.
The files under the user account ended up being owned by root, or even under /root
...
Only the "none" driver currently runs as root, since it is not intended for users but for CI https://minikube.sigs.k8s.io/docs/drivers/none/
The "docker" driver is somewhere half-way, so it runs minikube as the user but docker as root. Running all of Kubernetes as the user is an interesting project, but currently outside the scope.
You can check out the "Usernetes" project, if that is what you are after here ?
Ultimately we probably want to run also the none driver as non-root: #3760
Currently there are bunch of hacks and workarounds to move the config files afterwards. And most of them don't work, due to hardcoded /root paths in for instance the certificates.
So it is more straight-forward to do everything as the user, and then call "sudo" as needed. Even CI systems should not run as root by default, but rather use some functional user.
https://help.ubuntu.com/community/RootSudo
By default, the root account password is locked in Ubuntu
But we should try to avoid misleading statements like these:
https://github.com/kubernetes/minikube/commit/6276c1cb1da7529b1f8fad712f6f9b7be56a7a72
return registry.State{Error: fmt.Errorf("the 'none' driver must be run as the root user"), Healthy: false, Fix: "For non-root usage, try the newer 'docker' driver", Installed: true}
https://github.com/kubernetes/minikube/issues/3760#issuecomment-602111348
I encourage that users of
--driver=none
consider using the newer--driver=docker
, which does not require root access.
They seem to imply that running docker does not require root ?
I'm also not sure that the "docker" driver was ever intended as a substitute for "none" driver ? It was more available as a new alternative on Linux, to run KIND (or KIC) rather than a VM.
Then it spread to other platforms, in case you were already running a Docker Desktop VM... It requires less resources to run one VM than to run two VMs, but you could also choose ours.
Hi @afbjorklund Thank you very much for your long explanation!
First of all, I indeed messed up my bug report - I meant the docker driver, not none
. I fixed the bug report.
Some background: I am looking for a development environment without compromising (too much) on security nor usability - always a hard thing to do.
Options:
none
driver (setup as root) together with a restricted kubernetes account (usable from non-root account)docker
driver (setup as root) but used from non-root accountkvm2
driver (setup as root) but used from non-root accountpodman
driver - I had some issues with this one but that's another story I guessUsing a VM also requires admin privileges to set up*, but then root is inside the VM. So this might be a better option, rather than running privileged containers (as root) ?
- the libvirt group is also root-equivalent
Actually, this is what I ended up using. Without the libvirt group though, setup asked me for my password like 100 times but using kubectl
now works fine.
But back to my original question: Why isn't it allowed to use the docker driver as root?
But we should try to avoid misleading statements like these:
return registry.State{Error: fmt.Errorf("the 'none' driver must be run as the root user"), Healthy: false, Fix: "For non-root usage, try the newer 'docker' driver", Installed: true}
I encourage that users of
--driver=none
consider using the newer--driver=docker
, which does not require root access.
Exactly! This is what I mean with misleading :)
I'm also not sure that the "docker" driver was ever intended as a substitute for "none" driver ? It was more available as a new alternative on Linux, to run KIND (or KIC) rather than a VM.
I didn't know about KIND, might be interesting to look at, thanks!
Then it spread to other platforms, in case you were already running a Docker Desktop VM... It requires less resources to run one VM than to run two VMs, but you could also choose ours.
Actually, I don't run a VM normally. I try to stick to podman as much as I can as it plays a lot more nice with permissions etc.
But back to my original question: Why isn't it allowed to use the docker driver as root?
There is no particular technical reason why the root user wouldn't be allowed to run minikube.
Theoretically you could log in as the root
user, and have files end up under /root
and be fine.
But since it is regarded as a bad practice (logging in as root), we try to "encourage" avoiding it.
The practical issue was that people run sudo minikube
and ended up with unreadable config.
i.e. you would have files created by root, under ~/.minikube
So the next command would be unable to read e.g. config
Ending up with the ugly situation of having to sudo everything, even dashboard and kubectl We wanted the user to be able to work from their usual account, instead of doing "ssh root@"
I didn't know about KIND, might be interesting to look at, thanks!
KIC (the "docker" driver) is our adapation of KIND, it uses the same docker image etc.
The main difference is that KIC is for developers, while KIND is for testing kubernetes.
Both are from Kubernetetes SIGs...
And both use kubeadm
for setup.
Then it spread to other platforms, in case you were already running a Docker Desktop VM... It requires less resources to run one VM than to run two VMs, but you could also choose ours.
Actually, I don't run a VM normally. I try to stick to podman as much as I can as it plays a lot more nice with permissions etc.
But podman is only available for linux, so there are no "other platforms" to talk about
Same problem though, so we expose podman-env to not have to use podman-machine If we didn't, the user would end up with one VM for docker/podman and one VM for k8s ?
Once there is a Podman Desktop product, that could be considered as an option as well.
Ending up with the ugly situation of having to sudo everything, even dashboard and kubectl We wanted the user to be able to work from their usual account, instead of doing "ssh root@"
Isn't that still better than adding the user to the docker or kvm groups?
So maybe:
sudo minikube
and warn about that the config will be stored in root's home - maybe with a flag --yes-i-know-what-i-am-doing
? (quickfix)I think the sudo setup varies a little between the distributions, as well. Some use the same $HOME for both the user and the sudo user (root), while others change to put the sudo files under /root
instead.
Others (looking at you, CentOS) even have different $PATH for sudo and root so it won't find things. Basically it is a hassle to support it, and most users seem fine without running minikube as root ?
Security could be better documented...
It's not really one of the strong points.
What is the reasoning not allowing to run minikube as root? At least behind a flag --yes-i-know-what-i-am-doing
? Setups are quite different and as you pointed out there is no technical reason to not allow it.
This is disappointing. Finally, I just did sudo chmod 0777 /var/run/docker.sock
. And then minikube start
and it started working.
@jaybirdshain : if you are running as root, then you don't need to chmod. And if you are not (root), then using a docker
group is more fine-grained than letting every local user have docker access ?
most users seem fine without running minikube as root ?
This is probably because (when using the docker driver) they are running it as root, because as the OP mentions, the current recommended setup of adding the user to the docker group effectively grants that user root privileges. So the correct phrasing there would be "most users seem fine (or unaware) running minikube (and any other process they run as their logged-in user) as effectively-root".
Minikube is encouraging users to make their systems less secure while actively disabling (for a fairly silly reason, in my opinion) the most obvious option a more security-conscious user might try to work around it.
We don't yet have a good way of running some commands (like stop and start) as root, and some commands (like dashboard or service) as the regular user. The previous setup with sudo minikube
created some config files and caches in the wrong place with the wrong permissions. So that needs to be fixed, preferrably without something silly like the current suggestions from the "none" driver...
It would be nice if it could reach the state of "make && sudo make install", so that you could mix. (Then again that just might be a bad example, some Makefiles are poorly written and don't build everything the first time which means some stuff in the local dir ends up being owned by root.) There is also no good auth mechanism, asking for password every time isn't improving things (it makes it unusable).
The recommendations for docker and podman (and libvirt) are pretty much based on upstream.
But it does required escalated privileges (basically root-access) for the user running minikube...
This needs to be clear from the documentation, even if it is not something that minikube is doing. Currently we don't have a driver that works without it (maybe VirtualBox*?), at least not on Linux.
* VirtualBox does require privileges to install, but I'm not sure it exposes them when running ?
I agree @afbjorklund that allowing user to run as root causes a lot of headaches such as folders and files that are created and will make profile management more difficult,
and I don't think it will be in scope of minikube to run as root user. since we minikube's focus is local kubernetes developement.
kind is going to support rootless Docker
I had the same issue, a quick minikube start --force --driver=docker
fixed it for me
I had the same issue, a quick
minikube start --force --driver=docker
fixed it for me
This is worked for me too.Thanks a lot.
Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale
.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close
.
Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale
I agree @afbjorklund that allowing user to run as root causes a lot of headaches such as folders and files that are created and will make profile management more difficult,
and I don't think it will be in scope of minikube to run as root user. since we minikube's focus is local kubernetes developement.
Due to this I am closing this issue, feel free to respond/reopen if desired.
minikube start --force --driver=docker minikube start --force --driver=kvm2 minikube start --force --driver=virtualbox minikube start --force --driver=vmware workstation
First
sudo usermod -aG docker $USER && newgrp docker
and then
minikube start
I had the same issue, a quick
minikube start --force --driver=docker
fixed it for me
This works fine with me. 😄 minikube version: v1.20.0 btw.
--force
will get the deal done. However, this is HIGHLY UNRECOMMENDED inPROD
Follow this command It will resolve your issue
You don't need to run minikube with sudo.
sudo chmod 666 /var/run/docker.sock
Give permission to the docker socket, and minikube should work as expected
@AvikantSrivastava
sudo chmod 666 /var/run/docker.sock
Give permission to the docker socket, and minikube should work as expected
Just log in as root. Would be more honest as setting mod 666 on the docker socket gives root privileges to all users and applications on your system.
I had the same issue, a quick
minikube start --force --driver=docker
fixed it for me
Thanks.
Steps to reproduce the issue:
Users will end up reading comments like here: https://github.com/kubernetes/minikube/issues/7903#issuecomment-624074810 and setup access to the docker socket without realizing they are granting root access to those users. By doing so minikube encourages less secure systems in practise.