Rootless Kubernetes

[PedroRegisPOAR]

https://kubernetes.io/docs/tasks/administer-cluster/kubelet-in-userns/#running-rootless-kubernetes-directly-on-a-host

Akihiro Suda and Giuseppe Scrivano at FOSDEM 2019: Rootless Kubernetes Running Kubernetes and CRI/OCI Runtimes as an unprivileged user

List:

• https://www.redhat.com/sysadmin/fedora-31-control-group-v2
• https://github.com/dzintars/infra/issues/28#issuecomment-749713920
• https://unix.stackexchange.com/questions/739067/how-to-add-freezer-control-group-to-fedora-36-and-does-it-matter-for-virtuali

TODO: fix this messed transcript. We provide "Usernetes" which is experimental a distribution of rootless kubernetes. Such can be insurable and users home You can just download binary archive from X DoCoMo's release container as usual it is and just unpack the archive and just run run to the seeds and you can do kubectl with that miss and we also provide composed of Yammer for demonstrating shoot much of the crust or retrace kubernetes the cluster is composed of dough casinos cryo node and hinari node and the fronted by extra is enabled by . https://www.youtube.com/watch?v=mmbTeRhFlqw&t=882s

TODO:

In the most ideal scenario where we run rootless podman as non-root where both podman is running as an unprivileged user and the processes running in the container are also running as an unprivileged user.

• Red Hat's Principal Technical Account Manager Brian Smith in Red Hat's YouTube official channel: Overview of Rootless Podman: Part 1 - Understanding Root Inside and Outside a Container https://youtu.be/ZgXpWKgQclc?t=283
• Related: https://www.tutorialworks.com/podman-rootless-volumes/#how-to-allow-a-rootless-podman-container-to-write-to-a-volume

Today (05/17/2021) at work:

WARNING: Running pip as root will break packages and permissions. You should install packages reliably by using venv: https://pip.pypa.io/warnings/venv

I think one of the intentions of file system based capabilities was to allow admins to take control of their security risk profile and drop privileges of apps on their system independent of what application developers do. I suspect that the low rate of adoption for dropping privileges is because the old API made it tedious to do any task and therefore app developers just don't use it. How many apps have you seen that says you need to be root to use this program? This is because its just 1 line of code to check if you are the root user. The programmer probably knew that a specific capability was needed, but chose to take the shortcut instead. I wanted to change that by making an easy to use API. Its easier to accept a 3-4 line patch than one that adds some 20 lines of code. https://people.redhat.com/sgrubb/libcap-ng/

The last kind of app is the ones that run on a non-root account and have dropped all privs. https://people.redhat.com/sgrubb/libcap-ng/

TODO:

• https://www.youtube.com/watch?v=PliT2gjdxvo
• https://github.com/moby/moby/pull/41893
• https://github.com/rootless-containers/usernetes
• https://docs.docker.com/engine/security/rootless/
• https://medium.com/nttlabs/cgroup-v2-596d035be4d7
• https://github.com/microsoft/WSL/issues/6662
• Podman: A more secure way to run containers
• Dockerless, part 3: Moving development environment to containers with Podman
• Secure (and easy) Kubernetes-in-Docker
• Hardening Compiler Flags for NixOS

TODO:

• Adrian Mouat: Tips and Tricks of the Docker Captains
• Adrian Mouat: Tips and Tricks of the Docker Captains
• Adrian Mouat: Tips and Tricks of the Docker Captains
• Related Attaching to a docker container with base image scratch?, Adding bash binary to a “scratch” docker image, Issues using /bin/sh in CMD command in scratch docker container, Create the smallest possible Docker container, Daniël van Gils 05 April 2017 Updated: 11 June 2019 in How to create the smallest possible Docker image for your Golang application.

You can see the actual mappings by viewing the file /proc/self/uid_map inside your container. From: https://www.tutorialworks.com/podman-rootless-volumes/#how-to-allow-a-rootless-podman-container-to-write-to-a-volume

About distroless OCI image:

I like its compactness (smaller than registry.access.redhat.com/ubi8/ubi-micro:8.4!), glibc support, and most importantly, smaller attack surface for intracontainer attacks. I’d call it "semi-scratch.” Adapted from: https://github.com/docker-library/official-images/issues/10218#issuecomment-847432235

The final image is only 155 MB! Also, the attack surface has been reduced. Not even DNF (or microDNF) is installed in the final image. From: Build smaller containers

We periodically update the base image used for control plane images, but we only prioritize the upgrade for Critical and High severity CVEs.

On this specific CVE, it affects apt, but apt is only run during the image build stage for those images, so this won't affect these images running in production.

Starting in v1.19 these images will be built on distroless:static which doesn't include unused components like apt in the final image. From: https://github.com/kubernetes/kubernetes/issues/92616#issuecomment-657743572

Our other significant trouble stemmed from the fact that we are running a ‘distroless’ Nix container on an Amazon Linux 2 host. The NVIDIA container runtime ‘maps’ the driver files into a filesystem location that works for standard Linux distros but not Nix, which roots all system files under /nix/. We had to patch around this by adding /usr/lib64/ to the LD_LIBRARY_PATH variable which otherwise only included /nix/store/ paths. In general, patching around software that isn’t ‘Nix aware’ is a curse of Nix that you tradeoff against Nix’s benefits. Refs.:

• https://canvatechblog.com/supporting-gpu-accelerated-machine-learning-with-kubernetes-and-nix-7c1da8e42f61

Linux Processes and the proc File System

TODO: create a MWE https://github.com/kubernetes/kubernetes/issues/67014#issuecomment-760015899

Related: https://github.com/NixOS/nixpkgs/blob/2cbc110abba6712d7a3d133391dfe6acafc14370/nixos/modules/profiles/hardened.nix

The race to secure Kubernetes at run time