rancher / k3os

Purpose-built OS for Kubernetes, fully managed by Kubernetes.
https://k3os.io
Apache License 2.0
3.5k stars 397 forks source link

Full Disk Encryption #124

Open oscartbeaumont opened 5 years ago

oscartbeaumont commented 5 years ago

Is it possible to set up full disk encryption with k3OS? It would be a good feature to add to the distribution so it can be used in more data sensitive environments. Something like LUKS could be used to achieve disk encryption. Thanks for all your awesome work Rancher team.

ctr49 commented 4 years ago

While I would also like to see FDE, I would even go a step further and ask for remote unlock. Imho this is the only method to fully encrypt your disk with "zero knowledge" of the provider hosting the system. @oscartbeaumont is this what you had in mind? Otherwise I'd open a new issue to not hijack yours...

oscartbeaumont commented 4 years ago

Although I originally had in mind just using the terminal to unlock it as I was thinking of using this for a home server, I still would definitely be open to supporting remote unlock as if it were supported I would use it. I tried to implement this myself just after opening this issue but I struggled to successfully boot the system, installing k3os to an encrypted disk wasn't very difficult. I followed guides relating to full disk encryption on Alpine Linux and the bash commands it uses can be added to the install script with a few minor changes.

ctr49 commented 4 years ago

Great, I was thinking along the lines of: https://hamy.io/post/0009/how-to-install-luks-encrypted-ubuntu-18.04.x-server-and-enable-remote-unlocking/ Since an Ubuntu-based kernel ist used for k3s this should be possible in theory.

The only problem I can currently see is how to pass the authorized_keys file to the initramfs at runtime (opposed to storing inside the initramfs which would require every user to roll his/her own initramfs)

I'll open a PR in k3os-kernel once I found a solution for the remote-unlock specific parts.

immanuelfodor commented 4 years ago

Please add a crossreference here to that issue when you do so :)

oscartbeaumont commented 4 years ago

I can't find my old code but I just modified the install.sh specficically here to handle formatting an encrypted system. Then modified here again in the install.sh to allow it to mount that encrypted partition (possibly umounting code also needs to be updated). Packages can be added to the base Dockerfile here but that is Alpine Linux conflicting what @ctr49 said about k3os being Ubuntu based (I don't have enough of an understanding about k3os to know exactly what it is but I assumed Alpine (with a custom kernel) because of the Dockerfiles). I then had problems with Grub and after kinda fixing them I had problems in correctly starting the system if I recall correctly it was due to the k3os startup go lang program. Hopefully, that helps if anyone attempts to implement this feature.

ctr49 commented 4 years ago

unfortunately its not as easy as I thought. yes the kernel comes from Ubuntu. but the initrd is not created the Ubuntu mkinitramfs-way but custom (here in k3os, not in k3os-kernel btw). and init is a symlink to k3os. tbh I will have to spent much more time to dig into this and understand where we could place some hooks...

nogweii commented 4 years ago

Another approach could be looking at the possibility of including clevis into the OS image. Paired with another server running tang, it would allow the system to automatically boot a FDE root.

orange888 commented 4 years ago

Does the RancherOS's approach help?

https://github.com/rancher/os/pull/2545 https://github.com/rancher/os/issues/2551

deeliciouscode commented 2 years ago

Note: This used to be ticket #822, but since it fits here, I am going to close the ticket and leave it here.

Version (k3OS / kernel)

$ k3os --version: k3os version v0.22.2-k3s2r0-16-g17ddc46-dirty

$ uname --kernel-release --kernel-version: 5.4.0-88-generic #99 SMP Tue Oct 5 16:53:38 UTC 2021

Architecture

$ uname --machine: x86_64

Describe the bug

When booting from an encrypted disk, at the pivot_root stage I get a fatal error, the kernel panics.

To Reproduce

  1. clone https://github.com/deeliciouscode/k3os/
  2. checkout 'nolvm' (https://github.com/deeliciouscode/k3os/commit/129508e353e505fb15568ee489c9bf32798eeba7)
  3. in root: make
  4. spin up a VM with the iso built by make
  5. $ sudo /usr/libexec/k3os/install —encrypt-fs —debug /dev/sda [not yet linked to from k3os install]
  6. $ sudo reboot
  7. decrypt sda2
  8. select one of the grub entires
  9. wait for it to fail

Expected behavior

I would have expected that k3os is able to boot without an issue, since I set the flag K3OS_STATE to the device mapper, that points to the encrypted partition and should therefor work as if it was an unencrypted partition.

Actual behavior

Everything is fine up to the point where the pivot_root system call is made (https://github.com/deeliciouscode/k3os/blob/129508e353e505fb15568ee489c9bf32798eeba7/pkg/enterchroot/enter.go#L272). k3os_luks_boot_fail I don't have deep enough kernel knowledge to easily figure out what is happening here. I thought that everything after grub should work just fine, since the K3OS_STATE Label is set on a ext4 file system, only difference beeing that it is a mapper, not a partition itself.

Additional context

We are trying to implement Full Disk Encryption for k3os and were investigating the possibilities in the last couple of weeks. After figuring out how the installation works we were able to add optional encryption to sda2, and the installation can complete successfully. The issues come in after rebooting. Usually if one wants to encrypt the filesystem of a linux system, the disk gets partitioned, the necessary flags get set in the grub config and the mkinitfs config and then an inititfs and a grub.cfg is generated. k3os uses a different approach here. As far as I understand it, the initramfs is generated by the kernel-stage1 Dockerfile and the grub.cfg is hardcoded in the install.sh script. I think it would be preferable to have both generated during install, but especially for the initramfs I am not sure. Adding the required modules should also be possible through k3os.modules as far as I understand right now. I am at a point where I can encrypt sda2, and am also prompted to decrypt it after rebooting. Then I am even able to select a menu entry in grub and as far as I understand the kernel is loaded properly and initrd runs successfully too. Then it takes a while and I get the error described above. It would also be nice to get some input from one of the maintainers why the design decisions relevant here where made this way (the fact that the grub.cfg was hardcoded rather than generated for example.)