search

QubesOS / qubes-issues

The Qubes OS Project issue tracker
https://www.qubes-os.org/doc/issue-tracking/
543 stars 48 forks source link

Use in-vm kernel for Whonix (and possibly Kicksecure) #9570

Open hoppity2 opened 2 weeks ago

hoppity2 commented 2 weeks ago

How to file a helpful issue

Qubes OS release

Qubes 4.2

Brief summary

The added kernel boot parameters from Kicksecure's/Whonix's security-misc don't get added to the kernel upon VM boot.

Steps to reproduce

These can be reproduced in the templates as well as other VMs in general, but I'll just use template VMs here for simplicity.

  1. Startup a Whonix GW, Whonix WS and Debian (standard) template. (Assumption all have kernels provided by Dom0.)
  2. Notice the kernel boot parameters or lack of boot parameters in /etc/default/grub.d, in particular 40_cpu_mitigations.cfg and 40_kernel_hardening.cfg. ls /etc/default/grub.d
  3. Open sudo journalctl -r search for the start of the boot in the logs (search for: Command line:).
  4. Notice the lack of extra kernel boot parameters.
  5. Clone the Debian standard template.
  6. In the cloned Debian standard template install security-misc from Kicksecure/Whonix.
  7. Reboot this template, again still with the Dom0 provided kernel notice again how the parameters listed above are missing.
  8. In the cloned Debian standard template settings set it to use the template provided kernel (provided by qube). Apply this. You may need to change the HVM virtualization to get it to boot.
  9. Restart the cloned Debian standard template with security misc in.
  10. Look for the kernel parameters in journalctl for this boot, notice that the extra kernel params in /etc/default/grub.d are present.

For example:

Expected behavior

If I install security-misc or if it's already installed the kernel boot parameters are applied irrespective if you are using a kernel in the template or one provided by Dom0.

Actual behavior

When using the Dom0 provided kernel in a VM it doesn't apply the security-misc's kernel parameters located in /etc/default/grub.d.

A workaround for this is to simply copy the boot params into Dom0 and set them with qvm-prefs [VM NAME] kernelopts "[KERNEL PARAMS]". However unfortunately kernelopts can only hold up to a maximum of 512 characters, so there isn't enough characters to fit them all in, for future proofing in the meantime perhaps increase this up to say 5120 characters.

Other

Searched on Qubes issues and Kicksecure security-misc issues, couldn't find a similar issue.

Issue title adjusted based on the decision for the solution: https://github.com/QubesOS/qubes-issues/issues/9570#issuecomment-2468865938

hoppity2 commented 2 weeks ago

@adrelanos Tagging Patrick as it's relevant, this is a Qubes-Whonix related issue.

marmarek commented 2 weeks ago

This is very much expected - if a qube's kernel is provided by dom0 (the default), kernel parameters are set in dom0 (kernelopts property). If qube wants to control its own kernel (including kernel options), there is the "(provided by qube)" option (that you found already).

@adrelanos would you like for Whonix to use kernel from within qube by default? Related #5212

hoppity2 commented 2 weeks ago

Ok, but as a user who knows security-misc is installed I would expect all of those hardening settings to apply. Also it would be good if all of those hardening settings did apply. This functionality should be documented, as it's not something the user would expect and people are lacking that extra security hardening.

Anyway a partial work around is to add those parameters into the Dom0 VM KernelOpts, though as noted not all of them can fit it, max 512 chars.

adrelanos commented 2 weeks ago

@adrelanos would you like for Whonix to use kernel from within qube by default? Related #5212

I would appreciate very much if https://github.com/QubesOS/qubes-issues/issues/5212 was implemented and if Qubes in-VM kernel would become Qubes default (for Debian templates). However, only doing this for Qubes-Whonix might be troublesome.

This is problematic because it would derivate from Qubes Debian kernel default kernel.

This is unmaintainable, because if there are Qubes VM kernel specific crashes, issues, then these would be reported by users to Whonix where I would have no possibility to debug these except saying "please reproduce this in Qubes Debian and report the bug to Qubes".

hoppity2 commented 2 weeks ago

Ok, thanks for the responses. I'm doubtful much progress would be made on https://github.com/QubesOS/qubes-issues/issues/5212 anytime soon as the kernel is reliability critical.

@marmarek what would you suggest so I can enable all of the security-misc kernel boot time hardening options (I can manually workout and test to which ones work)? Again with my suggestion of increasing the length of the kernel opts VM attribute.

marmarek commented 2 weeks ago

I would appreciate very much if #5212 was implemented and if Qubes in-VM kernel would become Qubes default (for Debian templates).

That's not going to happen anytime soon because it's problematic for VMs with PCI devices (especially when using Debian kernel, due to old wifi drivers, but there are also other factors like significantly slower boot time in HVM mode). But those issues do not apply to Whonix, so it could be used there. The main point of #5212 (simplifying setting it) is done already - it's a single option named kernel "provided by qube". This option is regularly tested with both Fedora and Debian with all virt modes (this is how we know the HVM mode is problematic, but PVH works just fine). The test is disabled on Whonix but we can enable it there too (after ensuring the template(s) do have kernel and grub packages installed by default).

adrelanos commented 2 weeks ago

Given this excellent news to me...

@adrelanos would you like for Whonix to use kernel from within qube by default?

Yes, please. That would fix a lot of issues.

marmarek commented 6 days ago

Ok, since Whonix templates are based on minimal flavor, it normally doesn't include kernel and bootloader. So, this change proven to be a bit more complicated. Pull requests linked above should take care of it.