why remount-secure even needed? fix insecure mount options at the root, not here

adrelanos commented 1 year ago

Which software / source code sets the initial insecure mount options anyhow?

Could we fix it there instead of adding a band-aid on top?

monsieuremre commented 1 year ago

Well, actually there is a better way. A solution that is no band-aid. And it is modifiying the fstab. There is also another solution, that is direct and involves no remounting. We can write custom systemd mount targets and ship them with the package. The only problem with the latter is, fstab entries take precedence over those units. So we can't harden what is covered in fstab with this way.

monsieuremre commented 1 year ago

Like if the user has a seperate /home volume, we can't harden it with our custom systemd unit. I think the reason behind this is, systemd dynamically overrides our unit with the unit it auto generates from fstab on startup.

adrelanos commented 1 year ago

That doesn't really answer why it's insecure by Debian (or most Linux distributions) default.

monsieuremre commented 1 year ago

Because by default debian and most distributions don't do too much disk partitioning. If you want a seperate tmp or var partition, most of the time you manually configure them when installing. And bind mounting to itself is clever workaround to get all the good mounting option benefits without having the partition itself, but it is really not that surprising that no distro would mount a directory to itself by default.

monsieuremre commented 1 year ago

A very highly unrelated sidenote: the only distribution that I see who ships secure defaults by default is openSUSE. Debian by default ships with very slack file permissions for anything. OpenSUSE however ships with already mostly hardened defaults. Tumbleweed gets a score of 87 on lynis after a default install. Debian and most other distros get like 60. And this isn't surprising, because Debian does not try to be better desktop or server os. Debian tries to be the universal os, compatible with every device ever. So actually makes sense for them to ship slack defaults. A price to pay for universal compatibility.

adrelanos commented 1 year ago

Where is the source code where it says "use these default mount options for this and that partition"?

adrelanos commented 1 year ago

Like if the user has a seperate /home volume, we can't harden it with our custom systemd unit. I think the reason behind this is, systemd dynamically overrides our unit with the unit it auto generates from fstab on startup.

Created https://github.com/Kicksecure/security-misc/issues/161 for it.

adrelanos commented 1 year ago

quote https://lists.freedesktop.org/archives/systemd-devel/2019-December/043845.html

On a standard Debian system, the three tmpfs mounts (/run, /tmp, /dev/shm) already have the nosuid and nodev options – this is hardcoded in mount-setup.c. So you should first figure out why they are not present in your case to begin with.

This might be the referenced source file: https://github.com/hramrach/systemd-1/blob/master/src/core/mount-setup.c

a) The remount-secure.service unit file should run alone. Non-parallel. Until remount-secure.service is done, no other systemd unit files should be run. Is that possible with systemd?

Have it run before systemd (i.e. do it from the initramfs). If I remember correctly, if the initramfs pre-mounts /run &c. with the correct options, then systemd will just use that as-is and won't downgrade to the hardcoded options.

This implies that initramfs (initramfs-tools, dracut) is responsible for mount options too.

monsieuremre commented 1 year ago

Ok I tell you the source. Systemd calls the binart mount with the option --all on boot. And mount, unless the mount options are overriden with the option --option or -o, looks for the options in the fstab file, which by default has the path /etc/fstab. What we can do is:

We modify the mount package. Uninstall the one packaged by debian, package our own, modify the source, make the fstab path default to /something/kicksecure/fstab, and the world is good.
Modify the calling of mount --all, and make it be called with mount --all --fstab /something/kicksecure/fstab.
Don't play any games and just override the fstab, for simplicity.

These would not be necessary for stuff that have no entries in fstab. Like if the use has no seperate partition for var, we can create a var.mount systemd unit and define our default options there and life would be easy. But unfortunately, if there is a var entry in fstab, our unit will be overridden.

adrelanos commented 1 year ago

But dracut earlier also already uses mount? Or do you mean the systemd which already runs during dracut?

I couldn't find yet the mount options that dracut uses.

monsieuremre commented 1 year ago

When you mount, if you do -o or --options, then you define options, overriding what was the default. If you don't override, then mount looks into fstab for the options. When no options there, then it looks to the respective systemd unit. If it doesn't exist too, then it gives out an error. This is my understanding.

adrelanos commented 1 year ago

At least 3 different things we could accomplish here.

A) Harden mount options of newly built Kicksecure (or Whonix) VM images. And/or,
B) Harden mount options of newly built Kicksecure (or Whonix) ISO images. And/or,
C) Harden mount options of arbitrary systems (such as a Debian based server which gets distribution morphed into Kicksecure).

In case of A),

for non-Qubes Kicksecure based, the question is why isn't /etc/fstab hardened by default. It's because it's built by grml-debootstrap's (in Debian) chroot-script.
- We could contribute to grml-debootstrap to harden /etc/fstab as much as possible as upstream accepts as well as add an option --hardened-fstab or --custom-fstab or something like this with an alternative version that hardens it even more (such as in case upstream does not want to mount /home with noexec by default).
in case of Qubes, their /etc/fstab is here and I've recently sent a PR to https://github.com/QubesOS/qubes-core-agent-linux/pull/472 which will maybe be the basis for further improvements.
- Qubes may or may not accept /home mounted with noexec by default.
- Related tickets:
- https://github.com/QubesOS/qubes-issues/issues/5263
- https://github.com/QubesOS/qubes-issues/issues/5329

In case of B),

When installing, it will be probably calamares based. So calamares installer would create /etc/fstab, we'd need to look into how calamares handles that, contribute upstream or write a custom calamares module. But that is blocked until an ISO emerges. (Btw progress was made there recently on ISO building, see deriative-maker help-steps/image-to-iso.)

In case of C),

directly writing to /etc/fstab from security-misc is risky. Hence such a systemd unit + script makes sense.

adrelanos commented 1 year ago

dracut also supports /etc/fstab.sys. Not well documented. But can be seen in dracut source code.

monsieuremre commented 1 year ago

Yeah I don't think that will work. Pushing to debian upstream. Like it might, but the debian team is very unresponsive unless it is a bug fix. It would happen probably, but like, in about 3 debian releases later. We can just make our own good defaults ISO. Which isn't that hard to be honest. I feel like we are on our own for the most part tho.

I don't want to take every opportunity to shill suse, so I'm sorry. But opensuse puts in effort to harden their default fstab entries.

adrelanos commented 1 year ago

Pushing to debian upstream. Like it might, but the debian team is very unresponsive unless it is a bug fix. It would happen probably, but like, in about 3 debian releases later.

Debian isn't a conglomerate. It depends on the individual packages, maintainers. I was talking about grml-debootstrap. Pushing fixes to grml-debootstrap seems entirely doable. I managed to land a few patches in grml-debootstrap. The only condition is that it needs to be clean, high quality code. You can see my commits here: https://github.com/grml/grml-debootstrap/commits/master

It doesn't take long either until it reaches derivative-maker. I copied grml-debootstrap to derivative-maker source code folder. A soft fork if you want so. (Not with the intention splitting from upstream due to disagreement or running a separate project. Just to make the updated code available earlier in derivative-maker waiting for the next major Debian release. No big story.)

grml-debootstrap recently learned arm64 support which was contributed by a Whonix contributor. Upstream grml-debootstrap took even did the cleanup / refactoring commits. Building VM images with EFI and Secure Boot was also recently contributed and merged by another contributor.

Until now, grml-debootstrap upstream is entirely reasonable. I might even be capable of running a "full fork" of grml-debootstrap should upstream disappear which I don't hope because they've been doing great work for years.

Improving grml-debootstrap fstab will result in improved Kicksecure VM images. (Would also be possible to leave grml-debootstrap as is and add a better fstab at the derivative-maker level only but that would be more hacky and less beneficial for the overall ecosystem, less use and testing, less bugs only happening inside Kicksecure and derivatives.)

Non-responsiveness of other Debian maintainers is inapplicable here.

monsieuremre commented 1 year ago

Debian isn't a conglomerate. It depends on the individual packages, maintainers.

You are right. I was talking about changes you would suggest debian to make in their iso installer. In that case they would not be involved anyway.

And also the unresponsiveness of debian maintainers are not meant in a bad way. They are very responsive on security fixes and bug fixes, but thats pretty much it. If they wanted, they could be responsive to feature requests, if they weren't shipping one billion packages in their repositories, which even for them is really impossible to maintain. Instead they choose the way of trying to be the universal os. Which I think is something that holds kicksecure back. There are two things I see that are holding kicksecure back. First is QubesOS compatibility. Which is understandable and there isn't much to do against this. The second is debian. Debian's package freezing is really not we want. Like even if you fix your thing upstream, good luck waiting another 2 years till the next debian, in which that version still might not be in the repos anyway. And they even microcode updates too, between point releases. This is also, a no no for a security system. Actually imma open an issue in a sec with a suggestion.

adrelanos commented 1 year ago

Debian isn't a conglomerate. It depends on the individual packages, maintainers.

You are right. I was talking about changes you would suggest debian to make in their iso installer. In that case they would not be involved anyway.

Btw the installer would be calamares based which is also a project independent of Debian which apparently is popular, used by many Linux distributions nowadays, configurable, supports plugins, is responsive. It's also packaged for Debian.

Any fstab issues could be reported there. However, I am not sure how great the chances are of fstab hardening feature requests to be implemented without also sending a pull request doing the actual implementation. Testing calamares fstab and feature request if applicable would be a helpful contribution nonetheless to see what upstream calamares thinks about this.

(A calamare based installer is entirely realistic. There once was a Whonix-Host installer. Implying that the same would be possible for Kicksecure. The calamares config files can nowadays be found in Kicksecure package live-config-dist. It was deprecated for other reasons because it was based on isolinux bootloader, only supported legacy BIOS booting, EFI booting was unsupported with no known solution on how to fix that. The new upcoming implementation won't have these issues.)

adrelanos commented 1 year ago

A completely different approach would be having no fstab at all. I've had a chat with chatgpt on this topic: https://chat.openai.com/share/0c573e00-06bf-41a1-8f86-010624172bdc

In summary, use bootloader, dracut and systemd mounts to mount the root disk and everything else.

Not sure that is possible with grub. systemd-boot might be capable of that. But systemd-boot isn't signed by Debian yet. [1] For rather mysterious reasons if you read that ticket. [2]

Dunno if the Boot Loader Specification (BLS) is required which grub might not support. I couldn't find a ticket for grub. Seems like some distributions as Fedora are patching grub.

In summary, seems difficult to research and create a stable implementation to me.

[1] https://groups.google.com/g/linux.debian.bugs.dist/c/n0-eRL6eVtQ https://salsa.debian.org/systemd-team/systemd/-/merge_requests/132 [2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=996202

monsieuremre commented 1 year ago

In summary, seems difficult to research and create a stable implementation to me.

Well then I am really pleased to share that I have found a way. Real, no bandaid, no workaround. Just mounting stuff the way we want it. I tested it too. It's just writing mount units, but they have to be under etc.

From the manpage:

If a mount point is configured in both /etc/fstab and a unit file that is stored below /usr, the former will take precedence. If the unit file is stored below /etc, it will take precedence. This means: native unit files take precedence over traditional configuration files, but this is superseded by the rule that configuration in /etc will always take precedence over configuration in /usr.

It means exactly what it means. Tho fstab takes precedence over mount units, mount units under etc take precedence even over fstab. So we just create mount unis under etc. We can even do this under a something.mount.d directory to allow the user to override us. The only thing that is problematic is, the mount options. We can not preemptively know before shipping a mount unit for home if needs bind or not. If we tackle that, I think we got a solid solution.

adrelanos commented 1 year ago

How about /usr/lib/systemd/system folder?

monsieuremre commented 1 year ago

Unfortunately doesn't work. Gets overriden by fstab. You can see it in the manpage section I quoted.

adrelanos commented 1 year ago

The only thing that is problematic is, the mount options. We can not preemptively know before shipping a mount unit for home if needs bind or not.

systemd supports many conditionals: https://www.freedesktop.org/software/systemd/man/latest/systemd.unit.html#Conditions%20and%20Asserts

Such as ConditionPathExists, ConditionPathIsSymbolicLink. ConditionPathIsDirectory, ConditionPathIsMountPoint, ConditionDirectoryNotEmpty and many others.

Does that work? If not, please post a systemd feature request. 1) The feature might exist already anyhow 2) they might implement the feature.

adrelanos commented 1 year ago

I'd still prefer /usr/lib/systemd/system folder.

Then how about /etc/fstab? Well, that's "just a detail". "Just part of the migration plan to this feature."

A) For new Kicksecure (VM) builds: They would come without an /etc/fstab, or with an empty /etc/fstab or ideally if possible with an /etc/fstab that only contains comments. (phase 1)
B) For existing Kicksecure VM builds that are upgraded: If /etc/fstab is unmodified which it probably is for most users, it might be possible to safely delete it to upgrade users to the new method. This could be done at a later time. (phase 2)
C) For distribution morphed installations of Kicksecure such as servers: The migration is more difficult here as somebody or something (an installer) else might have created /etc/fstab. In that case, maybe security-misc should just show a warning during installation and recommend deprecation of /etc/fstab.

These migration details can be tackled later. First priority are the systemd units as a proof of concept.

adrelanos commented 1 year ago

Yeah, a systemd based implementation would be best. We could deploy the more restrictive mount settings by default. (/home mounted as noexec.) If users want to undo that, they could just use the standard systemd drop-in file mechanism to overrule this. This would be documented in the wiki.

Another detail: For Whonix, tb-starter would need a sudoers exception to run a script which mounts Tor Browser's folder (unfortunately resides in /home) with exec. But that's doable. Got similar code for that already.

monsieuremre commented 1 year ago

Actually we kinda need the fstab to read the mount points. At least initially. Else how would we know where to mount /boot for example? If we write mount units under /usr/lib with --bind options, this might be a good enough solution. If there is no partition, it gets binded to itself with the secure defaults. If there is a partition, it gets overriden by fstab anyway, so the unit will do nothing. This won't harden a the /home directory if it is on a seperate partition for example.

adrelanos commented 1 year ago

Yeah, a systemd based implementation would be best. We could deploy the more restrictive mount settings by default. (/home mounted as noexec.) If users want to undo that, they could just use the standard systemd drop-in file mechanism to overrule this. This would be documented in the wiki.

To avoid user complaints and get everyone on board: https://www.kicksecure.com/wiki/Noexec

adrelanos commented 1 year ago

Actually we kinda need the fstab to read the mount points. At least initially. Else how would we know where to mount /boot for example?

That is what I meant by

In summary, seems difficult to research and create a stable implementation to me.

in https://github.com/Kicksecure/security-misc/issues/157#issuecomment-1804328208 which also mentions how that might be possible to go completely fstab-less. (grub kernel parameter, dracut to mount the root disk, systemd to handle all other mounts)

It might be the case to go fstab-less would only be possible for (new) Kicksecure (VM) builds where we know the (simple) partitioning. Next version will be built using grml-debootstrap using --vmefi, GPT partition table, separate EFI partition (unfortunately, would prefer without but impossible), no separate /home or /boot partition needed). In that case, even disk UUID is a non-issue thanks to grml-debootstrap FIXED_DISK_IDENTIFIERS=yes build env var.

I had in mind here a completely static systemd mount files configuration. Nothing auto generated by any scripts. Just simple systemd config snippets. And no fstab at all.

This won't harden a the /home directory if it is on a seperate partition for example.

There could be two different systemd mount drop-ins? One with ConditionPathIsDirectory the other with ConditionPathIsMountPoint?

monsieuremre commented 1 year ago

Addressed in the new pull request. For the most part it is solved.

adrelanos commented 1 year ago

Why not "fix it in grml-debootstrap" as mentioned in https://github.com/Kicksecure/security-misc/pull/152#issuecomment-1813544569?

adrelanos commented 1 year ago

To re-iterate my question.

Why not "fix it in grml-debootstrap" as mentioned in #152 (comment)?

Addressed in the new pull request. For the most part it is solved.

For reference:

https://github.com/Kicksecure/security-misc/pull/165

monsieuremre commented 11 months ago

Please consider closing #137 and #161 to have the topic handled more compact here in this issue alone.

monsieuremre commented 11 months ago

I just want to reiterate. We can actually achieve secure mounting with setting these kernel variables:

systemd.mount-extra=tmpfs:/dev/shm[:tmpfs[:defaults,nodev,nosuid,noexec]]
systemd.mount-extra=udev:/dev[:devtmps[:defaults,nosuid,noexec]]
systemd.mount-extra=tmpfs:/run[:tmpfs[:defaults,nodev,nosuid,noexec]]
systemd.mount-extra=tmpfs:/tmp[:tmpfs[:defaults,nodev,nosuid,noexec]]
systemd.mount-extra=proc:/proc[:proc[:defaults,nodev,nosuid,noexec,hidepid=4]] # opt-in

and also having the drop in config files in etc. This will ensure that everything that is mounted is mounted securely. This solution does not cover creating bind mounts, which cause the most problems. This solution is also not applicable on the current version of debian stable, due to the systemd version being really old.

I think this is direct enough. Would you consider merging it?

adrelanos commented 11 months ago

No.

I plan to fix this at the root in the various build tools:

grml-debootstrap (upstream pull request, command line option probably)
calamares (upstream pull request or custom module)
Qubes (upstream pull request)

If that takes time (or gets refused) to get merged upstream, I would ship a hardened by default /etc/fstab in Kicksecure (and Whonix) builds.

adrelanos commented 11 months ago

Draft Kicksecure /etc/fstab for VMs: /usr/share/doc/security-misc/fstab-vm

TODO:

Not yet tested.
Wondering if PARTLABEL should be used. It might make the config more generic.
Mount /dev/cdrom with nosuid,nodev,noexec?
Should probably mount the EFI partition to /boot/efi with nosuid,nodev,noexec.

monsieuremre commented 11 months ago

Why not mount tmp to tmpfs?
You can't bind /dev or /dev/shm or /run or /proc, so those line are problematic, it just won't really work
Why remounting stuff in fstab?
Why are we binding at all? If we are the operating system we can do anything with the partitions.
We have to make a way to generate fstab correctly and not a hardcoded fstab, because problems will occur
A hardcoded approach is guaranteed to limit us to VMs, which I though kicksecure was supposed to be a non-VM bare metal OS unlike whonix
proc mounts with defaults + and all the hardening like noexec and stuff by default on debian so that line is not necessary
But most importantly, we shouldn't remount at any cost, anything

Wondering if PARTLABEL should be used. It might make the config more generic.

Yes most definetely then we can write a universal fstab, but gotta make sure the partitions are labeled, and thats up to the installer.

Mount /dev/cdrom with nosuid,nodev,noexec

Cdrom is probably a device, so chances are it needs dev. Also chances are there things that execute that are burnt to cd's, like program installers. Tho anything like that is old and not relevant, still this options are mostly non-necessary.

Should probably mount the EFI partition to /boot/efi with nosuid,nodev,noexec.

Yes. Hardcoding this would make the assumption that the computer has UEFI and not legacy bios, or some other thing like coreboot. Not having this would make the assumtion that it is not UEFI. Gotta say, this is better dealt with by modifying the generator.

So the lines for /dev /dev/shm /run (already hardened so not necessary) /proc (not necessary) and tmp (if we mount to tmpfs, which we should) are going to be universal and valid for all systems.

Anything else, binding a partition will result in not mounting that partition and a broken startup. You can't bind /home to /home if /home is its own partition. The clean way is modifying the generation process.

adrelanos commented 11 months ago

monsieuremre:

Why not mount tmp to tmpfs?

Not the Debian default.

/tmp should be persistent as per FHS. Changing that might break applications.

You can't bind /dev or /dev/shm or /run or /proc, so those line are problematic, it just won't really work

Yeah. Untested. This probably needs fixing.

Why remounting stuff in fstab?

Without it doesn't work or does it? If it works without happy to drop the remoutn stuff.

Why are we binding at all? If we are the operating system we can do anything with the partitions.

Kicksecure VM images don't have a lot partitions except (new builds) EFI, grub BIOS and root file system. There are no extra partitions for /home etc because grml-debootstrap (the base raw image creation tool by Kicksecure) doesn't have such a feature.

We have to make a way to generate fstab correctly and not a hardcoded fstab, because problems will occur

A hardcoded approach is guaranteed to limit us to VMs, which I though kicksecure was supposed to be a non-VM bare metal OS unlike whonix This fstab is named fstab-vm so it will only be for Kicksecure VMs. Bare metal we can handle separately once there actually is an ISO for bare metal. (Check forums, now finally big progress here.)

proc mounts with defaults + and all the hardening like noexec and stuff by default on debian so that line is not necessary

Ok.

But most importantly, we shouldn't remount at any cost, anything

Why?

Wondering if PARTLABEL should be used. It might make the config more generic.

Yes most definetely then we can write a universal fstab, but gotta make sure the partitions are labeled, and thats up to the installer.

In this case it's up to grml-deboostrap which as far as I know does not yet set PARTLABEL but one day I want to send a pull request for that. (Already sent one yesterday to correctly label the EFI partition.)

But won't be universal because there is no convention (let alone a standard) for partition labels.

Mount /dev/cdrom with nosuid,nodev,noexec

Cdrom is probably a device, so chances are it needs dev.

I think nodev might work it just means that a file /mnt/cdrom/dev/null would be broken.

(/dev/sda is also "a device".)

Also chances are there things that execute that are burnt to cd's, like program installers. Tho anything like that is old and not relevant, still this options are mostly non-necessary.

But if there was a malicious CD best to lower attack surface with nodev, nosuid and probably even noexec.

(This will break VirtualBox guest installer virtual DVD.)

Should probably mount the EFI partition to /boot/efi with nosuid,nodev,noexec.

Yes. Hardcoding this would make the assumption that the computer has UEFI and not legacy bios, or some other thing like coreboot. Not having this would make the assumtion that it is not UEFI. Gotta say, this is better dealt with by modifying the generator.

Hardcoding is OK for Kicksecure VMs.

Hardware will be handled separately but all knowledge learned and tested from VMs will become handy.

So the lines for /dev /dev/shm /run (already hardened so not necessary) /proc (not necessary) and tmp (if we mount to tmpfs, which we should) are going to be universal and valid for all systems.

Not sure /dev/shm is hardened by default yet with noexec.

/proc might be a systemd (or dracut?) thing and therefore not needed in /etc/fstab.

Anything else, binding a partition will result in not mounting that partition and a broken startup. You can't bind /home to /home if /home is its own partition. The clean way is modifying the generation process.

Hence hardcoded, tailored for Kicksecure VMs only.

monsieuremre commented 11 months ago

/tmp should be persistent as per FHS. Changing that might break applications.

I don't think that's true. A lot of distros ship temp on tmpfs and many more just don't persist the content of /tmp between boots. See here.

Not sure /dev/shm is hardened by default yet with noexec.

It is, but it is not hardened with nodev. I was talking about /run only in the quotation tho.

/proc is not needed in fstab in any case.

At the end it should look more or less like this

/dev/disk/by-uuid/26ada0c0-1165-4098-884d-aafd2220c2c6       /                auto      defaults  0      2

## Mounting correctly in the first place
#
# These are universal and can be in any fstab everywhere
#
udev                                                         /dev             devtmpfs  defaults,nosuid,noexec            0      0
tmpfs                                                        /dev/shm         tmpfs     defaults,nosuid,nodev,noexec      0      0
tmpfs                                                        /tmp             tmpfs     defaults,nodev,nosuid,noexec      0      0

## Already hardened to fullest
# /run

## We are not hardening this one anyway
# /proc

## Binds
/boot                                                        /boot            none      bind,defaults,nosuid,nodev,noexec 0      0
/lib                                                         /lib             none      binf,defaults,nosuid,nodev        0      0
/var                                                         /var             none      bind,defaults,nosuid,nodev        0      0
/var/tmp                                                     /var/tmp         none      bind,defaults,nosuid,nodev,noexec 0      0
/var/log                                                     /var/log         none      bind,defaults,nosuid,nodev,noexec 0      0
/home                                                        /home            none      bind,defaults,nosuid,nodev,noexec 0      0

But I have to say, I'm not a fan of hardcoding the partitions.

adrelanos commented 11 months ago

There is actually the Discoverable Partitions Specification (DPS).

DPS is also supported by systemd-boot and mkosi (by systemd project). Just mentioning this to show that there is traction behind DPS [1] so it is worth implementing this.

With DPS (an image created by mkosi) also the following things will be possible (might be off-topic):

booting without the root device (root=) on the kernel command line, and
without /etc/fstab.

By using DPS, we don't have to hardcode any arbitrary uuids or PARTLABEL but instead follow a specification.

mkosi provides also other security advantages, which I'll mention elsewhere (https://github.com/Kicksecure/security-misc/issues/136) but learning mkosi, checking feasibility and porting derivative-maker (build script) to it will take some time.

[1] history:

/tmp should be persistent as per FHS. Changing that might break applications.

I don't think that's true. A lot of distros ship temp on tmpfs and many more just don't persist the content of /tmp between boots. See here.

True.

monsieuremre commented 11 months ago

I see. Booting without fstab is possible of course and there are also other options like directly booting on uefi (not even using grub even), which can recognize partitions, but that would for example force us to use uefi (normally I would be in support of this 100% if we were just ditching legacy bios, but I wouldn't want to limit usage with coreboot etc.). And this specific one will force us to use guid partition table, which is the superior choice anyway, so let's go absolutely. Tho I would still prefer not handling this with dps because it is very non standard and can be problematic.

Integrating this within derivative-maker would also further strengthen the roots kicksecure already has in debian and thus make it even more difficult to change the base in the future, which eventhough unlikely, I hope that day would come one day.

And also adjusting for a ready to boot live system will be different from bare metal. I think kicksecure should focus on being an OS and not a VM image like whonix.

adrelanos commented 11 months ago

I see. Booting without fstab is possible of course and there are also other options like directly booting on uefi (not even using grub even), which can recognize partitions, but that would for example force us to use uefi (normally I would be in support of this 100% if we were just ditching legacy bios, but I wouldn't want to limit usage with coreboot etc.).

I am not a fan of going 100% EFI / EFIStub only to avoid breaking booting on legacy BIOS. Also complicated to build such a a bootable ISO. This is really hard if the base distribution does not do this by default or at least as a supported option.

Glad you agree though for different reasons (not closing the door for coreboot which I also want to keep open).

And this specific one will force us to use guid partition table, which is the superior choice anyway, so let's go absolutely.

GUID should be fine because compatible with both legacy BIOS and EFI booting. Next version of Kicksecure, Whonix images will be using GUID (and be EFI [and Secure Boot] booting compatible).

Tho I would still prefer not handling this with dps because it is very non standard and can be problematic.

Maybe could be both.

1) We can keep /etc/fstab file and root= kernel parameter with a hardcoded ID, and 2) that ID however would not be arbitrary but based on DPS.

I am operating under the unverified assumption that DPS defines such an ID which then can be used for fstab and kernel parameter.

Integrating this within derivative-maker would also further strengthen the roots kicksecure already has in debian

It would only strengthen Kicksecure's roots in derivative-maker as fstab isn't Debian (or DPS) specific.

And also adjusting for a ready to boot live system will be different from bare metal.

The live ISO already has some additional packages and different kernel parameters. That's already implemented.

I think kicksecure should focus on being an OS and not a VM image like whonix.

That's the plan.

monsieuremre commented 11 months ago

I think I found a more sound way of hardening mount options for non partition directories. We tend to make self binds for directories when there is no partition on disk. This is obviously not optimal, leaves a time slot where it is not secure, adds more complexity within the file structure and is not as fast as having the directory as a partition directly. We can make everything a partition on kicksecure by default, but that would be too restricting for those without very big hard drives.

The file system btrfs, used on openSUSE Tumbleweed, microOS, is also developed by the SUSE team. It is known for the best choice for rolling and immutable distros because it allows snapshots that the user can just roll back into. But also, it allows dynamic subvolumes, which are very very good. I have discovered that they also allow mount options. There is one root partition, but you can have a subpartition for /var and it can have its own snapshots and stuff and it can also have its own mount options, and it is not limited by some arbitrary diskspace and it also does not possibly waste space of other partitions either.

Having made my point a lot of times, I am still in support of having Tumbleweed as the base. But if not that, at the very least, having the default file system btrfs even on debian would be much better for kicksecure, because we can literally quasi declare mount options for directories without binding and without complexity in software and with no extra bloating. Snapshots would not be much necessary probably on debian because nothing gets updated ever, but still that will be possible too.

So I think you should look into btrfs as a potential candidate that kicksecure might use by default as the file system.

adrelanos commented 11 months ago

btrfs sounds interesting for many reasons. I'd like the rollback feature because even if Debian is stable, a release upgrade to the next major version or Kicksecure package upgrades have risks of breaking things. So more simple rollbacks would be nice. That would make testing less scary for testers.

btrfs might also play really nice with mkosi. From its man page:

• btrfs subvolume, with separate subvolumes for /var, /home, /srv, /var/tmp (subvolume)

Porting derivative-maker to mkosi might bring many advantages.

monsieuremre commented 11 months ago

Yes the rollback thing actually bigger than I thought. Because the subvolumes are very useful for rollbacks and snapshots are surprisingly space and time effective. I don't know how it works, but I know that it is seamless. Like when something new is installed or updated, snapshots only include the changes and stuff and user data is not rolled back so there is no data loss when you rollback. This seems to be a whole selling point for Tumbleweed and MicroOS. Mkosi already supports tumbleweed so btrfs support is likely there, and making that with debian should be possible I think too.

adrelanos commented 11 months ago

/run on Qubes isn't in /etc/fstab and yet it's not noexec. Same on Debian. Same for /dev/shm.

https://wiki.debian.org/UsrMerge will help implement this. https://packages.debian.org/usrmerge

I plan to install this package by default in the next Kicksecure release as well as fix any packages still using non-usr-merged paths.

adrelanos commented 11 months ago

nice documentation, needs to be worked through: Securing Debian Manual, Mounting partitions the right way

adrelanos commented 11 months ago

Added this to /etc/fstab:

/var                                                         /var             none      bind,remount,nofail,nosuid,nodev        0      0

Then sudo mount -a.

mount: /var: mount point not mounted or bad option.
       dmesg(1) may have more information after failed mount system call.
mount: (hint) your fstab has been modified, but systemd still uses
       the old version; use 'systemctl daemon-reload' to reload.

/var is a folder. Not a partition.

Unfortunately the bind line cannot be combined with the rest. As much as I would prefer a combined line for brevity, that does not work. Therefore I must revert that.

adrelanos commented 11 months ago

But if using 2 lines, these appear twice when running mount which seems also wrong.

adrelanos commented 11 months ago

It can be 1 line but then remount keyword has to be removed the the bind mounts.

adrelanos commented 11 months ago

mount: /mnt/cdrom: mount point does not exist.

I guess best to comment that line out by default. The other option of pre-creating /mnt/cdrom would probably also work but does not seem very important nowadays.

monsieuremre commented 11 months ago

There are some problems.

First of all, /run is mounted with noexec by default on debian 12. If it is different on a debian qube, that is a qubes os thing and not a debian thing. Debian hardens /run by default 100%, I have tested this once again just right now just to be sure. /dev/shm is not completely hardened on debian by default, true. That's why we should include the line in the fstab.

Secondly, anything that is not on disk should never be bound. Not to itself, not to anywhere. /run is not on disk, it is on ram, it should never be bind mounted. If you want to change the mount options for something like that, just add the lines without bind. For example hardening /dev would be like this:

udev                                                         /dev             devtmpfs  defaults,nosuid,noexec            0      0

and hardening /dev/shm would be done by adding the line:

tmpfs                                                        /dev/shm         tmpfs     defaults,nosuid,nodev,noexec      0      0

and so on. As such, the tmp line is also problematic, but not only in this sense. We are saying that we are binding /tmp to /tmp, which is on disk, which we assume is not a partition but a directory, and we declare the filesystem tmpfs, which implies the filesystem is on ram. Looks like a contradiction to me. So it should be replaced with this:

tmpfs                                                        /tmp             tmpfs     defaults,nodev,nosuid,noexec      0      0

You should never bind it. Binding on tmpfs or other file systems may throttle ram usage and will most certainly slow down the boot and shut down processes significantly. It is also completely unnecessary, because these are mounted either way. The default behavior is, these file systems are to be mounted 100% in any case. Fstab is checked first, if there is a line for it, that is used. If not, the default options are used from the os. If you add a bind line, then the default options are used from the os and your bind is used on top of it, which is very bad especially for these as they live on ram not disk.

The error you received is much expected because you are doing a remount and a bind at the same time. If there is no /var partition, there is nothing to remount.

I would recommend using the defaults option at the beginning. It dynamically sets some important stuff like rw, auto, nouser, and async, which are not always guaranteed to be set if not stated, at least not for everything. Also nofail should not be used when not debugging. We probably want the mounting to happen for sure. If not happening, we want it to fail and not boot.

Also, is this planned to be a complete fstab file? Because there are no lines for the regular real partitions, like the root partition /.

Kicksecure / security-misc

why remount-secure even needed? fix insecure mount options at the root, not here #157