nix-optimise doesn't run in a container of *any* host

rhendric commented 5 years ago

Issue description

Since #48098, the nix-optimise service contains a ConditionVirtualization attribute that prevents the service from running inside a containerized installation of NixOS. The rationale for this change, as stated, is that nix-optimise should run on a NixOS host and not its containers. However, the ConditionVirtualization restriction is too broad, as it disables nix-optimize even when running inside a non-NixOS host. Containerization is, after all, an increasingly popular way of sampling less-mainstream Linux distros from inside a more familiar host, and as such NixOS features shouldn't assume that they're being hosted by NixOS whenever they're running in a container.

Is testing the host OS from a module possible? Wrapping the ConditionVirtualization in a mkIf that does that would satisfy me (just as one example solution—so would other hypothetical solutions that let nix-optimise run under non-NixOS hosts).

Steps to reproduce

Install NixOS as a container in a non-NixOS host (see #9884 for tips)
Set nix.optimise.automatic = true; in configuration.nix
See that nix-optimise doesn't run (systemctl status nix-optimise)

Workaround

It's a hack, but: systemd.services.nix-optimise.unitConfig.ConditionVirtualization = pkgs.lib.mkForce null; will do the trick. (This does still leave an awkward ConditionVirtualization= empty entry in the resulting service file, if you bother to check with systemctl cat nix-optimise... not that it really matters, as far as I can tell, but mkUnset from #63553 would be cleaner than mkForce null.)

Technical details

 - system: `"x86_64-linux"`
 - host os: `Linux 5.1.5-arch1-2-ARCH, NixOS, 19.03.172927.30a82bba734 (Koi)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.2.2`
 - channels(root): `"nixos-19.03.172927.30a82bba734, nixos-unstable-19.09pre173349.07b42ccf2de"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

peterhoeg commented 5 years ago

Isn't the whole idea of containers, that they don't change? Having something change during runtime seems like a terrible idea. If you want to dedup, shouldn't that be done at the time of container creation?

rhendric commented 5 years ago

Wow, that claim surprises me a lot! Maybe the whole point of containers as a devops deployment vehicle is that they don't change, but containers have other uses than that! I would say that their ‘whole idea’ is allowing software to run within a custom Linux environment that is isolated from the host, without the overhead of full machine virtualization. That environment doesn't have to be immutable.

I keep a long-running NixOS container on my dev machine, as a platform for running NixOps deployments and developing and testing Nix configurations to be pushed to servers. What would I get out of recreating that container every time I want to update its Nix store? As a container, there's so little overhead to keeping it running and managing itself; I just put my NixOps management timers in the container, enable system.autoUpgrade, and leave it up all the time. So yeah, it's a container that changes at run time, just like my dev machine changes at run time, just like most computers (that aren't ephemeral, single-app VMs) change at run time. What's so terrible about that, as long as everything in the container is still reproducible and transactional?

peterhoeg commented 5 years ago

Fair enough - I suggest introducing a runInContainer option in the nix-optimise module (defaults to false) and setting the ConditionVirtualization based on that. A proper description that outlines the reasons for setting this to true would be great.

edolstra commented 5 years ago

No, it shouldn't be an option. What matters here is whether the container has its own Nix store. It should be possible to detect this automatically (e.g. by looking at whether /nix/var/nix/daemon-socket is a read-only bind mount) and make nix-optimise conditional on that.

Or just add systemd.services.nix-optimise.enable = false to modules/virtualisation/container-config.nix (since that's the module for containers that share the host's Nix store).

peterhoeg commented 5 years ago

Great idea. ConditionPathIsReadWrite = "/nix/var/nix/daemon-socket" should do the trick.

rhendric commented 5 years ago

I like the idea of doing both—disabling nix-optimise in container-config.nix is the cleanest solution for the most common (?) case of NixOS containers in NixOS hosts, but making the service conditional on the writability of /nix/var/nix/daemon-socket also safeguards against less common cases like, say, a NixOS container running on a host that isn't NixOS but that does have a Nix store that it shares with the container.

NixOS / nixpkgs