Gitea cannot use Postfix sendmail

[hmenke]

Describe the bug

Gitea cannot be configured with sendmail as the mailer when using Postfix (which wraps sendmail with a setuid wrapper). When trying to send a test email this is the result:

Failed to send a testing email to 'no-reply@localhost': gomail: could not send email 1: signal: aborted (core dumped)

To Reproduce

1. Save this configuration as gitea.nix

   
   { config, lib, pkgs, ... }:

{ imports = [ <nixpkgs/nixos/modules/virtualisation/qemu-vm.nix> ];

config = { networking.firewall.enable = false;

services.postfix.enable = true;

services.gitea.enable = true;
services.gitea.disableRegistration = true;
services.gitea.settings = {
  mailer = {
    ENABLED = true;
    MAILER_TYPE = "sendmail";
    FROM = "do-not-reply@example.org";
    SENDMAIL_PATH = "${pkgs.system-sendmail}/bin/sendmail";
  };
};

systemd.services.gitea-add-user = {
  description = "gitea-add-user";
  wants = [ "gitea.service" ];
  wantedBy = [ "multi-user.target" ];
  path = [ pkgs.gitea ];
  script = "${pkgs.gitea}/bin/gitea admin create-user --admin --username test --password totallysafe --email test@localhost";
  serviceConfig = {
    Restart = "always";
    User = "gitea";
    Group = "gitea";
    WorkingDirectory = config.services.gitea.stateDir;
  };
  environment = {
    GITEA_WORK_DIR = config.services.gitea.stateDir;
  };
};

environment.systemPackages = [ pkgs.gitea ];

services.openssh.enable = true;

users.users.root.initialHashedPassword = "";
users.mutableUsers = false;

}; }

2. Then in your terminal run
```console
$ nix-build '<nixpkgs/nixos>' -A vm --arg configuration ./gitea.nix
$ QEMU_NET_OPTS="hostfwd=tcp::3000-:3000" result/bin/run-nixos-vm

3. Navigate your browser to http://localhost:3000/ and login using the credentials

User: test Password: totallysafe

4. Then navigate to “Site Administration > Configuration > SMTP Mailer Configuration” and enter an email address in the “Send Testing Email” fields, e.g. no-reply@localhost

Expected behavior

Get a delivery error or something like that and not a core dump.

Screenshots

N/A

Additional context

I think this might be connected with the fact that Postfix wraps sendmail with a setuid wrapper and the systemd sandbox for Gitea is rather tight. I have not yet found out which system calls and/or capabilities have to be allowed to make this work.

Notify maintainers

@disassembler @aanderse @kolaente @ma27

Metadata Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

• system: "x86_64-linux"
• host os: Linux 5.8.18, NixOS, 20.09.1698.96052f35023 (Nightingale)
• multi-user?: yes
• sandbox: yes
• version: nix-env (Nix) 2.3.7
• nixpkgs: /etc/channels/nixos

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute:
# a list of nixos modules affected by the problem
module: gitea

[hmenke]

A very short reproducer for my hypothesis is this:

$ systemd-run -p PrivateDevices=true -p User=nobody -p Group=nobody /bin/sh -c 'echo "Subject: Hello" | /run/wrappers/bin/sendmail -f no-reply@localhost -i root@localhost'

Here is the log:

-- Logs begin at Wed 2020-11-11 13:36:02 UTC, end at Wed 2020-11-11 13:42:41 UTC. --
Nov 11 13:40:49 nixos systemd[1]: Started /bin/sh -c echo "Subject: Hello" | /run/wrappers/bin/sendmail -f no-reply@localhost -i root@localhost.
Nov 11 13:40:49 nixos sh[1250]: sendmail: /nix/store/v6l2sacryfr88yqq0pq7sia8wfgm9q31-wrapper.c:204: main: Assertion `!(st.st_mode & S_ISGID) || (st.st_gid == getegid())' failed.
Nov 11 13:40:50 nixos sh[1248]: /bin/sh: line 1:  1249 Done                    echo "Subject: Hello"
Nov 11 13:40:50 nixos sh[1248]:       1250 Aborted                 (core dumped) | /run/wrappers/bin/sendmail -f no-reply@localhost -i root@localhost
Nov 11 13:40:50 nixos systemd[1]: run-r9d3bc4db8e3a482390d4f10f49eb822b.service: Main process exited, code=exited, status=134/n/a
Nov 11 13:40:50 nixos systemd[1]: run-r9d3bc4db8e3a482390d4f10f49eb822b.service: Failed with result 'exit-code'.

Very related:

• https://www.mail-archive.com/search?l=postfix-users@postfix.org&q=subject:%22systemd%5C%2FNoNewPrivileges+%5C%2B+postdrop%22&o=newest&f=1
• https://github.com/systemd/systemd/issues/12476

[aanderse]

I'm using postfix with gitea on 20.03 and everything works fine... I probably can't find time to look at this today. I'll try over the next few days.

[hmenke]

The sandbox was updated in https://github.com/NixOS/nixpkgs/commit/dfd32f11f3ff1da571e499ed993dff99037e73bd which is only part of 20.09. Also pinging the “offender”: @Izorkin

[hmenke]

On further inspection, however, it seems as if PrivateDevices (which implies NoNewPrivileges which breaks the setuid) was already present in https://github.com/NixOS/nixpkgs/commit/e42036ee0e77ba7c5cfae572aefa768c06623c64 which is part of 20.03.

[hmenke]

Very related: https://github.com/NixOS/nixpkgs/issues/26611

[aanderse]

I am sorry I checked and I am not using postfix with gitea on 20.03. I was confused with the redmine service which I am using postfix with.

[hmenke]

Sorry for the confusion, I should have made this clearer in my earlier posts. The problem is not PrivateDevices but NoNewPrivileges. However, the latter is implied by a whole host of other sandboxing settings, e.g. PrivateDevices: https://www.freedesktop.org/software/systemd/man/systemd.exec.html#NoNewPrivileges=

The minimum set of sandboxing options which permits usage of sendmail that I was able to quickly narrow down is this:

{
  # Can remain enabled
  systemd.services.gitea.serviceConfig.PrivateMounts = lib.mkForce true;
  systemd.services.gitea.serviceConfig.PrivateTmp = lib.mkForce true;
  systemd.services.gitea.serviceConfig.ProtectControlGroups = lib.mkForce true;
  systemd.services.gitea.serviceConfig.ProtectHome = lib.mkForce true;
  systemd.services.gitea.serviceConfig.ProtectSystem = lib.mkForce "full"; # downgraded from "strict"

  # Have to be disabled
  systemd.services.gitea.serviceConfig.LockPersonality = lib.mkForce false;
  systemd.services.gitea.serviceConfig.MemoryDenyWriteExecute = lib.mkForce false;
  systemd.services.gitea.serviceConfig.NoNewPrivileges = lib.mkForce false;
  systemd.services.gitea.serviceConfig.PrivateDevices = lib.mkForce false;
  systemd.services.gitea.serviceConfig.PrivateUsers = lib.mkForce false;
  systemd.services.gitea.serviceConfig.ProtectClock = lib.mkForce false;
  systemd.services.gitea.serviceConfig.ProtectHostname = lib.mkForce false;
  systemd.services.gitea.serviceConfig.ProtectKernelLogs = lib.mkForce false;
  systemd.services.gitea.serviceConfig.ProtectKernelModules = lib.mkForce false;
  systemd.services.gitea.serviceConfig.ProtectKernelTunables = lib.mkForce false;
  systemd.services.gitea.serviceConfig.RestrictAddressFamilies = lib.mkForce [];
  systemd.services.gitea.serviceConfig.RestrictRealtime = lib.mkForce false;
  systemd.services.gitea.serviceConfig.RestrictSUIDSGID = lib.mkForce false;
  systemd.services.gitea.serviceConfig.SystemCallArchitectures = lib.mkForce "";
  systemd.services.gitea.serviceConfig.SystemCallFilter = lib.mkForce [];
}

However, I don't think that is a viable approach to just disable the entire sandbox only to be able to run sendmail. I wonder whether there is a possibility to specify exceptions to the seccomp filter.

[Izorkin]

@hmenke service gitea with MAILER_TYPE = "sendmail"; not correct work on sandboxing mode.

Custom working patch:

diff --git a/nixos/modules/services/misc/gitea.nix b/nixos/modules/services/misc/gitea.nix
index af80e99746b..bd546f6d538 100644
--- a/nixos/modules/services/misc/gitea.nix
+++ b/nixos/modules/services/misc/gitea.nix
@@ -516,33 +516,16 @@ in
         RuntimeDirectory = "gitea";
         RuntimeDirectoryMode = "0755";
         # Access write directories
-        ReadWritePaths = [ cfg.dump.backupDir cfg.repositoryRoot cfg.stateDir cfg.lfs.contentDir ];
+        ReadWritePaths = [ cfg.dump.backupDir cfg.repositoryRoot cfg.stateDir cfg.lfs.contentDir "/var/lib/postfix/queue/maildrop" ];
         UMask = "0027";
         # Capabilities
         CapabilityBoundingSet = "";
-        # Security
-        NoNewPrivileges = true;
         # Sandboxing
         ProtectSystem = "strict";
         ProtectHome = true;
         PrivateTmp = true;
-        PrivateDevices = true;
-        PrivateUsers = true;
-        ProtectHostname = true;
-        ProtectClock = true;
-        ProtectKernelTunables = true;
-        ProtectKernelModules = true;
-        ProtectKernelLogs = true;
         ProtectControlGroups = true;
-        RestrictAddressFamilies = [ "AF_UNIX AF_INET AF_INET6" ];
-        LockPersonality = true;
-        MemoryDenyWriteExecute = true;
-        RestrictRealtime = true;
-        RestrictSUIDSGID = true;
         PrivateMounts = true;
-        # System Call Filtering
-        SystemCallArchitectures = "native";
-        SystemCallFilter = "~@clock @cpu-emulation @debug @keyring @memlock @module @mount @obsolete @raw-io @reboot @resources @setuid @swap";
       };

       environment = {

[aanderse]

This is going to have impact on a number of services as we continue to harden more and more services.

ping @NixOS/systemd for discussion.

[picnoir]

I do think this is bound to happen as long as we keep on setting these hardening directives locally in NixOS.

I think this kind of matter should directly PRd and discussed upstream: https://github.com/go-gitea/gitea/blob/master/contrib/systemd/gitea.service

It's easy to overlook a niche use case, especially when you're not familiar with the software codebase. The devs are in a better position to dictate what kind of hardening can be performed than we are.

[aanderse]

@NinjaTrappeur I completely agree with your line of thinking. Recently I have been filling PRs (and encouraging others to do so was well) to have NixOS modules use units provided by upstream projects, as well as make changes upstream instead of our tree. I think this is the best way forward.

In the case of gitea the upstream unit is just an example and recommended values can vary depending on configuration. Unfortunately this means we can't directly include it, but we can make sure we track changes in it from release to release being sure to stick close to what upstream suggests.

[andir]

I've seen those upstream unit configuration files and I am worried that this will get us into a situation where the unit files are not doing exactly what we want and might silently break with a package upgrade. We are already letting a lot of stuff slip in our package updates. IMHO we can only accept upstream units if they are a) proven of better quality then what we are doing b) we have an extensive test suite to verify our module is still compatible with the upstream units. Having spent time on the unbound unit I do not see a way to contribute my version of the unit file upstream (since they have other opinions) but theirs isn't really compatible with our way of deploying services. Using their unit here would have be a net loss for us.

Just using the upstream unit is the cheap way out. We should have proper package and module maintainers that take care of adjusting those few lines of configuration when they are upgrading their packages.

[andir]

One more thought: Very often we can get away with DynamicUser=true these days while many of the upstream units still think they need a dedicated user, custom filesystem paths, maybe groups and they might only ever have tested it with e.g. Debian.

[stale[bot]]

I marked this as stale due to inactivity. → More info

[joepie91]

This probably should not have gone stale?

[SuperSandro2000]

Is there a postfix group we can use in combination with #231673 ?