systemd DynamicUser seems to be ignored

[apiraino]

Hi!

thank you for this pretty useful crate!

I am trying to customize a bit my systemd service, namely I'd like my service to run as unpriviledged user. This user should be created on service installation. This is supported by the DynamicUser directive. My unit is:

$ cat Cargo.toml
...
maintainer-scripts = "debian/"
systemd-units = { enable = true }
...

$ cat fakeuser@.service
...
[Service]
DynamicUser=true
RuntimeDirectory=myservice
WorkingDirectory=/tmp/myservice
...

If I am not wrong, I should expect to have a fakeuser somewhere and the service to be run as fakeuser:fakeuser. However I get inconsistent behaviours, like the files are always installed into the current user directory.

Am I doing something wrong?

Thanks!

[kornelski]

paging @ximon18

[ximon18]

Hi @apiraino,

cargo-deb can include systemd unit files in the deb package it creates but it doesn't implement the features described in them such as DynamicUser, that is done by systemd on the system in which the deb package is installed.

As such this isn't really the right place to ask this kind of question though with a bit of luck perhaps someone here knows the answer.

I've never used the DynamicUser feature but having quickly read http://0pointer.net/blog/dynamic-users-with-systemd.html it seems like your service should run with a dynamically assigned high UID and only have access to very limited parts of the filesystem, extendable through options like RuntimeDirectory that you used.

It's not clear to me from your description if you have verified that the created deb package includes your service file and that when the package is installed the service file is properly installed by apt/dpkg? If I recall correctly either 'dpkg -e' or 'dpkg -x' can be used to inspect the content of the created deb package.

I can only suggest that you check that you the systemd documentation, verify that the version of systemd on the test installation system is at least version 232, and perhaps ask in a systemd support forum/chat/issue tracker for help.

Maybe @kornelski is familiar with the feature and can offer some advice.

[ximon18]

Ah I see that @kornelski referred to me! When I get a chance I'll try to use the DynamicUser feature and see if I can offer any help after that.

[apiraino]

hi @ximon18 thanks for sharing your point of view.

Since this is out of scope for cargo-deb, I'll close this issue (don't worry about it). I think I'll simply park this idea for a while, waiting for some time to investigate by creating a .deb using other tooling (something I can't focus at this time).

regards,

[ximon18]

Hi @apiraino,

Please note that using other tooling to create a deb will likely not help as deb packages are not the issue here, the problem is with systemd which runs on the system on which the deb package is installed on.

Let us know if you solve it, you may help someone else in future if you post back here with a solution.

Thanks,

Ximon

[ximon18]

Hi @apiraino,

I just tried DynamicUser, below is a verbose report on what I did and showing that DynamicUser in my opinion did seem to do something. Apologies for the verbosity but I thought a real example would be more helpful to see.

Here's my service file debian/krill-ubuntu1804.krill.service:

[Unit]
Description=Krill
Documentation=man:krill(1)
After=network.target

[Service]
DynamicUser=true
ConfigurationDirectory=krill
ExecStart=/usr/bin/krill --config=/etc/krill/krill.conf
Type=simple
Restart=on-failure
User=krill
AmbientCapabilities=CAP_NET_BIND_SERVICE
CapabilityBoundingSet=CAP_NET_BIND_SERVICE
LockPersonality=yes
MemoryDenyWriteExecute=yes
PrivateDevices=yes
PrivateTmp=yes
ProtectControlGroups=yes
ProtectHome=yes
ProtectKernelModules=yes
ProtectKernelTunables=yes
ProtectSystem=strict
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6
RestrictNamespaces=yes
RestrictRealtime=yes
StateDirectory=krill/data
SystemCallArchitectures=native
SystemCallErrorNumber=EPERM
RestartSec=10
StartLimitInterval=10m
StartLimitBurst=5

[Install]
WantedBy=multi-user.target

Note that I also use ConfigurationDirectory and StateDirectory.

And I have a postinst file to pre-create the configuration file that my service expects to use:

$ cat debian/postinst 
#!/bin/sh
set -e

KRILL_CONF="/etc/krill/krill.conf"
KRILL_HOME="/var/lib/krill/"
KRILL_DATA="${KRILL_HOME}data/"
KRILL_USER="krill"

generate_password() {
    # Tries not to depend on too many other commmands
    # being installed.
    date | md5sum | awk '{print $1}'
}

create_first_time_configuration() {
    if [ ! -f "${KRILL_CONF}" ]; then
        # generate a token for authenticating with Krill
        GENERATED_TOKEN="$(generate_password)"

        # the systemd service unit ConfigurationDirectory directive doesn't pre-create this directory so we have to
    mkdir -p $(dirname ${KRILL_CONF})

        # generate a config file using our preferred filesystem locations
        # and generated token
        # note: we don't configure Krill to store its PID file under /var/run/
        # because that requires root privileges potentially at least once per
        # boot, and Krill doesn't drop privileges yet so when run as a non-root
        # user has no right to create the file or missing /var/run/subdir.
        # See: https://stackoverflow.com/a/28312577
        krillc config simple \
            --data "${KRILL_DATA}" \
            --token "${GENERATED_TOKEN}" |
            sed -e "s|^\(### log_type.\+\)|\1\nlog_type = \"syslog\"|" \
                > "${KRILL_CONF}"
    fi
}

case "$1" in
configure)
    create_first_time_configuration
    ;;
esac

#DEBHELPER#

Here are the cargo deb specific relevant extracts from my Cargo.toml file:

[package.metadata.deb]
name = "krill"
priority = "optional"
section = "net"
extended-description-file = "debian/description.txt"
license-file = ["LICENSE", "0"]
depends = "$auto, adduser, libssl1.1"
maintainer-scripts = "debian/"
changelog = "debian/changelog" # this will be generated by the pkg workflow
copyright = "Copyright (c) 2019, NLnet Labs. All rights reserved."
assets = [
    ["target/release/krill", "/usr/bin/krill", "755"],
    ["target/release/krillc", "/usr/bin/krillc", "755"],
    ["defaults/krill.conf", "/usr/share/doc/krill/krill.conf", "644"],
    ["doc/krill.1", "/usr/share/man/man1/krill.1", "644"],
    ["doc/krillc.1", "/usr/share/man/man1/krillc.1", "644"],
    ["debian/krill.service.preset", "/lib/systemd/system-preset/50-krill.preset", "644"],
]
systemd-units = { unit-name = "krill", enable = false }

[package.metadata.deb.variants.ubuntu1804]

I invoked cargo deb like so: (as in my case I have multiple variants in the Cargo.toml file I have to use --variant to select the correct configuration to use)

$ cargo deb --variant ubuntu1804 --deb-version 0.7.4 -v
...
    Finished release [optimized] target(s) in 0.08s
info: Compressing '/home/ximon/Documents/code/krill/krill-master/doc/krill.1'
info: Compressing '/home/ximon/Documents/code/krill/krill-master/doc/krillc.1'
info: Stripped '/home/ximon/Documents/code/krill/krill-master/target/release/krill'
info: Stripped '/home/ximon/Documents/code/krill/krill-master/target/release/krillc'
info: - -> usr/share/doc/krill/copyright
info: - -> usr/share/doc/krill/changelog.Debian.gz
info: /home/ximon/Documents/code/krill/krill-master/debian/krill-ubuntu1804.krill.service -> lib/systemd/system/krill.service
info: /home/ximon/Documents/code/krill/krill-master/target/release/krill -> usr/bin/krill
info: /home/ximon/Documents/code/krill/krill-master/target/release/krillc -> usr/bin/krillc
info: /home/ximon/Documents/code/krill/krill-master/defaults/krill.conf -> usr/share/doc/krill/krill.conf
info: /home/ximon/Documents/code/krill/krill-master/debian/krill.service.preset -> lib/systemd/system-preset/50-krill.preset
info: - -> usr/share/man/man1/krill.1.gz
info: - -> usr/share/man/man1/krillc.1.gz
info: Determining augmentations needed for systemd unit krill.service
info: Maintainer script postinst will be augmented with autoscript postinst-systemd-dont-enable
info: Maintainer script postrm will be augmented with autoscript postrm-systemd
info: Maintainer script postinst will be augmented with autoscript postinst-systemd-restart
info: Maintainer script prerm will be augmented with autoscript prerm-systemd-restart
info: Maintainer script postrm will be augmented with autoscript postrm-systemd-reload-only
info: Augmenting maintainer script debian/postinst
info: Augmenting maintainer script debian/preinst
info: Generating maintainer script prerm
info: Augmenting maintainer script debian/postrm
info: compressed/original ratio 7512688/29118976 (25%)
/home/ximon/Documents/code/krill/krill-master/target/debian/krill_0.7.4_amd64.deb

I then tested it in an LXC Ubuntu 18.04 container like so:

$ sudo lxc launch ubuntu:18.04 deleteme
$ sudo lxc file push target/debian/krill_0.7.4_amd64.deb deleteme/tmp/
$ sudo lxc exec deleteme -- /bin/bash
root@deleteme:~# apt install /tmp/krill_0.7.4_amd64.deb
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Note, selecting 'krill' instead of '/tmp/krill_0.7.4_amd64.deb'
The following package was automatically installed and is no longer required:
  libfreetype6
Use 'apt autoremove' to remove it.
The following NEW packages will be installed:
  krill
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/7515 kB of archives.
After this operation, 29.1 MB of additional disk space will be used.
Get:1 /tmp/krill_0.7.4_amd64.deb krill amd64 0.7.4 [7515 kB]
Selecting previously unselected package krill.
(Reading database ... 28741 files and directories currently installed.)
Preparing to unpack /tmp/krill_0.7.4_amd64.deb ...
Unpacking krill (0.7.4) ...
Setting up krill (0.7.4) ...
krill.service is a disabled or a static unit, not starting it.
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...

# systemctl status krill
● krill.service - Krill
   Loaded: loaded (/lib/systemd/system/krill.service; disabled; vendor preset: disabled)
   Active: inactive (dead)
     Docs: man:krill(1)

Now in the case of my service that status is correct, the user is supposed to edit the configuration file and then manually start the service. We can see that the package installation postinst script created the default configuration file:

root@deleteme:~# ls -la /etc/krill/krill.conf 
-rw-r--r-- 1 root root 6877 Sep  2 07:47 /etc/krill/krill.conf

The StateDirectory does NOT yet exist at this point, it is only created when the service unit is started:

root@deleteme:~# ls -la /var/lib/krill
ls: cannot access '/var/lib/krill': No such file or directory

Now let's start the service:

root@deleteme:~# systemctl enable --now krill
Created symlink /etc/systemd/system/multi-user.target.wants/krill.service → /lib/systemd/system/krill.service.
root@deleteme:~# systemctl status krill
● krill.service - Krill
   Loaded: loaded (/lib/systemd/system/krill.service; enabled; vendor preset: disabled)
   Active: activating (auto-restart) (Result: exit-code) since Wed 2020-09-02 07:52:58 UTC; 4s ago
     Docs: man:krill(1)
  Process: 616 ExecStart=/usr/bin/krill --config=/etc/krill/krill.conf (code=exited, status=1/FAILURE)
 Main PID: 616 (code=exited, status=1/FAILURE)

Now while that failed, it did still show that DynamicUser worked, let's see that:

root@deleteme:~# ls -la /var/lib/krill
total 4
drwxr-xr-x 1 root root   8 Sep  2 07:52 .
drwxr-xr-x 1 root root 522 Sep  2 07:52 ..
lrwxrwxrwx 1 root root  21 Sep  2 07:52 data -> ../private/krill/data

So the StateDirectory was created, and when we look at the actual directory we see a high UID was given ownership:

root@deleteme:~# ls -lad /var/lib/private/krill/data
drwxr-xr-x 1 61968 61968 0 Sep  2 07:52 /var/lib/private/krill/data

I can't explain yet why the Krill process, presumably run as user 61968, was unable to write its PID file into a file in /var/lib/private/krill/data where it should have write access:

root@deleteme:~# grep krill /var/log/syslog | grep -i error
Sep  2 07:58:13 deleteme krill[593]: Could not write PID file: File exists (os error 17)
Sep  2 07:58:23 deleteme krill[604]: Could not write PID file: File exists (os error 17)

But if I want to use DynamicUser I'd have to look into that more.

I wouldn't necessarily expect to find trace of the krill user as the systemd DynamicUser= docs say:

The user and group will not be added to /etc/passwd or /etc/group, but are managed transiently during runtime

[apiraino]

@ximon18 thank you so much for digging this out for me, really appreciate it :-) Now you got me curious, I have to solve this. I'll be back with my findings.

[ximon18]

Great, I'm curious too! 😀

[ximon18]

I noticed that I accidentally left 'User=krill' in my systemd service unit file, removing it didn't improve anything for me but I just wanted to point this out.

[apiraino]

ok, so your configuration and the systemd documentaton were fundamental to understand how that worked. Basically I had no idea and piece by piece I figured that out.

In hindsight, the only thing I would have changed in your Unit was removing User=krill (which you just did) because is inferred by the systemd unit name. My config is luckily much simpler than yours as I don't need postinst stuff, I can straight start up the service.

Specifying a specific dynamic user (like I tried in my first comment) should work just fine, here a minimal working example.

The problem arise when also setting custom config dirs (one or more in WorkingDirectory=, RootDirectory=, RootImage=, RuntimeDirectory=, StateDirectory=, CacheDirectory=, LogsDirectory= or ConfigurationDirectory=) which implies renaming all the paths because these variables are appended to these paths. Example:

DynamicUser=yes
RuntimeDirectory=something
LogsDirectory=asdf

my runtime dir will be /run/something and my logs dir /var/log/asdf. And these vars cannot be referenced inside the Unit itself, like:

# this doesnt work!
RuntimeDirectory=somewhere
PIDfile=$RUNTIME_DIRECTORY/service.pid
ExecStop=kill -15 $MAINPID

pretty confusing stuff.

Also, surprising the fact that one cannot set or read env variables in the [Service] section and env vars cannot be expanded but in Exec* directives, apparently all made on purpose to keep you inside the Unit without externalizing your config ¯\(ツ)/¯

The hardening part of your config is incredibly interesting and I have shamefully copied and pasted it :^) basically we have containerized (-ish) services without docker or lxc :tada: :tada:

thanks again for the help :+1:

[ximon18]

Thanks for the nice words but I think the real credit for those systemd unit file settings goes to @wk and @rfc1036 who contributed to the NLnetLabs/routinator deb packaging on which I based the NLnetLabs/krill systemd service file! 🤪

[ohsayan]

Heya there, just bumped into a similar issue.

We need a systemd unit which requires us to create an user, followed by a directory in /var/opt and so forth. Now I added a script in debian/preinst with a series of the commands and adding #DEBHELPER# at the end of the file along with the unit file debian/pkg.service.

However, it seems that cargo-deb is ignoring the preinst script. The logs (passing -v) show that cargo-deb picks up the unit file but not the preinst script. I think I might be misunderstanding something. Any help is appreciated, thanks!

[apiraino]

hi @ohsayan did you configure cargo-deb with maintainer-scripts = "debian/"? I have a postinst script and it is picked by cargo-deb (docs reference here).

[ohsayan]

Hey @apiraino; thanks for the quick response. Yes. I have the following relevant entries set:

maintainer-scripts = "debian/"
systemd-units = { enable = true }

FWIW, I'm running: cargo deb -p pkg from a workspace.

[apiraino]

hm ... then likely related to https://github.com/mmstick/cargo-deb/issues/151 I think

[ximon18]

I just commented on #151 that I'll try and take a look at this when I get time.