nixos/ddclient: race condition (in systemd?)

[bjornfor]

Describe the bug

There appears to be a race in the ddclient.service startup code, causing it to fail sometimes (very seldomly):

Oct 22 04:56:06 systemd[1]: Starting Dynamic DNS Client...
Oct 22 04:56:07 systemd[1]: ddclient.service: Deactivated successfully.
Oct 22 04:56:07 systemd[1]: Finished Dynamic DNS Client.
Oct 22 04:56:07 systemd[1]: ddclient.service: Consumed 687ms CPU time, received 4.1K IP traffic, sent 1.4K IP traffic.
Oct 22 05:06:09 systemd[1]: Starting Dynamic DNS Client...
Oct 22 05:06:09 llnp7r077wh33yhff5l88yi5p15b1nrf-ddclient-prestart[272874]: install: invalid user 'ddclient'
Oct 22 05:06:09 llnp7r077wh33yhff5l88yi5p15b1nrf-ddclient-prestart[272876]: Traceback (most recent call last):
Oct 22 05:06:09 llnp7r077wh33yhff5l88yi5p15b1nrf-ddclient-prestart[272876]:   File "/nix/store/2pbr37ml3m26k063m4qdap4mkr2nig24-replace-secret/bin/replace-secret", line 22, in <module>
Oct 22 05:06:09 llnp7r077wh33yhff5l88yi5p15b1nrf-ddclient-prestart[272876]:     with open(args.secret_file) as sf, open(args.file, 'r+') as f:
Oct 22 05:06:09 llnp7r077wh33yhff5l88yi5p15b1nrf-ddclient-prestart[272876]:                                        ^^^^^^^^^^^^^^^^^^^^^
Oct 22 05:06:09 llnp7r077wh33yhff5l88yi5p15b1nrf-ddclient-prestart[272876]: FileNotFoundError: [Errno 2] No such file or directory: '/run/ddclient/ddclient.conf'
Oct 22 05:06:09 systemd[1]: ddclient.service: Control process exited, code=exited, status=1/FAILURE
Oct 22 05:06:09 systemd[1]: ddclient.service: Failed with result 'exit-code'.
Oct 22 05:06:09 systemd[1]: Failed to start Dynamic DNS Client.
Oct 22 05:16:21 systemd[1]: Starting Dynamic DNS Client...
Oct 22 05:16:23 ddclient[273752]: SUCCESS:  updating *: good: IP address set to REDACTED
Oct 22 05:16:24 ddclient[273752]: SUCCESS:  updating @: good: IP address set to REDACTED
Oct 22 05:16:24 systemd[1]: ddclient.service: Deactivated successfully.

Steps To Reproduce

1. Enable services.ddclient = { enable = true; other settings; }
2. Run it for days/weeks/months.
3. Watch the above error happen.

Expected behavior

It always works.

Notify maintainers

Metadata

NixOS 24.05.

---

Add a :+1: reaction to issues you find important.

[bjornfor]

ddclient.service uses systemd DynamicUser, and it seems the user isn't set up at the time ExecPreStart runs.

The systemd version in NixOS 24.05:

$ systemctl --version
systemd 255 (255.9)
+PAM +AUDIT -SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -XKBCOMMON +UTMP -SYSVINIT default-hierarchy=unified

[bjornfor]

I noticed that the ExecStartPre= line (with the shell script that fails with install: invalid user 'ddclient') is prefixed with "!".

man systemd.service has this to say about the exclamation mark:

├────────┼─────────────────────────────────────────────────────────────────────────────────┤
│ "!"    │ Similar to the "+" character discussed above this permits invoking command      │
│        │ lines with elevated privileges. However, unlike "+" the "!" character           │
│        │ exclusively alters the effect of User=, Group= and SupplementaryGroups=, i.e.   │
│        │ only the stanzas that affect user and group credentials. Note that this setting │
│        │ may be combined with DynamicUser=, in which case a dynamic user/group pair is   │
│        │ allocated before the command is invoked, but credential changing is left to the │
│        │ executed process itself.                                                        │
├────────┼─────────────────────────────────────────────────────────────────────────────────┤

So it explicitly says that the dynamic user/group is allocated before the command is invoked. But it isn't fully reliable?