file.* global replacements trailing newline interaction with secrets

[kanashimia]

So basically when you do something like acme_dns cloudflare {file./path/cool-secret} it will error out if the secret contains a trailing newline, so you will need to remove the trailing newline from the file for it to work, which is against the unix convention. I think this is a bug, but not sure if it needs to be handled on the dns adapter side or here. Is it reasonable to just always strip newline for file.* replacements?

[francislavoie]

:thinking: good question... Probably makes sense to strip the newline, yeah. How do other "secrets in files" systems do it (Docker secrets, Systemd secrets) I wonder? I figure if users need to re-add the newline they could do so in the config where they use the placeholder, but it's kinda unwieldy cause we don't transform literal \n in config so you'd need to do something weird like:

acme_dns cloudflare "{file./path/cool-secret}
"

Or somewhat nicer (adding an extra newline because heredocs themselves also strip the final newline):

acme_dns cloudflare <<TXT
    {file./path/cool-secret}

    TXT

:man_shrugging:

[kanashimia]

systemd credentials:

› printf 'abcde' > test
› systemd-run -q --user -P --wait -G -p LoadCredential=abc:/home/kanashimia/test systemd-creds cat abc
abcde
› systemd-run -q --user -P --wait -G -p LoadCredential=abc:/home/kanashimia/test systemd-creds cat abc | cat
abcde%                                                                                                   
› printf 'abcde\n' > test
› systemd-run -q --user -P --wait -G -p LoadCredential=abc:/home/kanashimia/test systemd-creds cat abc | cat
abcde
› systemd-run -q --user -P --wait -G -p SetCredential=abc:testtest systemd-creds cat abc | cat
testtest%                                                                                                
› systemd-run -q --user -P --wait -G -p SetCredential=abc:testtest systemd-creds cat abc      
testtest

% indicates no trailing newline (zsh thing) When you run systemd-creds cat interactively it adds newline for some reason, that can be controlled with --newline=auto|yes|no SetCredential sets credential literally without a trailing newline. LoadCredential provides the secret as is without any stripping. Most users I've seen use file directly with LoadCredentialEncrypted without using systemd-creds, then it is obviously provided as is without any stripping, as systemd just decrypts the file and is done with it. Of course this is to be expected as systemd credentials designed to work as a carrier, applications supposed to do stripping themselves. On the consumer side with native creds support I only know systemd wireguard, but that uses PEM secrets, so whitespace is stripped anyways.

stalwart-mail: This one is interesting because the mechanism is very similar to caddy, it has %{file:/path}% macro analogous to the {file./path} in caddy. It had exactly the same problem, was fixed as per my report just for acme keys, not at macro level: https://github.com/stalwartlabs/mail-server/issues/382

nixos acme, lego: Configured through env variables, which support specifying it literally or from a file, when you specify CLOUDFLARE_DNS_API_TOKEN_FILE=/run/credentials/acme-fixperms.service/cool-secret2 It strips the trailing newline.

[steffenbusch]

I just ran into this issue as well while trying to use acme_dns ionos {file./path/to/api_key.txt} in my Caddyfile. By default (at least on Alma and Rocky Linux), vim will add a newline while maintaining a text file such as api_key.txt:

The convention for Unix text files is that every line is terminated by a newline, and that newlines are line terminators, not line separators.

When Vim saves a buffer as a file, it terminates every line with the end-of-line sequence for that file format, which for Unix is a newline.

Source: https://superuser.com/a/745135

@francislavoie

I figure if users need to re-add the newline they could do so in the config where they use the placeholder, but it's kinda unwieldy cause we don't transform literal \n in config so you'd need to do something weird like:

They could also simply add an additional newline to the file in question.