SearXNG service: environmentFile is not effective for uWSGI vassal

xieve commented 8 months ago

Describe the bug

When SearXNG in set to run in uWSGI and specifying services.searx.environmentFile, the environment variables within that file are not set for the SearXNG uWSGI vassal, but only for the searx-init systemd service.

Steps To Reproduce

Add this to system config:

services.searx = {
enable = true;
runInUwsgi = true;
environmentFile = ./searxng.env;
};

Next to system config, create file searxng.env:
```
SEARXNG_SECRET="myverysecretsecret"
```
This should have the same effect as specifying services.searx.settings.server.secret_key = "myverysecretsecret".
Rebuild

Expected behavior

The environment variables should be passed to the SearXNG process. In the example above, this would mean that SearXNG would be starting. However, since SEARXNG_SECRET has not been set, it does not, and the uWSGI logs show this error thrown by SearXNG:

ERROR:searx.webapp: server.secret_key is not changed. Please use something else instead of ultrasecretkey.

Additional context

Since I cannot specify the order of categories via services.searx.settings, I wanted to use services.searx.settingsFile instead, together with services.searx.environmentFile, to separate my secret into a git-crypt encrypted file. Apart from that, I feel like it would be cleaner to be able to utilize this feature that is already built-in to SearXNG (reading the secret from env) instead of the replacement suggested in the services.searx.settings docs.

I feel like it should be possible to read the environment file and then feed that into services.uwsgi.instance.vassals.searx.env array. I'm not 100% sure if the syntax would be the same in all cases.

Notify maintainers

@rnhmjoj @999eagle

Metadata

$ nix run nixpkgs#nix-info -- -m
 - system: `"x86_64-linux"`
 - host os: `Linux 6.1.77, NixOS, 24.05 (Uakari), 24.05.20240226.13aff9b`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.18.1`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Add a :+1: reaction to issues you find important.

xieve commented 8 months ago

I have added this to my personal config:

services.uwsgi.instance.vassals.searx.env = lib.strings.splitString "\n" (lib.readFile ./searxng.env);

and it does work.

rnhmjoj commented 8 months ago

I also use git-crypt and to do this I have set settings.server.secret_key = "@SEARXNG_SECRET@".

Anyway, I agree it make sense to import the environment into uwsgi, but I'm not sure how. Using the systemd environment file works but would expose the secrets to all other vassals, and reading the file into searx.env would copy the secrets to the Nix store; so both options are unsafe.

xieve commented 8 months ago

I also use git-crypt and to do this I have set settings.server.secret_key = "@SEARXNG_SECRET@".

Anyway, I agree it make sense to import the environment into uwsgi, but I'm not sure how. Using the systemd environment file works but would expose the secrets to all other vassals, and reading the file into searx.env would copy the secrets to the Nix store; so both options are unsafe.

Fair. uWSGI also has an envdir option, here is the format spec for that. The searx-init script could have a small section that converts from systemd's EnvironmentFile style to envdir. Maybe something like:

(
  umask 077
  mkdir -p "${runDir}/env"
  # read will parse POSIX backslashes, like (probably similar to) systemd
  while read line; do
    name=$(cut -d '=' -f 1 <<< "$line")
    # sed removes leading and trailing quotes, if present
    value=$(cut -d '=' -f 2- <<< "$line" | sed -e "s/^[\"']//" -e "s/[\"']$//")
    echo "$value" > "${runDir}/env/$name"
  done <"${environmentFile}"
)

I'm not sure whether this will do all the edge cases correctly, but it should be fine most of the time.

Alternatively, searx-init could parse the output from printenv and put that into an envdir.

zoechi commented 3 days ago

I'm running it without runInUwsgi and thought I ran into the same issue but in my case it was a # in the secret (generated by Bitwarden) that the sed expression didn't like.

I recently run into envsubst when trying to install https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/monitoring/prometheus/alertmanager.nix Perhaps this is a more stable approach?

NixOS / nixpkgs