Support for docker secrets / _FILE environment variables

[KillerTic]

I tried to do some research, but couldn't find what needs to be done. Unfortunatly the current docker image doesn't support *_FILE environment variables.

I have all my passwords, keys, etc. stored in files, name them in my docker-compose as secrets, give the container access to the needed secrets and then pass them to the environment variable.

For this you usually just append _FILE to any environment variable. For example: defined secret in the docker-compose:

secrets:
  blueriiot_passwd_secret:
    file: $DOCKERDIR/secrets/blueriiot_passwd_secret

in the service for blueriiotmqtt:

environment:
  BlueRiiot__Password_FILE: /run/secrets/blueriiot_passwd_secret
secrets:
  - blueriiot_passwd_secret

This does not work with this image (and one other I use). I guess it has to be configured, that it can be used?

(Thanks for building this, have been using it for ages now and it is just perfect!)

[LordMike]

Interesting - I haven't had any experience with this part of docker before, but from reading, I think the app itself needs to support it.

From reading this - it says

Docker secrets do not set environment variables directly. This was a conscious decision, because environment variables can unintentionally be leaked between containers (for instance, if you use --link).

And sure enough, if I use the following compose file:

version: "3.7"

services:
   ubuntu:
     image: ubuntu
     environment:
       MYSQL_PASSWORD_FILE: /run/secrets/db_password
     secrets:
       - db_password

secrets:
   db_password:
     file: db_password.txt

I can run it like this:

# docker-compose run ubuntu
root@787219d957c1:/# export
 ....
declare -x MYSQL_PASSWORD_FILE="/run/secrets/db_password"
 ....
root@787219d957c1:/#

That is, docker did not convert the MYSQL_PASSWORD_FILE variable into MYSQL_PASSWORD which would contain the contents of that file. The various images you've seen, like mysql where I took this example from, must have some script that sees the MYSQL_PASSWORD_FILE and then creates MYSQL_PASSWORD..

So the only way this is going to work is if I explicitly do something.

The variables you want to affect are all the password ones, right?.. MQTT, BlueRiiot, etc.. ?

[KillerTic]

That is pretty much what my research came to as well. There must be some type of extra script in place. Yes I would mainly use it for passwords, maybe even usernames. Not setting variables like ip addresses, settings, etc.

[KillerTic]

If you look at the git repo of the mysql docker image, you can clearly find the _FILE part in the docker- entrypoint.sh
https://github.com/docker-library/mysql/blob/master/8.0/docker-entrypoint.sh

Seems to be handled here...

file_env() {
    local var="$1"
    local fileVar="${var}_FILE"
    local def="${2:-}"
    if [ "${!var:-}" ] && [ "${!fileVar:-}" ]; then
        mysql_error "Both $var and $fileVar are set (but are exclusive)"
    fi
    local val="$def"
    if [ "${!var:-}" ]; then
        val="${!var}"
    elif [ "${!fileVar:-}" ]; then
        val="$(< "${!fileVar}")"
    fi
    export "$var"="$val"
    unset "$fileVar"
}  

docker_setup_env() {
    # Get config
    declare -g DATADIR SOCKET
    DATADIR="$(mysql_get_config 'datadir' "$@")"
    SOCKET="$(mysql_get_config 'socket' "$@")"

    # Initialize values that might be stored in a file
    file_env 'MYSQL_ROOT_HOST' '%'
    file_env 'MYSQL_DATABASE'
    file_env 'MYSQL_USER'
    file_env 'MYSQL_PASSWORD'
    file_env 'MYSQL_ROOT_PASSWORD'

    declare -g DATABASE_ALREADY_EXISTS
    if [ -d "$DATADIR/mysql" ]; then
        DATABASE_ALREADY_EXISTS='true'
    fi
}

[LordMike]

I may be able to do this through the config framework:

• https://www.nuget.org/packages/Microsoft.Extensions.Configuration.KeyPerFile/
• https://nvd.codes/post/use-docker-secrets-in-asp-net-core

[LordMike]

@KillerTic I've pushed a new version which should reach :dev shortly. It introduces a layer to the config system that automatically reads all settings from /run/secrets.

So you should be able to do this:

version: "3.7"

services:
   blueriiot:
     image: ... blueriiot ...
     environment:
       BlueRiiot__Username: MyUser
     secrets:
       - Blueriiot__Password

secrets:
   Blueriiot__Password:
     file: my_password.txt

Notice that you should not need to specify any environment variables, as they will automatically be read from the /run/secrets directory. If you do set the env var Blueriiot__Password - it will probably be overwritten by the secrets file.

[LordMike]

@KillerTic let me know if it works / doesn't work - I'll close this issue for now.

[KillerTic]

Hmm.. Would like to try it, but my docker build is failing on

ERROR [build 5/5] RUN dotnet publish -c release -o /app -r linux-musl-x64 --self-contained false

What am I doing wrong here? Could you push a image with a dev tag?

Also a question, if I understand your implementation correctly. The secret file is automatically being looked for and the name of the fail has to be the same as the evnvironment variable name, correct? MQTTPassword or BlueriiotPassword
Don't get me wrong, really appreciate, that you actually looked into this. I do like, that I can stay in my naming convention with the docker secrets and then just match them to the environment variable. This kind of "breaks" that concept (if I got it wright). At least the documentation would need to reflect that.

Again, not moaning about it, just pointing out, that it kinda works differently and others might struggle.

[LordMike]

The :dev should already be there - at least the github actions completed successfully.

Not sure why your own build would fail.

With the readily available provider that I can use, I cannot strip suffixes like "_FILE" from the key name. So for now, it will remain as-is. When you confirm it works etc, I can put it into the docs. :)

[KillerTic]

Don't know why I haven't seen that...

All right, just had a play around and can confirm it is working. I have also tried a few different variances:

• Secret name something else and file name like the env-names: Not working
• Secret name as env-names and file name something else: Working
• Secret name as env-names and file name as env-names: Working

Setting the environment variables has no effect at all once the secrets are set.

Really cool you looked at this! Makes the setup just a bit more secure :)

[LordMike]

I've added a text to the readme on this.