Closed RussellNS closed 3 months ago
Up! This would be awesome for Docker Swarm clusters.
I'll have a look at this, i don't use swarm so will have to setup a test environment.
There is more info available here & here.
Seems it's only relevant to swarm;
About secrets In terms of Docker Swarm services, a secret is a blob of data, such as a password, SSH private key, SSL certificate, or another piece of data that should not be transmitted over a network or stored unencrypted in a Dockerfile or in your application’s source code. In Docker 1.13 and higher, you can use Docker secrets to centrally manage this data and securely transmit it to only those containers that need access to it. Secrets are encrypted during transit and at rest in a Docker swarm. A given secret is only accessible to those services which have been granted explicit access to it, and only while those service tasks are running.
Also relevant to k8s, I guess.
If this is relevant to k8s then it should be next on my list. I can test that i just don't use swarm.
Was this solved?
Because I did a secret test right now with mariadb service.
I opened bash inside a mariadb container
Did #cat /run/secrets/mysecret
the value is correctly there
Did # printenv
it showed
MYSQL_ROOT_PASSWORD_FILE=/run/secrets/mysecret
But connecting to the DB tells me access denied
@dominictayloruk
if you add 01_secret-init.sh
(below) to /scripts/pre-init.d
, it should automatically detect environmental variables named var_name__FILE
(where the key is double-underscore-FILE). Due to the nature of posix sh, we have to use an 'eval' hack for indirect variable reference. If you install bash and update run.sh
and 01_secret-init.sh
to use bash, we can replace eval SECRETFILE="\$${VARNAME}"
with `SECRETFILE=${!VARNAME}, etc.
#!/bin/sh
# logic cribbed from linuxserver.io:
# https://github.com/linuxserver/docker-baseimage-ubuntu/blob/bionic/root/etc/cont-init.d/01-envfile
# iterate over environmental variables
# if variable ends in "__FILE"
for FULLVAR in $(env | grep "^.*__FILE="); do
# trim "=..." from variable name
VARNAME=$(echo $FULLVAR | sed "s/=.*//g")
echo "[secret-init] Evaluating ${VARNAME}"
# set SECRETFILE to the contents of the variable
# Use 'eval hack' for indirect expansion in sh: https://unix.stackexchange.com/questions/111618/indirect-variable-expansion-in-posix-as-done-in-bash
# WARNING: It's not foolproof is an arbitrary command injection vulnerability
eval SECRETFILE="\$${VARNAME}"
# echo "[secret-init] Setting SECRETFILE to ${SECRETFILE} ..." # DEBUG - rm for prod!
# if SECRETFILE exists
if [[ -f ${SECRETFILE} ]]; then
# strip the appended "__FILE" from environmental variable name
STRIPVAR=$(echo $VARNAME | sed "s/__FILE//g")
# echo "[secret-init] Set STRIPVAR to ${STRIPVAR}" # DEBUG - rm for prod!
# set value to contents of secretfile
eval ${STRIPVAR}=$(cat "${SECRETFILE}")
# echo "[secret_init] Set ${STRIPVAR} to $(eval echo \$${STRIPVAR})" # DEBUG - rm for prod!
export "${STRIPVAR}"
echo "[secret-init] Success! ${STRIPVAR} set from ${VARNAME}"
else
echo "[secret-init] ERROR: Cannot find secret in ${VARNAME}"
fi
done
I've just tested the latest version 10.5.8 and run a fresh container with secrets injected and they have created the database with the values i entered to the secret.
The bit needed for Kubernetes in the yaml:
spec:
containers:
- envFrom:
- secretRef:
name: this-is-the-secret-name
It should also work in swarm and most other orchestration systems.
BTW I'm using this on both Ubuntu & Alpine hosts running k3s 1.20.6 on amd64/armhf/arm64
Log from the test run (aarch64)
[i] mysqld not found, creating....
[i] MySQL data directory not found, creating initial DBs
[i] Creating database: kubernetes-secret-test
[i] with character set [utf8mb4] and collation [utf8mb4_general_ci]
[i] Creating user: kubernetes-secret-test with password thispasswordissecret
2021-04-17 20:27:49 0 [Note] /usr/bin/mysqld (mysqld 10.5.8-MariaDB) starting as process 55 ...
2021-04-17 20:27:49 0 [Note] InnoDB: Using Linux native AIO
2021-04-17 20:27:49 0 [Note] InnoDB: Uses event mutexes
2021-04-17 20:27:49 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2021-04-17 20:27:49 0 [Note] InnoDB: Number of pools: 1
2021-04-17 20:27:49 0 [Note] InnoDB: Using ARMv8 crc32 instructions
2021-04-17 20:27:49 0 [Note] mysqld: O_TMPFILE is not supported on /var/tmp (disabling future attempts)
2021-04-17 20:27:49 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728
2021-04-17 20:27:49 0 [Note] InnoDB: Completed initialization of buffer pool
2021-04-17 20:27:49 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2021-04-17 20:27:49 0 [Note] InnoDB: 128 rollback segments are active.
2021-04-17 20:27:49 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2021-04-17 20:27:49 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2021-04-17 20:27:49 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2021-04-17 20:27:49 0 [Note] InnoDB: 10.5.8 started; log sequence number 45118; transaction id 20
2021-04-17 20:27:49 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
2021-04-17 20:27:49 0 [Note] Plugin 'FEEDBACK' is disabled.
2021-04-17 20:27:49 0 [Note] InnoDB: Buffer pool(s) load completed at 210417 20:27:49
2021-04-17 20:27:49 1 [Warning] 'user' entry '@test-db-secrets-695ffb46cc-cc94l' ignored in --skip-name-resolve mode.
2021-04-17 20:27:49 1 [Warning] 'proxies_priv' entry '@% root@test-db-secrets-695ffb46cc-cc94l' ignored in --skip-name-resolve mode.
2021-04-17 20:27:49 1 [Warning] 'user' entry '@test-db-secrets-695ffb46cc-cc94l' ignored in --skip-name-resolve mode.
2021-04-17 20:27:49 1 [Warning] 'proxies_priv' entry '@% root@test-db-secrets-695ffb46cc-cc94l' ignored in --skip-name-resolve mode.
/scripts/run.sh: ignoring or entrypoint initdb empty /docker-entrypoint-initdb.d/*
MySQL init process done. Ready for start up.
exec /usr/bin/mysqld --user=mysql --console --skip-name-resolve --skip-networking=0
2021-04-17 20:27:50 0 [Note] /usr/bin/mysqld (mysqld 10.5.8-MariaDB) starting as process 1 ...
2021-04-17 20:27:50 0 [Note] InnoDB: Using Linux native AIO
2021-04-17 20:27:50 0 [Note] InnoDB: Uses event mutexes
2021-04-17 20:27:50 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2021-04-17 20:27:50 0 [Note] InnoDB: Number of pools: 1
2021-04-17 20:27:50 0 [Note] InnoDB: Using ARMv8 crc32 instructions
2021-04-17 20:27:50 0 [Note] mysqld: O_TMPFILE is not supported on /var/tmp (disabling future attempts)
2021-04-17 20:27:50 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728
2021-04-17 20:27:50 0 [Note] InnoDB: Completed initialization of buffer pool
2021-04-17 20:27:50 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2021-04-17 20:27:50 0 [Note] InnoDB: 128 rollback segments are active.
2021-04-17 20:27:50 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2021-04-17 20:27:50 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2021-04-17 20:27:50 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2021-04-17 20:27:50 0 [Note] InnoDB: 10.5.8 started; log sequence number 45130; transaction id 20
2021-04-17 20:27:50 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
2021-04-17 20:27:50 0 [Note] Plugin 'FEEDBACK' is disabled.
2021-04-17 20:27:50 0 [Note] InnoDB: Buffer pool(s) load completed at 210417 20:27:50
2021-04-17 20:27:50 0 [Note] Server socket created on IP: '::'.
2021-04-17 20:27:50 0 [Warning] 'user' entry '@test-db-secrets-695ffb46cc-cc94l' ignored in --skip-name-resolve mode.
2021-04-17 20:27:50 0 [Warning] 'proxies_priv' entry '@% root@test-db-secrets-695ffb46cc-cc94l' ignored in --skip-name-resolve mode.
2021-04-17 20:27:50 0 [Note] Reading of all Master_info entries succeeded
2021-04-17 20:27:50 0 [Note] Added new Master_info '' to hash table
2021-04-17 20:27:50 0 [Note] /usr/bin/mysqld: ready for connections.
Version: '10.5.8-MariaDB' socket: '/run/mysqld/mysqld.sock' port: 3306 MariaDB Server
Looking to close this issue as secrets can be used in Kubernetes.
Ideally if anyone can test in swarm that would be good.
Fixed, issue now closed.
This got closed but I have two remarks:
Yeah this got closed as the secrets are getting injected although i haven't confirmed on swarm as i don't use swarm but happy to reopen for testing if it's not working for you. In relation to your questions;
1, Yeah the readme doesn't contain any info on secrets, maybe i can add some more info with further testing to confirm. 2,The script folder is empty as it's there for you to include custom scripts, like the one above or any other script to be run before/after the container has initiated. You would bind mount a file into the directory like;
-v /data/mariadb/01_secret-init.sh:/scripts/pre-init.d/01_secret-init.sh
I'll have to spin up a swarm cluster to test as i only use Kubernetes. I'll also re-open the issue as it requires more testing.
Just done a quick check to test the secrets in swarm by creating some secrets and checking they are accessible inside the container
printf "hguyFtgfR4r9R4r76" | docker secret create MYSQL_ROOT_PASSWORD -
printf "wordpressdb" | docker secret create MYSQL_DATABASE -
printf "wordpressuser" | docker secret create MYSQL_USER -
printf "hguyFt6S95dgfR4ryb" | docker secret create MYSQL_PASSWORD -
printf "utf8mb4" | docker secret create MYSQL_CHARSET -
printf "utf8mb4_general_ci" | docker secret create MYSQL_COLLATION -
Check the secrets on the manager
docker secret ls
goku00ipcg2uki9lezb6n6eff MYSQL_CHARSET 22 minutes ago 22 minutes ago
kfpxv8cw0qen3ipa0o5yhsilr MYSQL_COLLATION 22 minutes ago 22 minutes ago
y9qki5mnx0v6mgkq71ow3ecre MYSQL_DATABASE 22 minutes ago 22 minutes ago
bkgnd76dgergp7vvpkssuo4fl MYSQL_PASSWORD 22 minutes ago 22 minutes ago
sov8sp3scwfdwikwhhpefyhif MYSQL_ROOT_PASSWORD 23 minutes ago 23 minutes ago
nkip4r67awrb6qpdnb092ujt4 MYSQL_USER 22 minutes ago 22 minutes ago
Now lets create the service
docker service create \
--name mariadb \
--secret MYSQL_ROOT_PASSWORD \
--secret MYSQL_DATABASE \
--secret MYSQL_USER \
--secret MYSQL_PASSWORD \
--secret MYSQL_CHARSET \
--secret MYSQL_COLLATION \
yobasystems/alpine-mariadb:latest
Check container logs...
[i] mysqld not found, creating....
[i] MySQL data directory not found, creating initial DBs
[i] MySQL root Password: wusee8aiSahyaesa
2022-08-30 10:55:10 0 [Note] /usr/bin/mysqld (server 10.6.8-MariaDB) starting as process 52 ...
2022-08-30 10:55:10 0 [Note] InnoDB: Compressed tables use zlib 1.2.12
2022-08-30 10:55:10 0 [Note] InnoDB: Number of pools: 1
2022-08-30 10:55:10 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
2022-08-30 10:55:10 0 [Note] mysqld: O_TMPFILE is not supported on /var/tmp (disabling future attempts)
2022-08-30 10:55:10 0 [Note] InnoDB: Using Linux native AIO
2022-08-30 10:55:10 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728
2022-08-30 10:55:10 0 [Note] InnoDB: Completed initialization of buffer pool
2022-08-30 10:55:10 0 [Note] InnoDB: 128 rollback segments are active.
2022-08-30 10:55:10 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2022-08-30 10:55:10 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2022-08-30 10:55:10 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2022-08-30 10:55:10 0 [Note] InnoDB: 10.6.8 started; log sequence number 42319; transaction id 14
2022-08-30 10:55:10 0 [Note] Plugin 'FEEDBACK' is disabled.
2022-08-30 10:55:10 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
2022-08-30 10:55:10 1 [Warning] 'user' entry '@05ec50372abd' ignored in --skip-name-resolve mode.
2022-08-30 10:55:10 0 [Note] InnoDB: Buffer pool(s) load completed at 220830 10:55:10
2022-08-30 10:55:10 1 [Warning] 'proxies_priv' entry '@% root@05ec50372abd' ignored in --skip-name-resolve mode.
2022-08-30 10:55:11 1 [Warning] 'user' entry '@05ec50372abd' ignored in --skip-name-resolve mode.
2022-08-30 10:55:11 1 [Warning] 'proxies_priv' entry '@% root@05ec50372abd' ignored in --skip-name-resolve mode.
/scripts/run.sh: ignoring or entrypoint initdb empty /docker-entrypoint-initdb.d/*
MySQL init process done. Ready for start up.
exec /usr/bin/mysqld --user=mysql --console --skip-name-resolve --skip-networking=0
2022-08-30 10:55:11 0 [Note] /usr/bin/mysqld (server 10.6.8-MariaDB) starting as process 1 ...
2022-08-30 10:55:11 0 [Note] InnoDB: Compressed tables use zlib 1.2.12
2022-08-30 10:55:11 0 [Note] InnoDB: Number of pools: 1
2022-08-30 10:55:11 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
2022-08-30 10:55:11 0 [Note] mysqld: O_TMPFILE is not supported on /var/tmp (disabling future attempts)
2022-08-30 10:55:11 0 [Note] InnoDB: Using Linux native AIO
2022-08-30 10:55:11 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728
2022-08-30 10:55:11 0 [Note] InnoDB: Completed initialization of buffer pool
2022-08-30 10:55:11 0 [Note] InnoDB: 128 rollback segments are active.
2022-08-30 10:55:11 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2022-08-30 10:55:11 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2022-08-30 10:55:11 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2022-08-30 10:55:11 0 [Note] InnoDB: 10.6.8 started; log sequence number 42331; transaction id 14
2022-08-30 10:55:11 0 [Note] Plugin 'FEEDBACK' is disabled.
2022-08-30 10:55:11 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
2022-08-30 10:55:11 0 [Note] Server socket created on IP: '0.0.0.0'.
2022-08-30 10:55:11 0 [Note] Server socket created on IP: '::'.
2022-08-30 10:55:11 0 [Warning] 'user' entry '@05ec50372abd' ignored in --skip-name-resolve mode.
2022-08-30 10:55:11 0 [Warning] 'proxies_priv' entry '@% root@05ec50372abd' ignored in --skip-name-resolve mode.
2022-08-30 10:55:11 0 [Note] InnoDB: Buffer pool(s) load completed at 220830 10:55:11
2022-08-30 10:55:12 0 [Note] /usr/bin/mysqld: ready for connections.
Version: '10.6.8-MariaDB' socket: '/run/mysqld/mysqld.sock' port: 3306 MariaDB Server
Secrets not used in container initialisation!!!
Tested another way but still no secrets in the container;
docker service create \
--name mariadb2 \
--replicas 1 \
--secret source=MYSQL_ROOT_PASSWORD,target=mysql_root_password \
--secret source=MYSQL_PASSWORD,target=mysql_password \
--secret source=MYSQL_USER,target=mysql_user \
--secret source=MYSQL_DATABASE,target=mysql_database \
-e MYSQL_ROOT_PASSWORD_FILE="/run/secrets/mysql_root_password" \
-e MYSQL_PASSWORD_FILE="/run/secrets/mysql_password" \
-e MYSQL_USER_FILE="/run/secrets/mysql_user" \
-e MYSQL_DATABASE_FILE="/run/secrets/mysql_database" \
yobasystems/alpine-mariadb:latest
This should be fixed in 10.11.8-8
The following is a docker swarm example, firstly create some secrets then create the service;
printf "hguyFtgfR4r9R4r76" | docker secret create MYSQL_ROOT_PASSWORD -
printf "wordpressdb" | docker secret create MYSQL_DATABASE -
printf "wordpressuser" | docker secret create MYSQL_USER -
printf "hguyFt6S95dgfR4ryb" | docker secret create MYSQL_PASSWORD -
printf "utf8mb4" | docker secret create MYSQL_CHARSET -
printf "utf8mb4_general_ci" | docker secret create MYSQL_COLLATION -
docker service create \
--name mariadb \
--replicas 1 \
--secret source=MYSQL_ROOT_PASSWORD,target=mysql_root_password \
--secret source=MYSQL_PASSWORD,target=mysql_password \
--secret source=MYSQL_USER,target=mysql_user \
--secret source=MYSQL_DATABASE,target=mysql_database \
-e MYSQL_ROOT_PASSWORD_FILE="/run/secrets/mysql_root_password" \
-e MYSQL_PASSWORD_FILE="/run/secrets/mysql_password" \
-e MYSQL_USER_FILE="/run/secrets/mysql_user" \
-e MYSQL_DATABASE_FILE="/run/secrets/mysql_database" \
yobasystems/alpine-mariadb:latest
There is no support for 'Docker Secrets' in this image. The setup for Docker Secrets may be different between Docker Swarm and Docker Compose. However, the end result is the same. A plain text file will end up inside the container at '/run/secrets/' where each file will contain 1 secret (account name, password, etc.). The contents of this plain text file is then available to the entrypoint of the container.
The current build of this container uses these example environment variables:
The code that handles the environment variables appears to be in 'run.sh':
The normal construct for using Docker Secrets is to use the same environment variable name(s), but with '_FILE' appended, thereby allowing the use of both the original environment variable or the path that contains the 'secret' value. The filename used for the secret is made up entirely by the user, but is usually something meaningful. Example:
Would it be possible to change this code to use docker secrets (i.e. parse the specified file in /run/secrets) for the information for these variables?
Proposed Example 'docker-compose.yml':
Proposed Example 'run.sh' Code Change:
Assuming that each 'db_*.txt' file referenced above contained the original example values, then this would allow users to provide both of these with the same result:
and
Also, in the event that a user provides both secret and non-secret values for the same thing (i.e. MYSQL_USER and MYSQL_USER_FILE), the secret value will override the non-secret value in this proposed code change.
Thank you for your attention.
EDIT: Typos (x3).