Support for docker secrets on first initialization broken

[AlexCherrypi]

Unfortunately the problem described in #385 reappeared again... The exact thing happens, that I describe in my comment on #385

Unfortunately this seems to be broken again...

When built with this docker-compose . yaml it still asks for database configuration. If built with an inline password on the other hand, it works like intended.

---
version: '3.3'

services:
nextcloud-db:
image: mariadb
container_name: nextcloud-db
command: --transaction-isolation=READ-COMMITTED --log-bin=ROW
networks:
- nextcloud-db
restart: always
environment:
MYSQL_ROOT_PASSWORD_FILE: /run/secrets/mysql_root_password
MYSQL_PASSWORD_FILE: /run/secrets/mysql_user_password
 // commented out // MYSQL_PASSWORD: 345Test
MYSQL_DATABASE: nextcloud
MYSQL_USER: nextcloud
MYSQL_INITDB_SKIP_TZINFO: 1
secrets:
- mysql_root_password
- mysql_user_password
--truncated--

nextcloud-app:
image: nextcloud
container_name: nextcloud-app
restart: always
depends_on:
- nextcloud-db
environment:
MYSQL_HOST: nextcloud-db
MYSQL_USER: nextcloud
MYSQL_DATABASE: nextcloud
MYSQL_PASSWORD_FILE: /run/secrets/mysql_user_password
// commented out // MYSQL_PASSWORD: 345Test
secrets:
- mysql_user_password
--truncated--

secrets:
mysql_root_password:
file: /opt/docker/secrets/mysql_root_password
mysql_user_password:
file: /opt/docker/secrets/mysql_user_password`

I would be very glad if someone might help me or point me in the right direction!

[0x326]

Can confirm. Using POSTGRES_PASSWORD_FILE results defaults to the SQLite database whereas using POSTGRES_PASSWORD auto-configures the database properly

[isdnfan]

@ptoulouse in reply to #1201 states NEXTCLOUD_ADMIN_USER and NEXTCLOUD_ADMIN_PASSWORD are absolutely necessary to make DB auto-config work..

[iFreilicht]

I can confirm this! It works perfectly when using regular environment variables, but not when using the _FILE suffix, despite what the documentation says.

@isdnfan I tried that as well, but couldn't reproduce the issue. For me it's working perfectly fine when specifying the values directly in environment variables, and doesn't work at all when a single one of them has the _FILE suffix. The ADMIN variables have no influence on this.

EDIT: Which is very weird, as the docker-entrypoint.sh suggests that the ADMIN variables are indeed a must. Maybe my setup wasn't entirely clean. I'll try again another day.

[tushev]

Any updates?

Currently, with 21.0.2, specifying _FILE secrets works only if NEXTCLOUD_ADMIN_USER and NEXTCLOUD_ADMIN_PASSWORD are provided.

Without _FILE secrets (specifying MYSQL_PASSWORD etc directly in docker-compose) DB autoconfigure works even if ADMIN variables are not set.

[tushev]

Actually, everything works as intended (if I got the things correctly):

https://github.com/nextcloud/docker/blob/05026b029d37fc5cd488d4a4a2a79480e39841ba/21.0/apache/entrypoint.sh#L121

MYSQL_PASSWORD_FILE will be read and loaded only if NEXTCLOUD_ADMIN_USER and NEXTCLOUD_ADMIN_PASSWORD are provided.

This happens because in this case Nextcloud is being installed via occ maintenance:install command.

Otherwise Nextcloud is not installed, and autoconfig.php is used: https://github.com/nextcloud/docker/blob/05026b029d37fc5cd488d4a4a2a79480e39841ba/21.0/apache/config/autoconfig.php#L9 Autoconfig reads only ordinary MYSQL_PASSWORD etc environment variables, which do not exist.

Solution:

Change autoconfig.php so it will use docker secrets

Can anyone please do it? @J0WI @tilosp

[iFreilicht]

I'm actually not sure it's worth it. You can use a single .env file or multiple *.env files to get the same effect with a built-in mechanism: https://docs.docker.com/compose/environment-variables/

It seems more useful to deprecate the _FILE suffixes and recommend using *.env files instead.

[tushev]

Well, that seems to be good alternative as well. Probably the documentation should be updated.

btw, would the environment variables remain in the container? From security point of view, isn't it better to remove sensitive ones from the .env files after configuration has been complete?

[snigge]

I do not agree that _FILE should be deprecated. It’s a security feature which allows passwords in the environment variables to be kept from log files, process tools and env dumps.

.env gives you similar features but the environment variables aren’t obfuscated inside the container.

[isdnfan]

it's definitely bad idea to skip secrets in favor of ENV variables. It is much more secure to use secrets rather then variables and this way should become preferred or at least remain open.

whole story: https://diogomonica.com/2017/03/27/why-you-shouldnt-use-env-variables-for-secret-data/ Short version:

Overall, secrets in ENV variables break the principle of least surprise, are a bad practice, and will lead to the eventual leak of secrets.

[Ninos]

For docker swarm or kubernetes-users we still need the _FILE functionality, otherwise we share secrets/passwords in clear type.. See also: https://docs.docker.com/engine/swarm/secrets/

I think it's enough to search for all *_FILE environments and write the password in the environment variable with removed _FILE from name: e.g. MYSQL_PASSWORD_FILE = /run/secrets/db_password replaces/adds environment variable MYSQL_PASSWORD with content of /run/secrets/db_password.

So you have an easy fix and compatibility with old docker-compose (not using docker secrets & _FILE).

Sry I currently have no time creating a MR :(

[francois-pasquier]

Hi, don't take it wrong but I just tried to install the project by following the README and the _FILE functionality should be removed from the README if it's broken for the moment.

I was surprised to see an issue this old being the culprit in basically preventing an installation.

[J0WI]

Does #1516 work for you?

[Pkuutn]

Issue is still relevant.

[J0WI]

@Pkuutn did you apply the changes in #1516 locally?

[Pkuutn]

Nope I'm using "latest" images from Dockerhub. Shouldn't this patch be applied to image or I should apply it locally and rebuild?

[J0WI]

It's only available from DockerHub after https://github.com/docker-library/official-images/pull/11418 is merged.

[PhilipWhiteside]

From my testing I believe this to be resolved.

I may have overlooked it in the docs, but for anyone else struggling. The use of _FILE must apply to all the DB settings, not just the password. I had MYSQL_USER and MYSQL_DATABASE as plain text env variables. Only MYSQL_PASSWORD_FILE as _FILE. However, the check for using _FILE needs them all to be _FILE, so converting the plain text to secrets resolved the issue.

You can see here it uses && operators, where they all must be true, rather than on a per value check. Personally, I'm fine with having them all as secrets, there is no harm in that approach. https://github.com/nextcloud/docker/blob/905972656ea10c399f8f971b96adbdcbd9460d1f/.config/autoconfig.php#L9

      MYSQL_HOST: db
      MYSQL_USER_FILE: /run/secrets/db_username
      MYSQL_DATABASE_FILE: /run/secrets/db_name
      MYSQL_PASSWORD_FILE: /run/secrets/db_password

Unfortunately for me, you can't set permissions without swarm (I use simple docker compose), so I'd have to make them public files so the permissions inside the container allow 33/www-data to read them. This defeats the point of using secrets over env vars.

[DerFireWal]

Despite all the helpful comments on this topic I'm sill struggling to get the initialisation running.

Snippet from docker-compose.yml:

    environment:
      - NEXTCLOUD_ADMIN_USER_FILE=/run/secrets/nextcloudadmin
      - NEXTCLOUD_ADMIN_PASSWORD_FILE=/run/secrets/nextcloudadminpwd
      - MYSQL_PASSWORD_FILE=/run/secrets/mysqluserpwd
      - MYSQL_DATABASE_FILE=/run/secrets/mysqldb
      - MYSQL_USER_FILE=/run/secrets/mysqluser
      - MYSQL_HOST=nextcloud-mariadb
      ...

Secrets should be readable by www-data inside the Nextcloud container:

root@db35ebf5bc32:/var/www/html#` ls -l /run/secrets/
total 28
-rw-r--r-- 1 root root 10 Jan 10 22:40 mysqldb
-rw-r--r-- 1 root root 18 Jan 11 08:06 mysqlhost
-rw-r--r-- 1 root root 10 Jan 10 22:41 mysqluser
-rw-r--r-- 1 root root 17 Jan 10 20:48 mysqluserpwd
-rw-r--r-- 1 root root 19 Jan 10 23:28 nextcloudadmin
-rw-r--r-- 1 root root 31 Jan 10 23:29 nextcloudadminpwd
-rw-r--r-- 1 root root 13 Jan 10 20:48 smtppwd

Result in config.php:

...
  'dbname' => 'nextcloud',
  'dbhost' => 'nextcloud-mariadb',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
...

Apparently dbuser and dbpassword are not parsed correctly.

Log output:

Initializing nextcloud 23.0.0.10 ...
Initializing finished
New nextcloud instance
Installing with MySQL database
starting nextcloud installation
Error while trying to create admin user: Failed to connect to the database: An exception occurred in the driver: SQLSTATE[HY000] [1045] Access denied for user 'nextcloud'@'nextcloud.nextcloud_nextcloud' (using password: YES)
Trace: #0 /var/www/html/lib/private/DB/ConnectionAdapter.php(69): OC\DB\Exceptions\DbalException::wrap(Object(Doctrine\DBAL\Exception))

I don't understand how the connection string is constructed. dbhostseems to be parsed correctly but isn't used during the DB connection.

All other configuration options set via environment settings or secrets are applied to the corresponding config files without issues (STMP Password..).

I'm aware that there are apparently some open issue regarding docker secrets.

@PhilipWhiteside Do I understand you correctly, that you got this to work (apart from your remark about secrets being readable for everyone)? Any hint on what I might be missing?

[PhilipWhiteside]

@PhilipWhiteside Do I understand you correctly, that you got this to work (apart from your remark about secrets being readable for everyone)? Any hint on what I might be missing?

I did not get secrets to work, as I do not use Docker Swarm just standalone Docker Compose. What I did instead was put the credentials as plain text into a .env file. This keeps them out of the compose file.

philip@nas~/appdata/nextcloud $ cat docker-compose.yml 
version: "3.9"

services:
  nextcloud:
  ...truncated output...
    environment:
      NEXTCLOUD_DATA_DIR: /data
      MYSQL_HOST: db
      MYSQL_USER: ${MYSQL_USERNAME}
      MYSQL_DATABASE: ${MYSQL_DATABASE}
      MYSQL_PASSWORD: ${MYSQL_PASSWORD}
      REDIS_HOST: ${APP_NAME}-cache
      NEXTCLOUD_TRUSTED_DOMAIN: ${NEXTCLOUD_DOMAIN_NAME}
      TRUSTED_PROXIES: traefik
  ...truncated output...

philip@nas~/appdata/nextcloud $ ll -a
total 24K
drwxrwx---  4 philip philip 4.0K Dec 12 16:28 .
drwxrwxr-x 15 philip philip 4.0K Jan 11 09:17 ..
drwxr-xr-x  4 philip philip 4.0K Dec 11 20:16 config
-rwxrwx---  1 philip philip 4.0K Dec 12 08:48 docker-compose.yml
-rwx------  1 philip philip  532 Dec 11 16:58 .env
drwxrwx---  2 philip philip 4.0K Dec 12 16:31 systemd
lrwxrwxrwx  1 philip philip    4 Dec 11 10:35 variables.env -> .env
philip@nas~/appdata/nextcloud $ cat variables.env 
APP_NAME=nextcloud
NEXTCLOUD_DOMAIN_NAME=nextcloud.lan
# Docker secrets does not support permissions without swarm. So have to be world readable. Using vars instead.
MYSQL_DATABASE=nextcloud
MYSQL_USERNAME=xxx
MYSQL_PASSWORD=yyy
MYSQL_ROOT_PASSWORD=zzz
philip@nas~/appdata/nextcloud $ 

[CamilleHbp]

This is still broken. I've followed all the help given. This was very useful, thanks @PhilipWhiteside !

Everything is parsed correctly except for the db user. It prepends the dbtableprefix to the file path instead of prepending it to the username!

Any ideas, have I missed something?

EDIT:

Indeed, I had missed something. You also need to pass the NEXTCLOUD_ADMIN_PASSWORD and NEXTCLOUD_ADMIN_USER by file NOT in clear. ^^ I had forgotten to add the _FILE suffix. These are the mandatory fields (for Postgres): **- NEXTCLOUD_ADMIN_PASSWORD_FILE

• NEXTCLOUD_ADMIN_USER_FILE
• POSTGRES_DB_FILE
• POSTGRES_HOST
• POSTGRES_PASSWORD_FILE
• POSTGRES_USER_FILE**

Now I just have a Previous: PDOException: SQLSTATE[42501]: Insufficient privilege: 7 ERROR: permission denied for schema public LINE 1: CREATE TABLE oc_migrations error to fix

EDIT 2:

I could not find the info in the Docker container's documentation but you can only use Postgres 14, not the latest. This has fixed everything now, yay!

[mabeett]

Hello all,

The problem about accessing the secrets is directly related with secrets permissions, here the scenario:

The code these commits will only work while the www-data user in the container has access to the /run/secret/file_id, ( I mean user/group file permissions).

• ec1af314c23b6be86062532447d4a0cbe1dc8659 ( #2232 ).
• e1b19238615970db8d0e8bf3d15601613da28adc (#1946).
• d447c5793a41537f737f822f0d5bcd294ff28eb9 (#1614).
• 2857b4b3f1d3aecbb80d1ec5497ff8a2801ef44a (#1516).
• 5d01cebeab368a6f528570236fee454448a24297 (#1471).

From what is see the compose-spec admits remaping the gid and uid for the secret, but

Note that the uid, gid, and mode attributes are implementation specific. So, not supported in the compose plugin while supported by docker swarm.

At the end a demo.

The entrypoint in the image handles secrets files generating a VAR for VAR_FILE , and normally has no problem because it is running as root by default, but the logic in PHP code takes precedence searching for the file defined in the variables considered in the mentioned commits.

I'll be working during my spare time in order to honor the precedence of secrets files over variable definition, as indicated in the README, but from the entrypoint which is executed as root. Other strategies could be applied ( cc @J0WI ).

About the dependency on NEXTCLOUD_ADMIN_USER ( or NEXTCLOUD_ADMIN_USER_FILE ), NEXTCLOUD_ADMIN_PASSWORD ( or NEXTCLOUD_ADMIN_PASSWORD_FILE ). the variables are a must in order to enter in the details of the installation (#2223, #1905,..)

https://github.com/nextcloud/docker/blob/75e1b80ba4d448e9bc82876d0be826e9c7fa2c9a/docker-entrypoint.sh#L190-L250

demo

Files:

```sh $ ls -ln total 12 -rw-r--r-- 1 1000 1000 1119 Aug 18 17:30 docker-compose.yml -rw------- 1 1000 1000 20 Aug 18 17:08 mysql_root_password -rw------- 1 1000 1000 20 Aug 18 17:08 mysql_user_password ``` `docker-compose.yml` ```yaml --- version: '3.3' services: nextcloud-db: image: mariadb container_name: nextcloud-db command: --transaction-isolation=READ-COMMITTED --log-bin=ROW # networks: # - nextcloud-db restart: always environment: MYSQL_ROOT_PASSWORD_FILE: /run/secrets/mysql_root_password MYSQL_PASSWORD_FILE: /run/secrets/mysql_user_password # // commented out // MYSQL_PASSWORD: 345Test MYSQL_DATABASE: nextcloud MYSQL_USER: nextcloud MYSQL_INITDB_SKIP_TZINFO: 1 secrets: - mysql_root_password - mysql_user_password nextcloud-app: image: nextcloud container_name: nextcloud-app restart: always depends_on: - nextcloud-db ports: - 80:80 environment: MYSQL_HOST: nextcloud-db MYSQL_USER: nextcloud MYSQL_DATABASE: nextcloud MYSQL_PASSWORD_FILE: /run/secrets/mysql_user_password # // commented out // MYSQL_PASSWORD: 345Test secrets: - mysql_user_password secrets: mysql_root_password: file: ./mysql_root_password mysql_user_password: file: ./mysql_user_password ```

Execution

```sh $ docker compose up -d WARN[0000] /home/debian/nextcloud/nc2/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion [+] Running 3/3 ✔ Network nc2_default Created 0.1s ✔ Container nextcloud-db Started 2.3s ✔ Container nextcloud-app Started ``` ```sh $ docker compose exec -it nextcloud-app /bin/bash -c 'curl -q http://localhost/' > /dev/null WARN[0000] /home/debian/nextcloud/nc2/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 6655 100 6655 0 0 5903 0 0:00:01 0:00:01 --:--:-- 5905 ``` ```sh $ docker compose exec -it nextcloud-app /bin/bash -c 'cat /var/www/html/data/nextcloud.log' WARN[0000] /home/debian/nextcloud/nc2/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion {"reqId":"UjaIPqzPiap1ZMsi7TG8","level":3,"time":"2024-08-18T17:50:30+00:00","remoteAddr":"127.0.0.1","user":"--","app":"PHP","method":"GET","url":"/","message":"fopen(/var/www/html/config/config.php): Failed to open stream: No such file or directory at /var/www/html/lib/private/Config.php#221","userAgent":"curl/7.88.1","version":"","data":{"app":"PHP"}} ``` ```sh $ docker compose exec -it nextcloud-app /bin/bash -c 'ps faxu' WARN[0000] /home/debian/nextcloud/nc2/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 117 5.5 0.3 8100 3940 pts/0 Rs+ 17:52 0:00 ps faxu root 1 0.0 4.3 406012 43416 ? Ss 17:49 0:00 apache2 -DFOREGROUND www-data 92 0.0 1.0 406052 10348 ? S 17:50 0:00 apache2 -DFOREGROUND www-data 93 0.1 5.1 408640 51936 ? S 17:50 0:00 apache2 -DFOREGROUND www-data 94 0.0 1.0 406052 10348 ? S 17:50 0:00 apache2 -DFOREGROUND www-data 95 0.0 1.0 406052 10348 ? S 17:50 0:00 apache2 -DFOREGROUND www-data 96 0.0 1.0 406052 10348 ? S 17:50 0:00 apache2 -DFOREGROUND www-data 103 0.0 1.0 406052 10348 ? S 17:50 0:00 apache2 -DFOREGROUND ``` ```sh $ docker compose exec -it nextcloud-app /bin/bash -c 'ls -l /run/secrets/' WARN[0000] /home/debian/nextcloud/nc2/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion total 4 -rw------- 1 1000 1000 20 Aug 18 17:08 mysql_user_password ``` ```sh $ docker compose exec -u www-data -it nextcloud-app /bin/bash -c 'cat /run/secrets/mysql_user_password' WARN[0000] /home/debian/nextcloud/nc2/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion cat: /run/secrets/mysql_user_password: Permission denied ``` ```json { "reqId": "UjaIPqzPiap1ZMsi7TG8", "level": 3, "time": "2024-08-18T17:50:30+00:00", "remoteAddr": "127.0.0.1", "user": "--", "app": "PHP", "method": "GET", "url": "/", "message": "fopen(/var/www/html/config/config.php): Failed to open stream: No such file or directory at /var/www/html/lib/private/Config.php#221", "userAgent": "curl/7.88.1", "version": "", "data": { "app": "PHP" } } ```

[mabeett]

I created #2280 2 weeks ago, your feedback will be appreciated.