MariaDB / mariadb-docker

Docker Official Image packaging for MariaDB
https://mariadb.org
GNU General Public License v2.0
767 stars 438 forks source link

.my-healthcheck.cnf permissions problems #573

Closed Daviid-P closed 1 month ago

Daviid-P commented 5 months ago

This is my compose file:

version: '3'

services:
  openfire:
    image: nasqueron/openfire
    restart: "unless-stopped"
    depends_on:
        mysql:
            condition: service_healthy
            restart: true
    mem_reservation: "1G"
    mem_limit: "1G"
    ports:
      - "32780:9090/tcp"
      - "32781:9091/tcp"
      - "32782:5222/tcp"
      - "32783:7777/tcp"
      - "32784:7070/tcp"
    volumes:
      - /mnt/h/containers/openfire:/var/lib/openfire
    environment:
      TZ: "Europe/Madrid"
      DAEMON_OPTS: -Xms32m -Xmx128m -Xss128k -Xoss128k -XX:ThreadStackSize=128 -XX:+PrintGCDetails -Xloggc:/var/log/openfire/gc.log -XX:+HeapDumpOnOutOfMemoryError
  mysql:
    image: mariadb:latest
    environment:
        MYSQL_ROOT_PASSWORD: root
        MYSQL_DATABASE: openfire
        MYSQL_USER: openfire
        MYSQL_PASSWORD: openfire
    healthcheck:
      interval: 5s
      retries: 5
      test:
        [
          "CMD",
          "healthcheck.sh",
          "--su-mysql",
          "--connect",
          "--innodb_initialized"
        ]
      timeout: 5s
    ports:
      - "32788:3306/tcp"
    volumes:
        - /mnt/h/containers/openfire/mysql:/var/lib/mysql
    restart: always

I want to start openfire after mariadb is up and running so I wanted to use the healthcheck script:

$ docker compose up -d
[+] Running 3/3
 ✔ Network openfire_default       Created                                                                                                              0.1s
 ✘ Container openfire-mysql-1     Error                                                                                                                0.1s
 ✔ Container openfire-openfire-1  Created                                                                                                              0.0s
dependency failed to start: container openfire-mysql-1 is unhealthy

Here are the full logs: docker logs openfire-mysql-1

2024-04-16 09:55:26+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:11.3.2+maria~ubu2204 started.
2024-04-16 09:55:26+00:00 [Warn] [Entrypoint]: /sys/fs/cgroup/name=systemd:/docker/e00da82e1166cb2b9f00dcca3cb34e491661ed94b23114b139cc739ff8264cce
14:misc:/docker/e00da82e1166cb2b9f00dcca3cb34e491661ed94b23114b139cc739ff8264cce
13:rdma:/docker/e00da82e1166cb2b9f00dcca3cb34e491661ed94b23114b139cc739ff8264cce
12:pids:/docker/e00da82e1166cb2b9f00dcca3cb34e491661ed94b23114b139cc739ff8264cce
11:hugetlb:/docker/e00da82e1166cb2b9f00dcca3cb34e491661ed94b23114b139cc739ff8264cce
10:net_prio:/docker/e00da82e1166cb2b9f00dcca3cb34e491661ed94b23114b139cc739ff8264cce
9:perf_event:/docker/e00da82e1166cb2b9f00dcca3cb34e491661ed94b23114b139cc739ff8264cce
8:net_cls:/docker/e00da82e1166cb2b9f00dcca3cb34e491661ed94b23114b139cc739ff8264cce
7:freezer:/docker/e00da82e1166cb2b9f00dcca3cb34e491661ed94b23114b139cc739ff8264cce
6:devices:/docker/e00da82e1166cb2b9f00dcca3cb34e491661ed94b23114b139cc739ff8264cce
5:memory:/docker/e00da82e1166cb2b9f00dcca3cb34e491661ed94b23114b139cc739ff8264cce
4:blkio:/docker/e00da82e1166cb2b9f00dcca3cb34e491661ed94b23114b139cc739ff8264cce
3:cpuacct:/docker/e00da82e1166cb2b9f00dcca3cb34e491661ed94b23114b139cc739ff8264cce
2:cpu:/docker/e00da82e1166cb2b9f00dcca3cb34e491661ed94b23114b139cc739ff8264cce
1:cpuset:/docker/e00da82e1166cb2b9f00dcca3cb34e491661ed94b23114b139cc739ff8264cce
0::/docker/e00da82e1166cb2b9f00dcca3cb34e491661ed94b23114b139cc739ff8264cce/memory.pressure not writable, functionality unavailable to MariaDB
2024-04-16 09:55:26+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
2024-04-16 09:55:26+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:11.3.2+maria~ubu2204 started.
2024-04-16 09:55:27+00:00 [Note] [Entrypoint]: Initializing database files

PLEASE REMEMBER TO SET A PASSWORD FOR THE MariaDB root USER !
To do so, start the server, then issue the following command:

'/usr/bin/mariadb-secure-installation'

which will also give you the option of removing the test
databases and anonymous user created by default.  This is
strongly recommended for production servers.

See the MariaDB Knowledgebase at https://mariadb.com/kb

Please report any problems at https://mariadb.org/jira

The latest information about MariaDB is available at https://mariadb.org/.

Consider joining MariaDB's strong and vibrant community:
https://mariadb.org/get-involved/

2024-04-16 09:55:34+00:00 [Note] [Entrypoint]: Database files initialized
2024-04-16 09:55:34+00:00 [Note] [Entrypoint]: Starting temporary server
2024-04-16 09:55:34+00:00 [Note] [Entrypoint]: Waiting for server startup
2024-04-16  9:55:34 0 [Warning] Setting lower_case_table_names=2 because file system for /var/lib/mysql/ is case insensitive
2024-04-16  9:55:34 0 [Note] Starting MariaDB 11.3.2-MariaDB-1:11.3.2+maria~ubu2204 source revision 068a6819eb63bcb01fdfa037c9bf3bf63c33ee42 as process 112
2024-04-16  9:55:34 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2024-04-16  9:55:34 0 [Note] InnoDB: Using transactional memory
2024-04-16  9:55:34 0 [Note] InnoDB: Number of transaction pools: 1
2024-04-16  9:55:34 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
2024-04-16  9:55:34 0 [Note] mariadbd: O_TMPFILE is not supported on /tmp (disabling future attempts)
2024-04-16  9:55:34 0 [Note] InnoDB: Using liburing
2024-04-16  9:55:34 0 [Note] InnoDB: Initializing buffer pool, total size = 128.000MiB, chunk size = 2.000MiB
2024-04-16  9:55:34 0 [Note] InnoDB: Completed initialization of buffer pool
2024-04-16  9:55:34 0 [Note] InnoDB: Buffered log writes (block size=512 bytes)
2024-04-16  9:55:34 0 [Note] InnoDB: End of log at LSN=47629
2024-04-16  9:55:34 0 [Note] InnoDB: Opened 3 undo tablespaces
2024-04-16  9:55:34 0 [Note] InnoDB: 128 rollback segments in 3 undo tablespaces are active.
2024-04-16  9:55:34 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ...
2024-04-16  9:55:34 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB.
2024-04-16  9:55:34 0 [Note] InnoDB: log sequence number 47629; transaction id 14
2024-04-16  9:55:34 0 [Note] Plugin 'FEEDBACK' is disabled.
2024-04-16  9:55:34 0 [Note] Plugin 'wsrep-provider' is disabled.
2024-04-16  9:55:34 0 [Warning] 'user' entry 'root@e00da82e1166' ignored in --skip-name-resolve mode.
2024-04-16  9:55:34 0 [Warning] 'proxies_priv' entry '@% root@e00da82e1166' ignored in --skip-name-resolve mode.
2024-04-16  9:55:34 0 [Note] mariadbd: Event Scheduler: Loaded 0 events
2024-04-16  9:55:34 0 [Note] mariadbd: ready for connections.
Version: '11.3.2-MariaDB-1:11.3.2+maria~ubu2204'  socket: '/run/mysqld/mysqld.sock'  port: 0  mariadb.org binary distribution
2024-04-16 09:55:35+00:00 [Note] [Entrypoint]: Temporary server started.
2024-04-16 09:55:37+00:00 [Note] [Entrypoint]: Creating database openfire
2024-04-16 09:55:37+00:00 [Note] [Entrypoint]: Creating user openfire
2024-04-16 09:55:37+00:00 [Note] [Entrypoint]: Giving user openfire access to schema openfire
2024-04-16 09:55:37+00:00 [Note] [Entrypoint]: Securing system users (equivalent to running mysql_secure_installation)

2024-04-16 09:55:37+00:00 [Note] [Entrypoint]: Stopping temporary server
2024-04-16  9:55:37 0 [Note] mariadbd (initiated by: unknown): Normal shutdown
2024-04-16  9:55:37 0 [Note] InnoDB: FTS optimize thread exiting.
2024-04-16  9:55:37 0 [Note] InnoDB: Starting shutdown...
2024-04-16  9:55:37 0 [Note] InnoDB: Dumping buffer pool(s) to /var/lib/mysql/ib_buffer_pool
2024-04-16  9:55:37 0 [Note] InnoDB: Buffer pool(s) dump completed at 240416  9:55:37
2024-04-16  9:55:37 0 [Note] InnoDB: Removed temporary tablespace data file: "./ibtmp1"
2024-04-16  9:55:37 0 [Note] InnoDB: Shutdown completed; log sequence number 47629; transaction id 15
2024-04-16  9:55:37 0 [Note] mariadbd: Shutdown complete

2024-04-16 09:55:37+00:00 [Note] [Entrypoint]: Temporary server stopped

2024-04-16 09:55:37+00:00 [Note] [Entrypoint]: MariaDB init process done. Ready for start up.

2024-04-16  9:55:37 0 [Warning] Setting lower_case_table_names=2 because file system for /var/lib/mysql/ is case insensitive
2024-04-16  9:55:37 0 [Note] Starting MariaDB 11.3.2-MariaDB-1:11.3.2+maria~ubu2204 source revision 068a6819eb63bcb01fdfa037c9bf3bf63c33ee42 as process 1
2024-04-16  9:55:37 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2024-04-16  9:55:37 0 [Note] InnoDB: Using transactional memory
2024-04-16  9:55:37 0 [Note] InnoDB: Number of transaction pools: 1
2024-04-16  9:55:37 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
2024-04-16  9:55:37 0 [Note] mariadbd: O_TMPFILE is not supported on /tmp (disabling future attempts)
2024-04-16  9:55:37 0 [Note] InnoDB: Using liburing
2024-04-16  9:55:37 0 [Note] InnoDB: Initializing buffer pool, total size = 128.000MiB, chunk size = 2.000MiB
2024-04-16  9:55:37 0 [Note] InnoDB: Completed initialization of buffer pool
2024-04-16  9:55:37 0 [Note] InnoDB: Buffered log writes (block size=512 bytes)
2024-04-16  9:55:37 0 [Note] InnoDB: End of log at LSN=47629
2024-04-16  9:55:37 0 [Note] InnoDB: Opened 3 undo tablespaces
2024-04-16  9:55:37 0 [Note] InnoDB: 128 rollback segments in 3 undo tablespaces are active.
2024-04-16  9:55:37 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ...
2024-04-16  9:55:37 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB.
2024-04-16  9:55:37 0 [Note] InnoDB: log sequence number 47629; transaction id 14
2024-04-16  9:55:37 0 [Note] Plugin 'FEEDBACK' is disabled.
2024-04-16  9:55:37 0 [Note] Plugin 'wsrep-provider' is disabled.
2024-04-16  9:55:37 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
2024-04-16  9:55:37 0 [Note] InnoDB: Buffer pool(s) load completed at 240416  9:55:37
2024-04-16  9:55:37 0 [Note] Server socket created on IP: '0.0.0.0'.
2024-04-16  9:55:37 0 [Note] Server socket created on IP: '::'.
2024-04-16  9:55:37 0 [Note] mariadbd: Event Scheduler: Loaded 0 events
2024-04-16  9:55:37 0 [Note] mariadbd: ready for connections.
Version: '11.3.2-MariaDB-1:11.3.2+maria~ubu2204'  socket: '/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution
2024-04-16  9:55:41 3 [Warning] Access denied for user 'mysql'@'127.0.0.1' (using password: NO)
2024-04-16  9:55:41 4 [Warning] Access denied for user 'mysql'@'localhost' (using password: NO)
2024-04-16  9:55:46 5 [Warning] Access denied for user 'mysql'@'127.0.0.1' (using password: NO)
2024-04-16  9:55:47 6 [Warning] Access denied for user 'mysql'@'localhost' (using password: NO)
2024-04-16  9:55:52 7 [Warning] Access denied for user 'mysql'@'127.0.0.1' (using password: NO)
2024-04-16  9:55:52 8 [Warning] Access denied for user 'mysql'@'localhost' (using password: NO)
2024-04-16  9:55:57 9 [Warning] Access denied for user 'mysql'@'127.0.0.1' (using password: NO)
2024-04-16  9:55:57 10 [Warning] Access denied for user 'mysql'@'localhost' (using password: NO)
2024-04-16  9:56:02 11 [Warning] Access denied for user 'mysql'@'127.0.0.1' (using password: NO)
2024-04-16  9:56:02 12 [Warning] Access denied for user 'mysql'@'localhost' (using password: NO)
2024-04-16  9:56:07 13 [Warning] Access denied for user 'mysql'@'127.0.0.1' (using password: NO)
2024-04-16  9:56:07 14 [Warning] Access denied for user 'mysql'@'localhost' (using password: NO)

docker exec -it openfire-mysql-1 bash

root@e00da82e1166:/# /usr/local/bin/healthcheck.sh --su-mysql --connect --innodb_initialized
Warning: World-writable config file '/var/lib/mysql/.my-healthcheck.cnf' is ignored
ERROR 1045 (28000): Access denied for user 'mysql'@'localhost' (using password: NO)
healthcheck innodb_initialized failed
root@e00da82e1166:/# /usr/local/bin/healthcheck.sh --connect --innodb_initialized
Warning: World-writable config file '/var/lib/mysql/.my-healthcheck.cnf' is ignored
ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: NO)
healthcheck innodb_initialized failed
root@e00da82e1166:/# ll /var/lib/mysql/
total 170173
drwxrwxrwx 1 1000 1000       512 Apr 16 09:55 ./
drwxr-xr-x 1 root root      4096 Mar  6 02:56 ../
-rwxrwxrwx 1 1000 1000       131 Apr 16 09:55 .my-healthcheck.cnf*
-rwxrwxrwx 1 1000 1000  16932864 Apr 16 09:55 aria_log.00000001*
-rwxrwxrwx 1 1000 1000        52 Apr 16 09:55 aria_log_control*
-rwxrwxrwx 1 1000 1000         9 Apr 16 09:55 ddl_recovery.log*
-rwxrwxrwx 1 1000 1000       690 Apr 16 09:55 ib_buffer_pool*
-rwxrwxrwx 1 1000 1000 100663296 Apr 16 09:55 ib_logfile0*
-rwxrwxrwx 1 1000 1000  12582912 Apr 16 09:55 ibdata1*
-rwxrwxrwx 1 1000 1000  12582912 Apr 16 09:55 ibtmp1*
-rwxrwxrwx 1 1000 1000        14 Apr 16 09:55 mariadb_upgrade_info*
-rwxrwxrwx 1 1000 1000         0 Apr 16 09:55 multi-master.info*
drwxrwxrwx 1 1000 1000       512 Apr 16 09:55 mysql/
drwxrwxrwx 1 1000 1000       512 Apr 16 09:55 openfire/
drwxrwxrwx 1 1000 1000       512 Apr 16 09:55 performance_schema/
drwxrwxrwx 1 1000 1000       512 Apr 16 09:55 sys/
-rwxrwxrwx 1 1000 1000     24576 Apr 16 09:55 tc.log*
-rwxrwxrwx 1 1000 1000  10485760 Apr 16 09:55 undo001*
-rwxrwxrwx 1 1000 1000  10485760 Apr 16 09:55 undo002*
-rwxrwxrwx 1 1000 1000  10485760 Apr 16 09:55 undo003*

Is this due to being in WSL and using /mnt/h/containers/openfire/mysql as volume?

Daviid-P commented 5 months ago

Yes, using /container_data/openfire/mysql:/var/lib/mysql as volume works.

grooverdan commented 5 months ago

"Warning: World-writable config file '/var/lib/mysql/.my-healthcheck.cnf' is ignored"

The credentials of the healthcheck are in this file. Removing the check requires a change to the mariadb client implementation.

best I think of so far is moving .my-healthcheck.cnf to a different volume and mounting that volume as read only. Replace /var/lib/mysql/.my-healthcheck.cnf with a symlink to the final volume location.

Hope I can think of something better later, this isn't particularly great.

Daviid-P commented 5 months ago

"Warning: World-writable config file '/var/lib/mysql/.my-healthcheck.cnf' is ignored"

The credentials of the healthcheck are in this file. Removing the check requires a change to the mariadb client implementation.

best I think of so far is moving .my-healthcheck.cnf to a different volume and mounting that volume as read only. Replace /var/lib/mysql/.my-healthcheck.cnf with a symlink to the final volume location.

Hope I can think of something better later, this isn't particularly great.

I though of putting the file as readonly but then in the edgecase I ever need the process to re-run I'm stuck with the old password in .my-healthcheck.cnf, no?

Technically I don't need the mysql folder to be accessible from windows /mnt/h/containers/openfire/mysql so for now I've chosen to create /container_data/openfire/mysql inside WSL

grooverdan commented 5 months ago

I think I remember someone using Windows readonly file permissions to avoid this ignoring of config files. Thanks for the reminder. I don't think you'll get that stuck. Not sure if you've seen, on "re-run" - MARIADB_AUTO_UPGRADE=1 will reset password/recreate healthcheck user if .my-healthcheck.cnf is missing.

Do named volumes with WSL get created inside WSL and hence have unix like permissions too? I'm thinking what to write on a FAQ page.

Also technically --su-mysql no longer needed and its an additional fork during the healthcheck.

mholubinka1 commented 3 months ago

I'm getting this problem when running on an ntfs formatted externally mounted drive attached to a Raspberry Pi. Any advice for a proper fix for this rather than a hacky workaround?

Attempted solutions:

  1. --su-mysql: same issue but with mysql instead of root
energy-monitor-db  | 2024-06-19 13:54:35 3 [Warning] Access denied for user 'mysql'@'::1' (using password: NO)
energy-monitor-db  | 2024-06-19 13:54:35 4 [Warning] Access denied for user 'mysql'@'localhost' (using password: NO)
  1. mounting as a ro volume: permissions issues
energy-monitor-db  | 2024-06-19 13:40:29+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:11.4.2+maria~ubu2404 started.
energy-monitor-db  | chown: changing ownership of '/var/lib/mysql/.my-healthcheck.cnf': Read-only file system
energy-monitor-db exited with code 1

EDIT: for now I've resorted to a custom healthcheck that avoids the perms problems

  mariadb:
    container_name: mariadb
    image: mariadb:11.4.2
    environment:
      MARIADB_RANDOM_ROOT_PASSWORD: ${MARIADB_RANDOM_ROOT_PASSWORD}
      MARIADB_USER: ${MARIADB_USER}
      MARIADB_PASSWORD: ${MARIADB_PASSWORD}
      MARIADB_DATABASE: ${MARIADB_DATABASE}
    ports:
      - "3306:3306"
    volumes:
      - /mnt/path/to/mariadb/data:/var/lib/mysql/
      - /mnt/path/to/init.sql:/docker-entrypoint-initdb.d/init.sql
    healthcheck:
      test: "mariadb --user=$${MARIADB_USER} --password=$${MARIADB_PASSWORD} --execute \"SHOW DATABASES;\""
      start_interval: 2m
      start_period: 10s
      interval: 10s
      timeout: 5s
      retries: 3
    restart: unless-stopped
grooverdan commented 3 months ago

Added solution for 2. in #595 - feedback welcome.

on 1. - this would require MARIADB_MYSQL_LOCALHOST_USER=1 to be set of startup to create the user. Or explicitly CREATE USER mysql@localhost IDENTIFIED VIA unix_socket

grooverdan commented 3 months ago

Oh for Windows WSL users - watch out for https://github.com/microsoft/WSL/issues/8443 - occurs on a table rebuild.

mholubinka1 commented 3 months ago

Added solution for 2. in #595 - feedback welcome.

on 1. - this would require MARIADB_MYSQL_LOCALHOST_USER=1 to be set of startup to create the user. Or explicitly CREATE USER mysql@localhost IDENTIFIED VIA unix_socket

I should have been explicit. That variable was set when I attempted it as a solution.

Simbiat commented 3 months ago

So I have a test like (with custom config since require_secure_transport=ON)

test: [ "CMD", "healthcheck.sh", "--defaults-file=/etc/mysql/conf.d/my.cnf", "--connect", "--innodb_initialized" ]

And it's spamming a lot of warnings like

2024-06-24 11:44:15 120 [Warning] Aborted connection 120 to db: 'unconnected' user: 'unauthenticated' host: '::1' (This connection closed normally without authentication)

And looks like it's spamming this specifically if there is a --connect argument, even though validation succeeds. I tested this specifically spamming the respective command manually on the container. I understand that this is "kind of" normal, since --connect is expected to just try to establish TCP connection, not authenticate, but can we somehow suppress the warnings for the check? The settings in manual do suggest using 10s interval, but then it means spamming the false-positive warning every 10 seconds, even when connection is technically established. Or am I doing something wrong?

grooverdan commented 3 months ago

@Simbiat - I broke it with #594 (accidentality). Removing 'protcol=tcp' from "$DATADIR"/.my-healthcheck.cnf might be the best workaround and remove --connect until I fix it.

Simbiat commented 3 months ago

Will it be fixed as part of this issue (573) or as part of something else? Want to know, so that I can monitor it properly.

grooverdan commented 3 months ago

Nope, different issue. #596. Don't be afraid to create new issues.

Simbiat commented 3 months ago

I am not afraid, just know as a tech support for 15 years, that sometimes it's better to ask first if there is already a record :D Thanks