MariaDB / mariadb-docker

Docker Official Image packaging for MariaDB
https://mariadb.org
GNU General Public License v2.0
751 stars 436 forks source link

MARIADB_AUTO_UPGRADE=1/mariadb-upgrade not workling if ssl is enabled #592

Closed egabosh closed 3 weeks ago

egabosh commented 1 month ago

Moin,

I'm using mariadb:lts. While updating the container the last days the MARIADB_AUTO_UPGRADE fails if ssl is enabled:

[mariadbd]
ssl=1
ssl-cert=/etc/mysql/certificate.pem
ssl-key=/etc/mysql/key.pem

Container output:

2024-06-03 12:55:42+02:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:11.4.2+maria~ubu2404 started.
2024-06-03 12:55:43+02:00 [Warn] [Entrypoint]: /sys/fs/cgroup///memory.pressure not writable, functionality unavailable to MariaDB
2024-06-03 12:55:43+02:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
2024-06-03 12:55:43+02:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:11.4.2+maria~ubu2404 started.
2024-06-03 12:55:43+02:00 [Note] [Entrypoint]: MariaDB upgrade information missing, assuming required
2024-06-03 12:55:43+02:00 [Note] [Entrypoint]: Starting temporary server
2024-06-03 12:55:43+02:00 [Note] [Entrypoint]: Waiting for server startup
2024-06-03 12:55:43 0 [Note] Starting MariaDB 11.4.2-MariaDB-ubu2404 source revision 3fca5ed772fb75e3e57c507edef2985f8eba5b12 as process 56
2024-06-03 12:55:43 0 [Note] InnoDB: Compressed tables use zlib 1.3
2024-06-03 12:55:43 0 [Note] InnoDB: Number of transaction pools: 1
2024-06-03 12:55:43 0 [Note] InnoDB: Using ARMv8 crc32 + pmull instructions
2024-06-03 12:55:43 0 [Note] mariadbd: O_TMPFILE is not supported on /tmp (disabling future attempts)
2024-06-03 12:55:43 0 [Warning] mariadbd: io_uring_queue_init() failed with errno 95
2024-06-03 12:55:43 0 [Warning] InnoDB: liburing disabled: falling back to innodb_use_native_aio=OFF
2024-06-03 12:55:43 0 [Note] InnoDB: Initializing buffer pool, total size = 128.000MiB, chunk size = 2.000MiB
2024-06-03 12:55:43 0 [Note] InnoDB: Completed initialization of buffer pool
2024-06-03 12:55:43 0 [Note] InnoDB: File system buffers for log disabled (block size=4096 bytes)
2024-06-03 12:55:43 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=109661056
2024-06-03 12:55:43 0 [Note] InnoDB: End of log at LSN=109712824
2024-06-03 12:55:43 0 [Note] InnoDB: To recover: 133 pages
2024-06-03 12:55:43 0 [Note] InnoDB: Opened 3 undo tablespaces
2024-06-03 12:55:43 0 [Note] InnoDB: 128 rollback segments in 3 undo tablespaces are active.
2024-06-03 12:55:43 0 [Note] InnoDB: Removed temporary tablespace data file: "./ibtmp1"
2024-06-03 12:55:43 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ...
2024-06-03 12:55:43 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB.
2024-06-03 12:55:43 0 [Note] InnoDB: log sequence number 109712824; transaction id 392998
2024-06-03 12:55:43 0 [Note] Plugin 'FEEDBACK' is disabled.
2024-06-03 12:55:43 0 [Note] Plugin 'wsrep-provider' is disabled.
2024-06-03 12:55:43 0 [Note] Recovering after a crash using tc.log
2024-06-03 12:55:43 0 [Note] Starting table crash recovery...
2024-06-03 12:55:43 0 [Note] Crash table recovery finished.
2024-06-03 12:55:43 0 [Note] mariadbd: ready for connections.
Version: '11.4.2-MariaDB-ubu2404'  socket: '/run/mysqld/mysqld.sock'  port: 0  mariadb.org binary distribution
2024-06-03 12:55:44 3 [Warning] Aborted connection 3 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
2024-06-03 12:55:45 4 [Warning] Aborted connection 4 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
2024-06-03 12:55:46 5 [Warning] Aborted connection 5 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
2024-06-03 12:55:47 6 [Warning] Aborted connection 6 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
2024-06-03 12:55:48 7 [Warning] Aborted connection 7 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
2024-06-03 12:55:49 8 [Warning] Aborted connection 8 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
2024-06-03 12:55:50 9 [Warning] Aborted connection 9 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
2024-06-03 12:55:51 10 [Warning] Aborted connection 10 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
2024-06-03 12:55:53 11 [Warning] Aborted connection 11 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
[...]

Switching off ssl fixes this and after mariadb-upgrade ran through i can reenable ssl and it works.

Same if I create a fresh new container with ssl enabled. Is there a way to auto update when SSL is enabled too?

Thank you olli

grooverdan commented 1 month ago

Should be easily possible. Thanks for the bug report.

crysknife commented 1 month ago

I can confirm this too. Unfortunately, the client tools also behave different and reject to connect because of SSL/TLS issues.

Btw. I'm using socket auth for the local root account. While in the previous version (10.11.8), the MariaDB client within the container worked perfectly fine, the new version fails:

root@mariadb:/# mariadb
ERROR 2026 (HY000): TLS/SSL error: unable to get local issuer certificate

The error changes a bit when I add the following client config to provide the Root certificate:

[client-mariadb] ssl_ca = /etc/mysql/conf.d/certificates/root-ca.pem

root@mariadb:/# mariadb
ERROR 2026 (HY000): TLS/SSL error: Validation of SSL server certificate failed

Well, I've tried to include just the root in this file and up to all certs the server uses (server, intermediate, root) of course but without any luck. I guess this is because the connection (via socket) is not really using the server name from the cert.

Of course a workaround is either to disable SSL/TLS or the certificate validation in the client section but none of these feels correct. Also why this change in behavior?

grooverdan commented 4 weeks ago

Something else is at play here:

$ podman volume create m104
m104
$ podman run -v m104:/var/lib/mysql --env MARIADB_ROOT_PASSWORD=bob mariadb:10.4
...
$ podman run --rm -v ./:/etc/mysql/conf.d:z --env MARIADB_ROOT_PASSWORD=bob -v m104:/var/lib/mysql --env MARIADB_AUTO_UPGRADE=1  mariadb:11.4
2024-06-07 08:59:33+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:11.4.2+maria~ubu2404 started.
....
2024-06-07 08:59:35+00:00 [Note] [Entrypoint]: Backing up complete
2024-06-07 08:59:35+00:00 [Note] [Entrypoint]: Starting mariadb-upgrade
The --upgrade-system-tables option was used, user tables won't be touched.
Phase 1/8: Checking and upgrading mysql database
Processing databases
mysql
mysql.column_stats                                 OK
mysql.columns_priv                                 OK
mysql.db                                           OK
mysql.event                                        OK
mysql.func                                         OK
mysql.global_priv                                  OK
mysql.gtid_slave_pos                               OK
mysql.help_category                                OK
mysql.help_keyword                                 OK
mysql.help_relation                                OK
mysql.help_topic                                   OK
mysql.index_stats                                  OK
mysql.innodb_index_stats                           OK
mysql.innodb_table_stats                           OK
mysql.plugin                                       OK
mysql.proc                                         OK
mysql.procs_priv                                   OK
mysql.proxies_priv                                 OK
mysql.roles_mapping                                OK
mysql.servers                                      OK
mysql.table_stats                                  OK
mysql.tables_priv                                  OK
mysql.time_zone                                    OK
mysql.time_zone_leap_second                        OK
mysql.time_zone_name                               OK
mysql.time_zone_transition                         OK
mysql.time_zone_transition_type                    OK
mysql.transaction_registry                         OK
Phase 2/8: Installing used storage engines... Skipped
Phase 3/8: Running 'mysql_fix_privilege_tables'
Phase 4/8: Fixing views... Skipped
Phase 5/8: Fixing table and database names ... Skipped
Phase 6/8: Checking and upgrading tables... Skipped
Phase 7/8: uninstalling plugins
Phase 8/8: Running 'FLUSH PRIVILEGES'
OK
2024-06-07 08:59:36+00:00 [Note] [Entrypoint]: Finished mariadb-upgrade

Config:

$ podman run --rm -v ./:/etc/mysql/conf.d:z --env MARIADB_ROOT_PASSWORD=bob -v m104:/var/lib/mysql --env MARIADB_AUTO_UPGRADE=1  mariadb:11.4 my_print_defaults --mysqld
--socket=/run/mysqld/mysqld.sock
--host-cache-size=0
--skip-name-resolve
--pid-file=/run/mysqld/mysqld.pid
--basedir=/usr
--expire_logs_days=10
--character-set-server=utf8mb4
--character-set-collations=utf8mb4=uca1400_ai_ci
--ssl=1
--ssl-cert=/etc/mysql/conf.d/cert.pem
--ssl-key=/etc/mysql/conf.d/private.key

Could it be x509v1 certs - https://jira.mariadb.org/browse/MDEV-33592?

I created mine with (gnutls):

$   certtool --generate-privkey --bits 4096 --outfile private.key
** Note: You may use '--sec-param High' instead of '--bits 4096'
Generating a 4096 bit RSA private key...
$ certtool --generate-self-signed --load-privkey private.key --outfile cert.pem
Generating a self signed certificate...
LinuxJedi commented 4 weeks ago

In 11.4, --ssl-verify-server-cert effectively became the default (MDEV-31857).

In the cases outlined here, it should be implicitly turned off by maria client due to certain rules met. But if you are using an older (10.11) client with a newer (11.4) MariaDB, this won't happen. Could that be the case here?

egabosh commented 4 weeks ago

@grooverdan In my case I'm using a Letsencrypt-Cert created by Traefik.

crysknife commented 3 weeks ago

@grooverdan, I'm using not LE but my own CA infrastructure (root & intermediate), created by using XCA. Both CA certificates are ECC based ones while the leave cert is RSA.

egabosh commented 3 weeks ago

@grooverdan Tried your way with creating Certs by certtool. With them in my setup no problems too. But I think it should work the ways @crysknife and I do as well. Normal SSL connections (from outside) also work. Is there a way to get more Informations why our Cert/Key makes theres problems in this (special) case? The error Message "Aborted connection" is a bit generic. Or is there a way to do mariadb-upgrade independent from SSL? In the Container-World it runs in the same container and I think in this case SSL is not needed at all for mariadb-upgrade.

egabosh commented 3 weeks ago

In 11.4, --ssl-verify-server-cert effectively became the default (MDEV-31857).

In the cases outlined here, it should be implicitly turned off by maria client due to certain rules met. But if you are using an older (10.11) client with a newer (11.4) MariaDB, this won't happen. Could that be the case here?

@LinuxJedi For my unserstanding mariadb-upgrade runs in the same (11.4) container. It happens in a "fresh" setup with 11.4 too. So I think there is no older client in use.

egabosh commented 3 weeks ago

Could it be x509v1 certs - https://jira.mariadb.org/browse/MDEV-33592?

@grooverdan As I understand this issue should be fixed in 11.4.2 but it still happens

grooverdan commented 3 weeks ago

The upgrade is a server in --skip-grant-tables and --skip-networking. The authentication failures are from https://github.com/MariaDB/mariadb-docker/blob/master/docker-entrypoint.sh#L136 where is just seeing if its awake. Is something really checking tls certs on a unix socket when the server is in skip-grant-tables?

Can I get the logs when using the following options at startup just to check I'm looking in the right location.

--env MARIADB_AUTO_UPGRADE=1 --entrypoint /bin/bash --user mysql  mariadb:11.4 -x -v /usr/local/bin/docker-entrypoint.sh mariadbd

And:

docker run ... mariadb:11.4 my_print_defaults --mariadbd also just to ensure additional options aren't at play.

egabosh commented 3 weeks ago

Here you are:

$ docker run --user mysql --rm -v ./:/etc/mysql/conf.d:z --env MARIADB_ROOT_PASSWORD=bob -v ./dbdata:/var/lib/mysql --env MARIADB_AUTO_UPGRADE=1 --entrypoint /bin/bash  mariadb:11.4 -x -v /usr/local/bin/docker-entrypoint.sh mariadbd
#!/bin/bash
set -eo pipefail
+ set -eo pipefail
shopt -s nullglob
+ shopt -s nullglob

# logging functions
mysql_log() {
    local type="$1"; shift
    printf '%s [%s] [Entrypoint]: %s\n' "$(date --rfc-3339=seconds)" "$type" "$*"
}
mysql_note() {
    mysql_log Note "$@"
}
mysql_warn() {
    mysql_log Warn "$@" >&2
}
mysql_error() {
    mysql_log ERROR "$@" >&2
    exit 1
}

# usage: file_env VAR [DEFAULT]
#    ie: file_env 'XYZ_DB_PASSWORD' 'example'
# (will allow for "$XYZ_DB_PASSWORD_FILE" to fill in the value of
#  "$XYZ_DB_PASSWORD" from a file, especially for Docker's secrets feature)
file_env() {
    local var="$1"
    local fileVar="${var}_FILE"
    local def="${2:-}"
    if [ "${!var:-}" ] && [ "${!fileVar:-}" ]; then
        mysql_error "Both $var and $fileVar are set (but are exclusive)"
    fi
    local val="$def"
    if [ "${!var:-}" ]; then
        val="${!var}"
    elif [ "${!fileVar:-}" ]; then
        val="$(< "${!fileVar}")"
    fi
    export "$var"="$val"
    unset "$fileVar"
}

# set MARIADB_xyz from MYSQL_xyz when MARIADB_xyz is unset
# and make them the same value (so user scripts can use either)
_mariadb_file_env() {
    local var="$1"; shift
    local maria="MARIADB_${var#MYSQL_}"
    file_env "$var" "$@"
    file_env "$maria" "${!var}"
    if [ "${!maria:-}" ]; then
        export "$var"="${!maria}"
    fi
}

# check to see if this file is being run or sourced from another script
_is_sourced() {
    # https://unix.stackexchange.com/a/215279
    [ "${#FUNCNAME[@]}" -ge 2 ] \
        && [ "${FUNCNAME[0]}" = '_is_sourced' ] \
        && [ "${FUNCNAME[1]}" = 'source' ]
}

# usage: docker_process_init_files [file [file [...]]]
#    ie: docker_process_init_files /always-initdb.d/*
# process initializer files, based on file extensions
docker_process_init_files() {
    # mysql here for backwards compatibility "${mysql[@]}"
    # ShellCheck: mysql appears unused. Verify use (or export if used externally)
    # shellcheck disable=SC2034
    mysql=( docker_process_sql )

    echo
    local f
    for f; do
        case "$f" in
            *.sh)
                # https://github.com/docker-library/postgres/issues/450#issuecomment-393167936
                # https://github.com/docker-library/postgres/pull/452
                if [ -x "$f" ]; then
                    mysql_note "$0: running $f"
                    "$f"
                else
                    mysql_note "$0: sourcing $f"
                    # ShellCheck can't follow non-constant source. Use a directive to specify location.
                    # shellcheck disable=SC1090
                    . "$f"
                fi
                ;;
            *.sql)     mysql_note "$0: running $f"; docker_process_sql < "$f"; echo ;;
            *.sql.gz)  mysql_note "$0: running $f"; gunzip -c "$f" | docker_process_sql; echo ;;
            *.sql.xz)  mysql_note "$0: running $f"; xzcat "$f" | docker_process_sql; echo ;;
            *.sql.zst) mysql_note "$0: running $f"; zstd -dc "$f" | docker_process_sql; echo ;;
            *)         mysql_warn "$0: ignoring $f" ;;
        esac
        echo
    done
}

# arguments necessary to run "mariadbd --verbose --help" successfully (used for testing configuration validity and for extracting default/configured values)
_verboseHelpArgs=(
    --verbose --help
)
+ _verboseHelpArgs=(--verbose --help)

mysql_check_config() {
    local toRun=( "$@" "${_verboseHelpArgs[@]}" ) errors
    if ! errors="$("${toRun[@]}" 2>&1 >/dev/null)"; then
        mysql_error $'mariadbd failed while attempting to check config\n\tcommand was: '"${toRun[*]}"$'\n\t'"$errors"
    fi
}

# Fetch value from server config
# We use mariadbd --verbose --help instead of my_print_defaults because the
# latter only show values present in config files, and not server defaults
mysql_get_config() {
    local conf="$1"; shift
    "$@" "${_verboseHelpArgs[@]}" 2>/dev/null \
        | awk -v conf="$conf" '$1 == conf && /^[^ \t]/ { sub(/^[^ \t]+[ \t]+/, ""); print; exit }'
    # match "datadir      /some/path with/spaces in/it here" but not "--xyz=abc\n     datadir (xyz)"
}

# Do a temporary startup of the MariaDB server, for init purposes
docker_temp_server_start() {
    "$@" --skip-networking --default-time-zone=SYSTEM --socket="${SOCKET}" --wsrep_on=OFF \
        --expire-logs-days=0 \
        --loose-innodb_buffer_pool_load_at_startup=0 &
    declare -g MARIADB_PID
    MARIADB_PID=$!
    mysql_note "Waiting for server startup"
    # only use the root password if the database has already been initialized
    # so that it won't try to fill in a password file when it hasn't been set yet
    extraArgs=()
    if [ -z "$DATABASE_ALREADY_EXISTS" ]; then
        extraArgs+=( '--dont-use-mysql-root-password' )
    fi
    local i
    for i in {30..0}; do
        if docker_process_sql "${extraArgs[@]}" --database=mysql <<<'SELECT 1' &> /dev/null; then
            break
        fi
        sleep 1
    done
    if [ "$i" = 0 ]; then
        mysql_error "Unable to start server."
    fi
}

# Stop the server. When using a local socket file mariadb-admin will block until
# the shutdown is complete.
docker_temp_server_stop() {
    kill "$MARIADB_PID"
    wait "$MARIADB_PID"
}

# Verify that the minimally required password settings are set for new databases.
docker_verify_minimum_env() {
    # Restoring from backup requires no environment variables
    declare -g DATABASE_INIT_FROM_BACKUP
    for file in /docker-entrypoint-initdb.d/*.tar{.gz,.xz,.zst}; do
        if [ -f "${file}" ]; then
            DATABASE_INIT_FROM_BACKUP='true'
            return
        fi
    done
    if [ -z "$MARIADB_ROOT_PASSWORD" ] && [ -z "$MARIADB_ROOT_PASSWORD_HASH" ] && [ -z "$MARIADB_ALLOW_EMPTY_ROOT_PASSWORD" ] && [ -z "$MARIADB_RANDOM_ROOT_PASSWORD" ]; then
        mysql_error $'Database is uninitialized and password option is not specified\n\tYou need to specify one of MARIADB_ROOT_PASSWORD, MARIADB_ROOT_PASSWORD_HASH, MARIADB_ALLOW_EMPTY_ROOT_PASSWORD and MARIADB_RANDOM_ROOT_PASSWORD'
    fi
    # More preemptive exclusions of combinations should have been made before *PASSWORD_HASH was added, but for now we don't enforce due to compatibility.
    if [ -n "$MARIADB_ROOT_PASSWORD" ] || [ -n "$MARIADB_ALLOW_EMPTY_ROOT_PASSWORD" ] || [ -n "$MARIADB_RANDOM_ROOT_PASSWORD" ] && [ -n "$MARIADB_ROOT_PASSWORD_HASH" ]; then
        mysql_error "Cannot specify MARIADB_ROOT_PASSWORD_HASH and another MARIADB_ROOT_PASSWORD* option."
    fi
    if [ -n "$MARIADB_PASSWORD" ] && [ -n "$MARIADB_PASSWORD_HASH" ]; then
        mysql_error "Cannot specify MARIADB_PASSWORD_HASH and MARIADB_PASSWORD option."
    fi
    if [ -n "$MARIADB_REPLICATION_USER" ]; then
        if [ -z "$MARIADB_MASTER_HOST" ]; then
            # its a master, we're creating a user
            if [ -z "$MARIADB_REPLICATION_PASSWORD" ] && [ -z "$MARIADB_REPLICATION_PASSWORD_HASH" ]; then
                mysql_error "MARIADB_REPLICATION_PASSWORD or MARIADB_REPLICATION_PASSWORD_HASH not found to create replication user for master"
            fi
        else
            # its a replica
            if [ -z "$MARIADB_REPLICATION_PASSWORD" ] ; then
                mysql_error "MARIADB_REPLICATION_PASSWORD is mandatory to specify the replication on the replica image."
            fi
            if [ -n "$MARIADB_REPLICATION_PASSWORD_HASH" ] ; then
                mysql_warn "MARIADB_REPLICATION_PASSWORD_HASH cannot be specified on a replica"
            fi
        fi
    fi
    if [ -n "$MARIADB_MASTER_HOST" ] && { [ -z "$MARIADB_REPLICATION_USER" ] || [ -z "$MARIADB_REPLICATION_PASSWORD" ] ; }; then
        mysql_error "For a replica, MARIADB_REPLICATION_USER and MARIADB_REPLICATION is mandatory."
    fi
}

# creates folders for the database
# also ensures permission for user mysql of run as root
docker_create_db_directories() {
    local user; user="$(id -u)"

    # TODO other directories that are used by default? like /var/lib/mysql-files
    # see https://github.com/docker-library/mysql/issues/562
    mkdir -p "$DATADIR"

    if [ "$user" = "0" ]; then
        # this will cause less disk access than `chown -R`
        find "$DATADIR" \! -user mysql -exec chown mysql: '{}' +
        # See https://github.com/MariaDB/mariadb-docker/issues/363
        find "${SOCKET%/*}" -maxdepth 0 \! -user mysql -exec chown mysql: '{}' \;

        # memory.pressure
        local cgroup; cgroup=$(</proc/self/cgroup)
        local mempressure="/sys/fs/cgroup/${cgroup:3}/memory.pressure"
        if [ -w "$mempressure" ]; then
            chown mysql: "$mempressure" || mysql_warn "unable to change ownership of $mempressure, functionality unavailable to MariaDB"
        else
            mysql_warn "$mempressure not writable, functionality unavailable to MariaDB"
        fi
    fi
}

_mariadb_version() {
    echo -n "11.4.2-MariaDB"
}

# initializes the database directory
docker_init_database_dir() {
    mysql_note "Initializing database files"
    installArgs=( --datadir="$DATADIR" --rpm --auth-root-authentication-method=normal )
    # "Other options are passed to mariadbd." (so we pass all "mariadbd" arguments directly here)

    local mariadbdArgs=()
    for arg in "${@:2}"; do
        # Check if the argument contains whitespace
        if [[ "$arg" =~ [[:space:]] ]]; then
            mysql_warn "Not passing argument \'$arg\' to mariadb-install-db because mariadb-install-db does not support arguments with whitespace."
        else
            mariadbdArgs+=("$arg")
        fi
    done
    mariadb-install-db "${installArgs[@]}" "${mariadbdArgs[@]}" \
        --skip-test-db \
        --old-mode='UTF8_IS_UTF8MB3' \
        --default-time-zone=SYSTEM --enforce-storage-engine= \
        --skip-log-bin \
        --expire-logs-days=0 \
        --loose-innodb_buffer_pool_load_at_startup=0 \
        --loose-innodb_buffer_pool_dump_at_shutdown=0
    mysql_note "Database files initialized"
}

# Loads various settings that are used elsewhere in the script
# This should be called after mysql_check_config, but before any other functions
docker_setup_env() {
    # Get config
    declare -g DATADIR SOCKET PORT
    DATADIR="$(mysql_get_config 'datadir' "$@")"
    SOCKET="$(mysql_get_config 'socket' "$@")"
    PORT="$(mysql_get_config 'port' "$@")"

    # Initialize values that might be stored in a file
    _mariadb_file_env 'MYSQL_ROOT_HOST' '%'
    _mariadb_file_env 'MYSQL_DATABASE'
    _mariadb_file_env 'MYSQL_USER'
    _mariadb_file_env 'MYSQL_PASSWORD'
    _mariadb_file_env 'MYSQL_ROOT_PASSWORD'
    # No MYSQL_ compatibility needed for new variables
    file_env 'MARIADB_PASSWORD_HASH'
    file_env 'MARIADB_ROOT_PASSWORD_HASH'
    # env variables related to replication
    file_env 'MARIADB_REPLICATION_USER'
    file_env 'MARIADB_REPLICATION_PASSWORD'
    file_env 'MARIADB_REPLICATION_PASSWORD_HASH'
    # env variables related to master
    file_env 'MARIADB_MASTER_HOST'
    file_env 'MARIADB_MASTER_PORT' 3306

    # set MARIADB_ from MYSQL_ when it is unset and then make them the same value
    : "${MARIADB_ALLOW_EMPTY_ROOT_PASSWORD:=${MYSQL_ALLOW_EMPTY_PASSWORD:-}}"
    export MYSQL_ALLOW_EMPTY_PASSWORD="$MARIADB_ALLOW_EMPTY_ROOT_PASSWORD" MARIADB_ALLOW_EMPTY_ROOT_PASSWORD
    : "${MARIADB_RANDOM_ROOT_PASSWORD:=${MYSQL_RANDOM_ROOT_PASSWORD:-}}"
    export MYSQL_RANDOM_ROOT_PASSWORD="$MARIADB_RANDOM_ROOT_PASSWORD" MARIADB_RANDOM_ROOT_PASSWORD
    : "${MARIADB_INITDB_SKIP_TZINFO:=${MYSQL_INITDB_SKIP_TZINFO:-}}"
    export MYSQL_INITDB_SKIP_TZINFO="$MARIADB_INITDB_SKIP_TZINFO" MARIADB_INITDB_SKIP_TZINFO

    declare -g DATABASE_ALREADY_EXISTS
    if [ -d "$DATADIR/mysql" ]; then
        DATABASE_ALREADY_EXISTS='true'
    fi
}

# Execute the client, use via docker_process_sql to handle root password
docker_exec_client() {
    # args sent in can override this db, since they will be later in the command
    if [ -n "$MYSQL_DATABASE" ]; then
        set -- --database="$MYSQL_DATABASE" "$@"
    fi
    mariadb --protocol=socket -uroot -hlocalhost --socket="${SOCKET}" "$@"
}

# Execute sql script, passed via stdin
# usage: docker_process_sql [--dont-use-mysql-root-password] [mysql-cli-args]
#    ie: docker_process_sql --database=mydb <<<'INSERT ...'
#    ie: docker_process_sql --dont-use-mysql-root-password --database=mydb <my-file.sql
docker_process_sql() {
    if [ '--dont-use-mysql-root-password' = "$1" ]; then
        shift
        MYSQL_PWD='' docker_exec_client "$@"
    else
        MYSQL_PWD=$MARIADB_ROOT_PASSWORD docker_exec_client "$@"
    fi
}

# SQL escape the string $1 to be placed in a string literal.
# escape, \ followed by '
docker_sql_escape_string_literal() {
    local newline=$'\n'
    local escaped=${1//\\/\\\\}
    escaped="${escaped//$newline/\\n}"
    echo "${escaped//\'/\\\'}"
}

# Creates replication user
create_replica_user() {
    if [ -n  "$MARIADB_REPLICATION_PASSWORD_HASH" ]; then
        echo "CREATE USER '$MARIADB_REPLICATION_USER'@'%' IDENTIFIED BY PASSWORD '$MARIADB_REPLICATION_PASSWORD_HASH';"
    else
        # SQL escape the user password, \ followed by '
        local userPasswordEscaped
        userPasswordEscaped=$(docker_sql_escape_string_literal "${MARIADB_REPLICATION_PASSWORD}")
        echo "CREATE USER '$MARIADB_REPLICATION_USER'@'%' IDENTIFIED BY '$userPasswordEscaped';"
    fi
    echo "GRANT REPLICATION REPLICA ON *.* TO '$MARIADB_REPLICATION_USER'@'%';"
}

# Create healthcheck users
create_healthcheck_users() {
    local healthCheckGrant=USAGE
    local healthCheckConnectPass
    local healthCheckConnectPassEscaped
    healthCheckConnectPass="$(pwgen --numerals --capitalize --symbols --remove-chars="=#'\\" -1 32)"
    healthCheckConnectPassEscaped=$(docker_sql_escape_string_literal "${healthCheckConnectPass}")
    if [ -n "$MARIADB_HEALTHCHECK_GRANTS" ]; then
        healthCheckGrant="$MARIADB_HEALTHCHECK_GRANTS"
    fi
    for host in 127.0.0.1 ::1 localhost; do
        echo "CREATE USER IF NOT EXISTS healthcheck@'$host' IDENTIFIED BY '$healthCheckConnectPassEscaped';"
        # doing this so if the users exists, we're just setting the password, and not replacing the existing grants
        echo "SET PASSWORD FOR healthcheck@'$host' = PASSWORD('$healthCheckConnectPassEscaped');"
        echo "GRANT $healthCheckGrant ON *.* TO healthcheck@'$host';"
    done
    local maskPreserve
    maskPreserve=$(umask -p)
    umask 0077
    echo -e "[mariadb-client]\\nport=$PORT\\nsocket=$SOCKET\\nuser=healthcheck\\npassword=$healthCheckConnectPass\\nprotocol=tcp\\n" > "$DATADIR"/.my-healthcheck.cnf
    $maskPreserve
}

# Initializes database with timezone info and root password, plus optional extra db/user
docker_setup_db() {
    # Load timezone info into database
    if [ -z "$MARIADB_INITDB_SKIP_TZINFO" ]; then
        # --skip-write-binlog usefully disables binary logging
        # but also outputs LOCK TABLES to improve the IO of
        # Aria (MDEV-23326) for 10.4+.
        mariadb-tzinfo-to-sql --skip-write-binlog /usr/share/zoneinfo \
            | docker_process_sql --dont-use-mysql-root-password --database=mysql
        # tell docker_process_sql to not use MYSQL_ROOT_PASSWORD since it is not set yet
    fi
    # Generate random root password
    if [ -n "$MARIADB_RANDOM_ROOT_PASSWORD" ]; then
        MARIADB_ROOT_PASSWORD="$(pwgen --numerals --capitalize --symbols --remove-chars="'\\" -1 32)"
        export MARIADB_ROOT_PASSWORD MYSQL_ROOT_PASSWORD=$MARIADB_ROOT_PASSWORD
        mysql_note "GENERATED ROOT PASSWORD: $MARIADB_ROOT_PASSWORD"
    fi

    # Creates root users for non-localhost hosts
    local rootCreate=
    local rootPasswordEscaped=
    if [ -n "$MARIADB_ROOT_PASSWORD" ]; then
        # Sets root password and creates root users for non-localhost hosts
        rootPasswordEscaped=$(docker_sql_escape_string_literal "${MARIADB_ROOT_PASSWORD}")
    fi

    # default root to listen for connections from anywhere
    if [ -n "$MARIADB_ROOT_HOST" ] && [ "$MARIADB_ROOT_HOST" != 'localhost' ]; then
        # ref "read -d ''", no, we don't care if read finds a terminating character in this heredoc
        # https://unix.stackexchange.com/questions/265149/why-is-set-o-errexit-breaking-this-read-heredoc-expression/265151#265151
        if [ -n "$MARIADB_ROOT_PASSWORD_HASH" ]; then
            read -r -d '' rootCreate <<-EOSQL || true
                CREATE USER 'root'@'${MARIADB_ROOT_HOST}' IDENTIFIED BY PASSWORD '${MARIADB_ROOT_PASSWORD_HASH}' ;
                GRANT ALL ON *.* TO 'root'@'${MARIADB_ROOT_HOST}' WITH GRANT OPTION ;
                GRANT PROXY ON ''@'%' TO 'root'@'${MARIADB_ROOT_HOST}' WITH GRANT OPTION;
            EOSQL
        else
            read -r -d '' rootCreate <<-EOSQL || true
                CREATE USER 'root'@'${MARIADB_ROOT_HOST}' IDENTIFIED BY '${rootPasswordEscaped}' ;
                GRANT ALL ON *.* TO 'root'@'${MARIADB_ROOT_HOST}' WITH GRANT OPTION ;
                GRANT PROXY ON ''@'%' TO 'root'@'${MARIADB_ROOT_HOST}' WITH GRANT OPTION;
            EOSQL
        fi
    fi

    local mysqlAtLocalhost=
    local mysqlAtLocalhostGrants=
    # Install mysql@localhost user
    if [ -n "$MARIADB_MYSQL_LOCALHOST_USER" ]; then
        read -r -d '' mysqlAtLocalhost <<-EOSQL || true
        CREATE USER mysql@localhost IDENTIFIED VIA unix_socket;
        EOSQL
        if [ -n "$MARIADB_MYSQL_LOCALHOST_GRANTS" ]; then
            if [ "$MARIADB_MYSQL_LOCALHOST_GRANTS" != USAGE ]; then
                mysql_warn "Excessive privileges ON *.* TO mysql@localhost facilitates risks to the confidentiality, integrity and availability of data stored"
            fi
            mysqlAtLocalhostGrants="GRANT ${MARIADB_MYSQL_LOCALHOST_GRANTS} ON *.* TO mysql@localhost;";
        fi
    fi

    local createHealthCheckUsers
    createHealthCheckUsers=$(create_healthcheck_users)

    local rootLocalhostPass=
    if [ -z "$MARIADB_ROOT_PASSWORD_HASH" ]; then
        # handle MARIADB_ROOT_PASSWORD_HASH for root@localhost after /docker-entrypoint-initdb.d
        rootLocalhostPass="SET PASSWORD FOR 'root'@'localhost'= PASSWORD('${rootPasswordEscaped}');"
    fi

    local createDatabase=
    # Creates a custom database and user if specified
    if [ -n "$MARIADB_DATABASE" ]; then
        mysql_note "Creating database ${MARIADB_DATABASE}"
        createDatabase="CREATE DATABASE IF NOT EXISTS \`$MARIADB_DATABASE\`;"
    fi

    local createUser=
    local userGrants=
    if  [ -n "$MARIADB_PASSWORD" ] || [ -n "$MARIADB_PASSWORD_HASH" ] && [ -n "$MARIADB_USER" ]; then
        mysql_note "Creating user ${MARIADB_USER}"
        if [ -n "$MARIADB_PASSWORD_HASH" ]; then
            createUser="CREATE USER '$MARIADB_USER'@'%' IDENTIFIED BY PASSWORD '$MARIADB_PASSWORD_HASH';"
        else
            # SQL escape the user password, \ followed by '
            local userPasswordEscaped
            userPasswordEscaped=$(docker_sql_escape_string_literal "${MARIADB_PASSWORD}")
            createUser="CREATE USER '$MARIADB_USER'@'%' IDENTIFIED BY '$userPasswordEscaped';"
        fi

        if [ -n "$MARIADB_DATABASE" ]; then
            mysql_note "Giving user ${MARIADB_USER} access to schema ${MARIADB_DATABASE}"
            userGrants="GRANT ALL ON \`${MARIADB_DATABASE//_/\\_}\`.* TO '$MARIADB_USER'@'%';"
        fi
    fi

    # To create replica user
    local createReplicaUser=
    local changeMasterTo=
    local startReplica=
    if  [ -n "$MARIADB_REPLICATION_USER" ] ; then
        if [ -z "$MARIADB_MASTER_HOST" ]; then
            # on master
            mysql_note "Creating user ${MARIADB_REPLICATION_USER}"
            createReplicaUser=$(create_replica_user)
        else
            # on replica
            local rplPasswordEscaped
            rplPasswordEscaped=$(docker_sql_escape_string_literal "${MARIADB_REPLICATION_PASSWORD}")
            # SC cannot follow how MARIADB_MASTER_PORT is assigned a default value.
            # shellcheck disable=SC2153
            changeMasterTo="CHANGE MASTER TO MASTER_HOST='$MARIADB_MASTER_HOST', MASTER_USER='$MARIADB_REPLICATION_USER', MASTER_PASSWORD='$rplPasswordEscaped', MASTER_PORT=$MARIADB_MASTER_PORT, MASTER_CONNECT_RETRY=10;"
            startReplica="START REPLICA;"
        fi
    fi

    mysql_note "Securing system users (equivalent to running mysql_secure_installation)"
    # tell docker_process_sql to not use MARIADB_ROOT_PASSWORD since it is just now being set
    # --binary-mode to save us from the semi-mad users go out of their way to confuse the encoding.
    docker_process_sql --dont-use-mysql-root-password --database=mysql --binary-mode <<-EOSQL
        -- Securing system users shouldn't be replicated
        SET @orig_sql_log_bin= @@SESSION.SQL_LOG_BIN;
        SET @@SESSION.SQL_LOG_BIN=0;
                -- we need the SQL_MODE NO_BACKSLASH_ESCAPES mode to be clear for the password to be set
        SET @@SESSION.SQL_MODE=REPLACE(@@SESSION.SQL_MODE, 'NO_BACKSLASH_ESCAPES', '');

        DROP USER IF EXISTS root@'127.0.0.1', root@'::1';
        EXECUTE IMMEDIATE CONCAT('DROP USER IF EXISTS root@\'', @@hostname,'\'');

        ${rootLocalhostPass}
        ${rootCreate}
        ${mysqlAtLocalhost}
        ${mysqlAtLocalhostGrants}
        ${createHealthCheckUsers}
        -- end of securing system users, rest of init now...
        SET @@SESSION.SQL_LOG_BIN=@orig_sql_log_bin;
        -- create users/databases
        ${createDatabase}
        ${createUser}
        ${createReplicaUser}
        ${userGrants}

        ${changeMasterTo}
        ${startReplica}
    EOSQL
}

# create a new installation
docker_mariadb_init()
{

    # check dir permissions to reduce likelihood of half-initialized database
    ls /docker-entrypoint-initdb.d/ > /dev/null

    if [ -n "$DATABASE_INIT_FROM_BACKUP" ]; then
        shopt -s dotglob
        for file in /docker-entrypoint-initdb.d/*.tar{.gz,.xz,.zst}; do
            mkdir -p "$DATADIR"/.init
            tar --auto-compress --extract --file "$file" --directory="$DATADIR"/.init
            mariadb-backup --target-dir="$DATADIR"/.init --datadir="$DATADIR"/.restore --move-back

            mv "$DATADIR"/.restore/** "$DATADIR"/
            if [ -f "$DATADIR/.init/backup-my.cnf" ]; then
                mv "$DATADIR/.init/backup-my.cnf" "$DATADIR/.my.cnf"
                mysql_note "Adding startup configuration:"
                my_print_defaults --defaults-file="$DATADIR/.my.cnf" --mariadbd
            fi
            rm -rf "$DATADIR"/.init "$DATADIR"/.restore
            if [ "$(id -u)" = "0" ]; then
                # this will cause less disk access than `chown -R`
                find "$DATADIR" \! -user mysql -exec chown mysql: '{}' +
            fi
        done
        if _check_if_upgrade_is_needed; then
            docker_mariadb_upgrade "$@"
        fi
        return
    fi
    docker_init_database_dir "$@"

    mysql_note "Starting temporary server"
    docker_temp_server_start "$@"
    mysql_note "Temporary server started."

    docker_setup_db
    docker_process_init_files /docker-entrypoint-initdb.d/*
    # Wait until after /docker-entrypoint-initdb.d is performed before setting
    # root@localhost password to a hash we don't know the password for.
    if [ -n "${MARIADB_ROOT_PASSWORD_HASH}" ]; then
        mysql_note "Setting root@localhost password hash"
        docker_process_sql --dont-use-mysql-root-password --binary-mode <<-EOSQL
            SET @@SESSION.SQL_LOG_BIN=0;
            SET PASSWORD FOR 'root'@'localhost'= '${MARIADB_ROOT_PASSWORD_HASH}';
        EOSQL
    fi

    mysql_note "Stopping temporary server"
    docker_temp_server_stop
    mysql_note "Temporary server stopped"

    echo
    mysql_note "MariaDB init process done. Ready for start up."
    echo
}

# backup the mysql database
docker_mariadb_backup_system()
{
    if [ -n "$MARIADB_DISABLE_UPGRADE_BACKUP" ] \
        && [ "$MARIADB_DISABLE_UPGRADE_BACKUP" = 1 ]; then
        mysql_note "MariaDB upgrade backup disabled due to \$MARIADB_DISABLE_UPGRADE_BACKUP=1 setting"
        return
    fi
    local backup_db="system_mysql_backup_unknown_version.sql.zst"
    local oldfullversion="unknown_version"
    if [ -r "$DATADIR"/mariadb_upgrade_info ]; then
        read -r -d '' oldfullversion < "$DATADIR"/mariadb_upgrade_info || true
        if [ -n "$oldfullversion" ]; then
            backup_db="system_mysql_backup_${oldfullversion}.sql.zst"
        fi
    fi

    mysql_note "Backing up system database to $backup_db"
    if ! mariadb-dump --skip-lock-tables --replace --databases mysql --socket="${SOCKET}" | zstd > "${DATADIR}/${backup_db}"; then
        mysql_error "Unable backup system database for upgrade from $oldfullversion."
    fi
    mysql_note "Backing up complete"
}

# perform mariadb-upgrade
# backup the mysql database if this is a major upgrade
docker_mariadb_upgrade() {
    if [ -z "$MARIADB_AUTO_UPGRADE" ] \
        || [ "$MARIADB_AUTO_UPGRADE" = 0 ]; then
        mysql_note "MariaDB upgrade (mariadb-upgrade or creating healthcheck users) required, but skipped due to \$MARIADB_AUTO_UPGRADE setting"
        return
    fi
    mysql_note "Starting temporary server"
    docker_temp_server_start "$@" --skip-grant-tables \
        --loose-innodb_buffer_pool_dump_at_shutdown=0 \
        --skip-slave-start
    mysql_note "Temporary server started."

    docker_mariadb_backup_system

    if [ ! -f "$DATADIR"/.my-healthcheck.cnf ]; then
        mysql_note "Creating healthcheck users"
        local createHealthCheckUsers
        createHealthCheckUsers=$(create_healthcheck_users)
        docker_process_sql --dont-use-mysql-root-password --binary-mode <<-EOSQL
        -- Healthcheck users shouldn't be replicated
        SET @@SESSION.SQL_LOG_BIN=0;
                -- we need the SQL_MODE NO_BACKSLASH_ESCAPES mode to be clear for the password to be set
        SET @@SESSION.SQL_MODE=REPLACE(@@SESSION.SQL_MODE, 'NO_BACKSLASH_ESCAPES', '');
        FLUSH PRIVILEGES;
        $createHealthCheckUsers
EOSQL
        mysql_note "Stopping temporary server"
        docker_temp_server_stop
        mysql_note "Temporary server stopped"

        if _check_if_upgrade_is_needed; then
            # need a restart as FLUSH PRIVILEGES isn't reversable
            mysql_note "Restarting temporary server for upgrade"
            docker_temp_server_start "$@" --skip-grant-tables \
                --loose-innodb_buffer_pool_dump_at_shutdown=0 \
                --skip-slave-start
        else
            return 0
        fi
    fi

    mysql_note "Starting mariadb-upgrade"
    mariadb-upgrade --upgrade-system-tables
    mysql_note "Finished mariadb-upgrade"

    mysql_note "Stopping temporary server"
    docker_temp_server_stop
    mysql_note "Temporary server stopped"
}

_check_if_upgrade_is_needed() {
    if [ ! -f "$DATADIR"/mariadb_upgrade_info ]; then
        mysql_note "MariaDB upgrade information missing, assuming required"
        return 0
    fi
    local mariadbVersion
    mariadbVersion="$(_mariadb_version)"
    IFS='.-' read -ra newversion <<<"$mariadbVersion"
    IFS='.-' read -ra oldversion < "$DATADIR"/mariadb_upgrade_info || true

    if [[ ${#newversion[@]} -lt 2 ]] || [[ ${#oldversion[@]} -lt 2 ]] \
        || [[ ${oldversion[0]} -lt ${newversion[0]} ]] \
        || [[ ${oldversion[0]} -eq ${newversion[0]} && ${oldversion[1]} -lt ${newversion[1]} ]]; then
        return 0
    fi
    if [ ! -f "$DATADIR"/.my-healthcheck.cnf ]; then
        mysql_note "MariaDB heathcheck configation file missing, assuming desirable"
        return 0
    fi
    mysql_note "MariaDB upgrade not required"
    return 1
}

# check arguments for an option that would cause mariadbd to stop
# return true if there is one
_mysql_want_help() {
    local arg
    for arg; do
        case "$arg" in
            -'?'|--help|--print-defaults|-V|--version)
                return 0
                ;;
        esac
    done
    return 1
}

_main() {
    # if command starts with an option, prepend mariadbd
    if [ "${1:0:1}" = '-' ]; then
        set -- mariadbd "$@"
    fi

    #ENDOFSUBSTITUTIONS
    # skip setup if they aren't running mysqld or want an option that stops mysqld
    if [ "$1" = 'mariadbd' ] || [ "$1" = 'mysqld' ] && ! _mysql_want_help "$@"; then
        mysql_note "Entrypoint script for MariaDB Server ${MARIADB_VERSION} started."

        mysql_check_config "$@"
        # Load various environment variables
        docker_setup_env "$@"
        docker_create_db_directories

        # If container is started as root user, restart as dedicated mysql user
        if [ "$(id -u)" = "0" ]; then
            mysql_note "Switching to dedicated user 'mysql'"
            exec gosu mysql "${BASH_SOURCE[0]}" "$@"
        fi

        # there's no database, so it needs to be initialized
        if [ -z "$DATABASE_ALREADY_EXISTS" ]; then
            docker_verify_minimum_env

            docker_mariadb_init "$@"
        # MDEV-27636 mariadb_upgrade --check-if-upgrade-is-needed cannot be run offline
        #elif mariadb-upgrade --check-if-upgrade-is-needed; then
        elif _check_if_upgrade_is_needed; then
            docker_mariadb_upgrade "$@"
        fi
    fi
    exec "$@"
}

# If we are sourced from elsewhere, don't perform any further actions
if ! _is_sourced; then
    _main "$@"
fi
+ _is_sourced
+ '[' 2 -ge 2 ']'
+ '[' _is_sourced = _is_sourced ']'
+ '[' main = source ']'
+ _main mariadbd
+ '[' m = - ']'
+ '[' mariadbd = mariadbd ']'
+ _mysql_want_help mariadbd
+ local arg
+ for arg in "$@"
+ case "$arg" in
+ return 1
+ mysql_note 'Entrypoint script for MariaDB Server 1:11.4.2+maria~ubu2404 started.'
+ mysql_log Note 'Entrypoint script for MariaDB Server 1:11.4.2+maria~ubu2404 started.'
+ local type=Note
+ shift
++ date --rfc-3339=seconds
+ printf '%s [%s] [Entrypoint]: %s\n' '2024-06-10 09:25:36+00:00' Note 'Entrypoint script for MariaDB Server 1:11.4.2+maria~ubu2404 started.'
+ mysql_check_config mariadbd
2024-06-10 09:25:36+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:11.4.2+maria~ubu2404 started.
+ toRun=('mariadbd' '--verbose' '--help')
+ local toRun errors
++ mariadbd --verbose --help
+ errors=
+ docker_setup_env mariadbd
+ declare -g DATADIR SOCKET PORT
++ mysql_get_config datadir mariadbd
++ local conf=datadir
++ shift
++ mariadbd --verbose --help
++ awk -v conf=datadir '$1 == conf && /^[^ \t]/ { sub(/^[^ \t]+[ \t]+/, ""); print; exit }'
+ DATADIR=/var/lib/mysql/
++ mysql_get_config socket mariadbd
++ local conf=socket
++ shift
++ mariadbd --verbose --help
++ awk -v conf=socket '$1 == conf && /^[^ \t]/ { sub(/^[^ \t]+[ \t]+/, ""); print; exit }'
+ SOCKET=/run/mysqld/mysqld.sock
++ mysql_get_config port mariadbd
++ local conf=port
++ shift
++ mariadbd --verbose --help
++ awk -v conf=port '$1 == conf && /^[^ \t]/ { sub(/^[^ \t]+[ \t]+/, ""); print; exit }'
+ PORT=3306
+ _mariadb_file_env MYSQL_ROOT_HOST %
+ local var=MYSQL_ROOT_HOST
+ shift
+ local maria=MARIADB_ROOT_HOST
+ file_env MYSQL_ROOT_HOST %
+ local var=MYSQL_ROOT_HOST
+ local fileVar=MYSQL_ROOT_HOST_FILE
+ local def=%
+ '[' '' ']'
+ local val=%
+ '[' '' ']'
+ '[' '' ']'
+ export MYSQL_ROOT_HOST=%
+ MYSQL_ROOT_HOST=%
+ unset MYSQL_ROOT_HOST_FILE
+ file_env MARIADB_ROOT_HOST %
+ local var=MARIADB_ROOT_HOST
+ local fileVar=MARIADB_ROOT_HOST_FILE
+ local def=%
+ '[' '' ']'
+ local val=%
+ '[' '' ']'
+ '[' '' ']'
+ export MARIADB_ROOT_HOST=%
+ MARIADB_ROOT_HOST=%
+ unset MARIADB_ROOT_HOST_FILE
+ '[' % ']'
+ export MYSQL_ROOT_HOST=%
+ MYSQL_ROOT_HOST=%
+ _mariadb_file_env MYSQL_DATABASE
+ local var=MYSQL_DATABASE
+ shift
+ local maria=MARIADB_DATABASE
+ file_env MYSQL_DATABASE
+ local var=MYSQL_DATABASE
+ local fileVar=MYSQL_DATABASE_FILE
+ local def=
+ '[' '' ']'
+ local val=
+ '[' '' ']'
+ '[' '' ']'
+ export MYSQL_DATABASE=
+ MYSQL_DATABASE=
+ unset MYSQL_DATABASE_FILE
+ file_env MARIADB_DATABASE ''
+ local var=MARIADB_DATABASE
+ local fileVar=MARIADB_DATABASE_FILE
+ local def=
+ '[' '' ']'
+ local val=
+ '[' '' ']'
+ '[' '' ']'
+ export MARIADB_DATABASE=
+ MARIADB_DATABASE=
+ unset MARIADB_DATABASE_FILE
+ '[' '' ']'
+ _mariadb_file_env MYSQL_USER
+ local var=MYSQL_USER
+ shift
+ local maria=MARIADB_USER
+ file_env MYSQL_USER
+ local var=MYSQL_USER
+ local fileVar=MYSQL_USER_FILE
+ local def=
+ '[' '' ']'
+ local val=
+ '[' '' ']'
+ '[' '' ']'
+ export MYSQL_USER=
+ MYSQL_USER=
+ unset MYSQL_USER_FILE
+ file_env MARIADB_USER ''
+ local var=MARIADB_USER
+ local fileVar=MARIADB_USER_FILE
+ local def=
+ '[' '' ']'
+ local val=
+ '[' '' ']'
+ '[' '' ']'
+ export MARIADB_USER=
+ MARIADB_USER=
+ unset MARIADB_USER_FILE
+ '[' '' ']'
+ _mariadb_file_env MYSQL_PASSWORD
+ local var=MYSQL_PASSWORD
+ shift
+ local maria=MARIADB_PASSWORD
+ file_env MYSQL_PASSWORD
+ local var=MYSQL_PASSWORD
+ local fileVar=MYSQL_PASSWORD_FILE
+ local def=
+ '[' '' ']'
+ local val=
+ '[' '' ']'
+ '[' '' ']'
+ export MYSQL_PASSWORD=
+ MYSQL_PASSWORD=
+ unset MYSQL_PASSWORD_FILE
+ file_env MARIADB_PASSWORD ''
+ local var=MARIADB_PASSWORD
+ local fileVar=MARIADB_PASSWORD_FILE
+ local def=
+ '[' '' ']'
+ local val=
+ '[' '' ']'
+ '[' '' ']'
+ export MARIADB_PASSWORD=
+ MARIADB_PASSWORD=
+ unset MARIADB_PASSWORD_FILE
+ '[' '' ']'
+ _mariadb_file_env MYSQL_ROOT_PASSWORD
+ local var=MYSQL_ROOT_PASSWORD
+ shift
+ local maria=MARIADB_ROOT_PASSWORD
+ file_env MYSQL_ROOT_PASSWORD
+ local var=MYSQL_ROOT_PASSWORD
+ local fileVar=MYSQL_ROOT_PASSWORD_FILE
+ local def=
+ '[' '' ']'
+ local val=
+ '[' '' ']'
+ '[' '' ']'
+ export MYSQL_ROOT_PASSWORD=
+ MYSQL_ROOT_PASSWORD=
+ unset MYSQL_ROOT_PASSWORD_FILE
+ file_env MARIADB_ROOT_PASSWORD ''
+ local var=MARIADB_ROOT_PASSWORD
+ local fileVar=MARIADB_ROOT_PASSWORD_FILE
+ local def=
+ '[' bob ']'
+ '[' '' ']'
+ local val=
+ '[' bob ']'
+ val=bob
+ export MARIADB_ROOT_PASSWORD=bob
+ MARIADB_ROOT_PASSWORD=bob
+ unset MARIADB_ROOT_PASSWORD_FILE
+ '[' bob ']'
+ export MYSQL_ROOT_PASSWORD=bob
+ MYSQL_ROOT_PASSWORD=bob
+ file_env MARIADB_PASSWORD_HASH
+ local var=MARIADB_PASSWORD_HASH
+ local fileVar=MARIADB_PASSWORD_HASH_FILE
+ local def=
+ '[' '' ']'
+ local val=
+ '[' '' ']'
+ '[' '' ']'
+ export MARIADB_PASSWORD_HASH=
+ MARIADB_PASSWORD_HASH=
+ unset MARIADB_PASSWORD_HASH_FILE
+ file_env MARIADB_ROOT_PASSWORD_HASH
+ local var=MARIADB_ROOT_PASSWORD_HASH
+ local fileVar=MARIADB_ROOT_PASSWORD_HASH_FILE
+ local def=
+ '[' '' ']'
+ local val=
+ '[' '' ']'
+ '[' '' ']'
+ export MARIADB_ROOT_PASSWORD_HASH=
+ MARIADB_ROOT_PASSWORD_HASH=
+ unset MARIADB_ROOT_PASSWORD_HASH_FILE
+ file_env MARIADB_REPLICATION_USER
+ local var=MARIADB_REPLICATION_USER
+ local fileVar=MARIADB_REPLICATION_USER_FILE
+ local def=
+ '[' '' ']'
+ local val=
+ '[' '' ']'
+ '[' '' ']'
+ export MARIADB_REPLICATION_USER=
+ MARIADB_REPLICATION_USER=
+ unset MARIADB_REPLICATION_USER_FILE
+ file_env MARIADB_REPLICATION_PASSWORD
+ local var=MARIADB_REPLICATION_PASSWORD
+ local fileVar=MARIADB_REPLICATION_PASSWORD_FILE
+ local def=
+ '[' '' ']'
+ local val=
+ '[' '' ']'
+ '[' '' ']'
+ export MARIADB_REPLICATION_PASSWORD=
+ MARIADB_REPLICATION_PASSWORD=
+ unset MARIADB_REPLICATION_PASSWORD_FILE
+ file_env MARIADB_REPLICATION_PASSWORD_HASH
+ local var=MARIADB_REPLICATION_PASSWORD_HASH
+ local fileVar=MARIADB_REPLICATION_PASSWORD_HASH_FILE
+ local def=
+ '[' '' ']'
+ local val=
+ '[' '' ']'
+ '[' '' ']'
+ export MARIADB_REPLICATION_PASSWORD_HASH=
+ MARIADB_REPLICATION_PASSWORD_HASH=
+ unset MARIADB_REPLICATION_PASSWORD_HASH_FILE
+ file_env MARIADB_MASTER_HOST
+ local var=MARIADB_MASTER_HOST
+ local fileVar=MARIADB_MASTER_HOST_FILE
+ local def=
+ '[' '' ']'
+ local val=
+ '[' '' ']'
+ '[' '' ']'
+ export MARIADB_MASTER_HOST=
+ MARIADB_MASTER_HOST=
+ unset MARIADB_MASTER_HOST_FILE
+ file_env MARIADB_MASTER_PORT 3306
+ local var=MARIADB_MASTER_PORT
+ local fileVar=MARIADB_MASTER_PORT_FILE
+ local def=3306
+ '[' '' ']'
+ local val=3306
+ '[' '' ']'
+ '[' '' ']'
+ export MARIADB_MASTER_PORT=3306
+ MARIADB_MASTER_PORT=3306
+ unset MARIADB_MASTER_PORT_FILE
+ : ''
+ export MYSQL_ALLOW_EMPTY_PASSWORD= MARIADB_ALLOW_EMPTY_ROOT_PASSWORD
+ MYSQL_ALLOW_EMPTY_PASSWORD=
+ : ''
+ export MYSQL_RANDOM_ROOT_PASSWORD= MARIADB_RANDOM_ROOT_PASSWORD
+ MYSQL_RANDOM_ROOT_PASSWORD=
+ : ''
+ export MYSQL_INITDB_SKIP_TZINFO= MARIADB_INITDB_SKIP_TZINFO
+ MYSQL_INITDB_SKIP_TZINFO=
+ declare -g DATABASE_ALREADY_EXISTS
+ '[' -d /var/lib/mysql//mysql ']'
+ DATABASE_ALREADY_EXISTS=true
+ docker_create_db_directories
+ local user
++ id -u
+ user=999
+ mkdir -p /var/lib/mysql/
+ '[' 999 = 0 ']'
++ id -u
+ '[' 999 = 0 ']'
+ '[' -z true ']'
+ _check_if_upgrade_is_needed
+ '[' '!' -f /var/lib/mysql//mariadb_upgrade_info ']'
+ local mariadbVersion
++ _mariadb_version
++ echo -n 11.4.2-MariaDB
+ mariadbVersion=11.4.2-MariaDB
+ IFS=.-
+ read -ra newversion
+ IFS=.-
+ read -ra oldversion
+ true
+ [[ 4 -lt 2 ]]
+ [[ 4 -lt 2 ]]
+ [[ 11 -lt 11 ]]
+ [[ 11 -eq 11 ]]
+ [[ 4 -lt 4 ]]
+ '[' '!' -f /var/lib/mysql//.my-healthcheck.cnf ']'
+ mysql_note 'MariaDB heathcheck configation file missing, assuming desirable'
+ mysql_log Note 'MariaDB heathcheck configation file missing, assuming desirable'
+ local type=Note
+ shift
++ date --rfc-3339=seconds
2024-06-10 09:25:37+00:00 [Note] [Entrypoint]: MariaDB heathcheck configation file missing, assuming desirable
+ printf '%s [%s] [Entrypoint]: %s\n' '2024-06-10 09:25:37+00:00' Note 'MariaDB heathcheck configation file missing, assuming desirable'
+ return 0
+ docker_mariadb_upgrade mariadbd
+ '[' -z 1 ']'
+ '[' 1 = 0 ']'
+ mysql_note 'Starting temporary server'
+ mysql_log Note 'Starting temporary server'
+ local type=Note
+ shift
++ date --rfc-3339=seconds
2024-06-10 09:25:37+00:00 [Note] [Entrypoint]: Starting temporary server
+ printf '%s [%s] [Entrypoint]: %s\n' '2024-06-10 09:25:37+00:00' Note 'Starting temporary server'
+ docker_temp_server_start mariadbd --skip-grant-tables --loose-innodb_buffer_pool_dump_at_shutdown=0 --skip-slave-start
+ declare -g MARIADB_PID
+ MARIADB_PID=29
+ mysql_note 'Waiting for server startup'
+ mysql_log Note 'Waiting for server startup'
+ local type=Note
+ shift
+ mariadbd --skip-grant-tables --loose-innodb_buffer_pool_dump_at_shutdown=0 --skip-slave-start --skip-networking --default-time-zone=SYSTEM --socket=/run/mysqld/mysqld.sock --wsrep_on=OFF --expire-logs-days=0 --loose-innodb_buffer_pool_load_at_startup=0
++ date --rfc-3339=seconds
2024-06-10 09:25:37+00:00 [Note] [Entrypoint]: Waiting for server startup
+ printf '%s [%s] [Entrypoint]: %s\n' '2024-06-10 09:25:37+00:00' Note 'Waiting for server startup'
+ extraArgs=()
+ '[' -z true ']'
+ local i
+ for i in {30..0}
+ docker_process_sql --database=mysql
+ sleep 1
2024-06-10  9:25:37 0 [Note] Starting MariaDB 11.4.2-MariaDB-ubu2404 source revision 3fca5ed772fb75e3e57c507edef2985f8eba5b12 as process 29
2024-06-10  9:25:37 0 [Note] InnoDB: Compressed tables use zlib 1.3
2024-06-10  9:25:37 0 [Note] InnoDB: Number of transaction pools: 1
2024-06-10  9:25:37 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
2024-06-10  9:25:37 0 [Note] mariadbd: O_TMPFILE is not supported on /tmp (disabling future attempts)
2024-06-10  9:25:37 0 [Note] InnoDB: Initializing buffer pool, total size = 128.000MiB, chunk size = 2.000MiB
2024-06-10  9:25:37 0 [Note] InnoDB: Completed initialization of buffer pool
2024-06-10  9:25:37 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes)
2024-06-10  9:25:37 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=46300
2024-06-10  9:25:37 0 [Note] InnoDB: End of log at LSN=47943
2024-06-10  9:25:37 0 [Note] InnoDB: To recover: 22 pages
2024-06-10  9:25:37 0 [Note] InnoDB: Opened 3 undo tablespaces
2024-06-10  9:25:37 0 [Note] InnoDB: 128 rollback segments in 3 undo tablespaces are active.
2024-06-10  9:25:37 0 [Note] InnoDB: Removed temporary tablespace data file: "./ibtmp1"
2024-06-10  9:25:37 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ...
2024-06-10  9:25:37 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB.
2024-06-10  9:25:37 0 [Note] InnoDB: log sequence number 47943; transaction id 15
2024-06-10  9:25:37 0 [Note] Plugin 'FEEDBACK' is disabled.
2024-06-10  9:25:37 0 [Note] Plugin 'wsrep-provider' is disabled.
2024-06-10  9:25:37 0 [Note] Recovering after a crash using tc.log
2024-06-10  9:25:37 0 [Note] Starting table crash recovery...
2024-06-10  9:25:37 0 [Note] Crash table recovery finished.
2024-06-10  9:25:37 0 [Note] mariadbd: ready for connections.
Version: '11.4.2-MariaDB-ubu2404'  socket: '/run/mysqld/mysqld.sock'  port: 0  mariadb.org binary distribution
+ for i in {30..0}
+ docker_process_sql --database=mysql
2024-06-10  9:25:38 3 [Warning] Aborted connection 3 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
+ sleep 1
+ for i in {30..0}
+ docker_process_sql --database=mysql
2024-06-10  9:25:39 4 [Warning] Aborted connection 4 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
+ sleep 1
+ for i in {30..0}
+ docker_process_sql --database=mysql
2024-06-10  9:25:40 5 [Warning] Aborted connection 5 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
+ sleep 1
+ for i in {30..0}
+ docker_process_sql --database=mysql
2024-06-10  9:25:41 6 [Warning] Aborted connection 6 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
+ sleep 1
+ for i in {30..0}
+ docker_process_sql --database=mysql
2024-06-10  9:25:42 7 [Warning] Aborted connection 7 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
+ sleep 1
+ for i in {30..0}
+ docker_process_sql --database=mysql
2024-06-10  9:25:43 8 [Warning] Aborted connection 8 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
+ sleep 1
+ for i in {30..0}
+ docker_process_sql --database=mysql
2024-06-10  9:25:44 9 [Warning] Aborted connection 9 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
+ sleep 1
+ for i in {30..0}
+ docker_process_sql --database=mysql
2024-06-10  9:25:45 10 [Warning] Aborted connection 10 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
+ sleep 1
+ for i in {30..0}
+ docker_process_sql --database=mysql
2024-06-10  9:25:46 11 [Warning] Aborted connection 11 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
+ sleep 1
+ for i in {30..0}
+ docker_process_sql --database=mysql
2024-06-10  9:25:47 12 [Warning] Aborted connection 12 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
+ sleep 1
+ for i in {30..0}
+ docker_process_sql --database=mysql
2024-06-10  9:25:48 13 [Warning] Aborted connection 13 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
+ sleep 1
+ for i in {30..0}
+ docker_process_sql --database=mysql
2024-06-10  9:25:49 14 [Warning] Aborted connection 14 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
+ sleep 1
+ for i in {30..0}
+ docker_process_sql --database=mysql
2024-06-10  9:25:50 15 [Warning] Aborted connection 15 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
+ sleep 1
+ for i in {30..0}
+ docker_process_sql --database=mysql
2024-06-10  9:25:51 16 [Warning] Aborted connection 16 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
+ sleep 1
+ for i in {30..0}
+ docker_process_sql --database=mysql
2024-06-10  9:25:53 17 [Warning] Aborted connection 17 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
+ sleep 1
+ for i in {30..0}
+ docker_process_sql --database=mysql
2024-06-10  9:25:54 18 [Warning] Aborted connection 18 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
+ sleep 1
+ for i in {30..0}
+ docker_process_sql --database=mysql
2024-06-10  9:25:55 19 [Warning] Aborted connection 19 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
+ sleep 1
+ for i in {30..0}
+ docker_process_sql --database=mysql
2024-06-10  9:25:56 20 [Warning] Aborted connection 20 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
+ sleep 1
+ for i in {30..0}
+ docker_process_sql --database=mysql
2024-06-10  9:25:57 21 [Warning] Aborted connection 21 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
+ sleep 1
+ for i in {30..0}
+ docker_process_sql --database=mysql
2024-06-10  9:25:58 22 [Warning] Aborted connection 22 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
+ sleep 1
+ for i in {30..0}
+ docker_process_sql --database=mysql
2024-06-10  9:25:59 23 [Warning] Aborted connection 23 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
+ sleep 1
+ for i in {30..0}
+ docker_process_sql --database=mysql
2024-06-10  9:26:00 24 [Warning] Aborted connection 24 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
+ sleep 1
+ for i in {30..0}
+ docker_process_sql --database=mysql
2024-06-10  9:26:01 25 [Warning] Aborted connection 25 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
+ sleep 1
+ for i in {30..0}
+ docker_process_sql --database=mysql
2024-06-10  9:26:02 26 [Warning] Aborted connection 26 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
+ sleep 1
+ for i in {30..0}
+ docker_process_sql --database=mysql
2024-06-10  9:26:03 27 [Warning] Aborted connection 27 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
+ sleep 1
+ for i in {30..0}
+ docker_process_sql --database=mysql
2024-06-10  9:26:04 28 [Warning] Aborted connection 28 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
+ sleep 1
+ for i in {30..0}
+ docker_process_sql --database=mysql
2024-06-10  9:26:05 29 [Warning] Aborted connection 29 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
+ sleep 1
+ for i in {30..0}
+ docker_process_sql --database=mysql
2024-06-10  9:26:06 30 [Warning] Aborted connection 30 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
+ sleep 1
+ for i in {30..0}
+ docker_process_sql --database=mysql
2024-06-10  9:26:07 31 [Warning] Aborted connection 31 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
+ sleep 1
+ for i in {30..0}
+ docker_process_sql --database=mysql
2024-06-10  9:26:08 32 [Warning] Aborted connection 32 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
+ sleep 1
+ '[' 0 = 0 ']'
+ mysql_error 'Unable to start server.'
+ mysql_log ERROR 'Unable to start server.'
+ local type=ERROR
+ shift
++ date --rfc-3339=seconds
+ printf '%s [%s] [Entrypoint]: %s\n' '2024-06-10 09:26:09+00:00' ERROR 'Unable to start server.'
2024-06-10 09:26:09+00:00 [ERROR] [Entrypoint]: Unable to start server.
+ exit 1
crysknife commented 3 weeks ago

In 11.4, --ssl-verify-server-cert effectively became the default (MDEV-31857). In the cases outlined here, it should be implicitly turned off by maria client due to certain rules met. But if you are using an older (10.11) client with a newer (11.4) MariaDB, this won't happen. Could that be the case here?

@LinuxJedi For my unserstanding mariadb-upgrade runs in the same (11.4) container. It happens in a "fresh" setup with 11.4 too. So I think there is no older client in use.

True, mariadb-upgrade or just the mariadb cli client runs in the same container (11.4). It is however, also true that the new default of ssl-verify-server-cert seems the culprit here. It also fails when I explicitly activate this with in MariaDB containers < 11.4.

On one hand this is logical when a tcp connection is being used to connect locally to the MariaDB service via localhost or 127.0.0.1, while the certificate typically doesn't include localhost nor the private IP as SubjectAlternativeName, that such a verification fails. On the other hand, when a socket is used there, neither of these are used anyway. Why is it then validating?

Is there an option for the cli clients to assume a hostname even when a socket is used to connect to the db?

grooverdan commented 3 weeks ago
$ docker run --rm -v ./:/etc/mysql/conf.d:z --env MARIADB_ROOT_PASSWORD=bob -v ./dbdata:/var/lib/mysql --env MARIADB_AUTO_UPGRADE=1   mariadb:11.4 my_print_defaults --mariadbd
--socket=/run/mysqld/mysqld.sock
--host-cache-size=0
--skip-name-resolve
--pid-file=/run/mysqld/mysqld.pid
--basedir=/usr
--expire_logs_days=10
--character-set-server=utf8mb4
--character-set-collations=utf8mb4=uca1400_ai_ci
--ssl=1
--ssl-cert=/etc/mysql/conf.d/cert.pem
--ssl-key=/etc/mysql/conf.d/key.pem

ssl-ca may help make the connection resolveable - https://mariadb.com/kb/en/ssltls-system-variables/#ssl_ca

I haven't yet found and client and/server option(s) that used in combination will allow a connection to a temp server started with --skip-networking --skip-grant-tables. Even something as brute force as:

diff --git a/11.4/docker-entrypoint.sh b/11.4/docker-entrypoint.sh
index e5b4aa9..e05a58d 100755
--- a/11.4/docker-entrypoint.sh
+++ b/11.4/docker-entrypoint.sh
@@ -121,7 +121,7 @@ mysql_get_config() {
 docker_temp_server_start() {
        "$@" --skip-networking --default-time-zone=SYSTEM --socket="${SOCKET}" --wsrep_on=OFF \
                --expire-logs-days=0 \
-               --loose-innodb_buffer_pool_load_at_startup=0 &
+               --loose-innodb_buffer_pool_load_at_startup=0 --skip-ssl &
        declare -g MARIADB_PID
        MARIADB_PID=$!
        mysql_note "Waiting for server startup"
@@ -133,7 +133,7 @@ docker_temp_server_start() {
        fi
        local i
        for i in {30..0}; do
-               if docker_process_sql "${extraArgs[@]}" --database=mysql <<<'SELECT 1' &> /dev/null; then
+               if docker_process_sql "${extraArgs[@]}" --database=mysql --skip-ssl --skip-ssl-verify-server-cert  <<<'SELECT 1' &> /dev/null; then
                        break
                fi
                sleep 1

Appears to fail even without certificates specified.

crysknife commented 3 weeks ago
$ docker run --rm -v ./:/etc/mysql/conf.d:z --env MARIADB_ROOT_PASSWORD=bob -v ./dbdata:/var/lib/mysql --env MARIADB_AUTO_UPGRADE=1   mariadb:11.4 my_print_defaults --mariadbd
--socket=/run/mysqld/mysqld.sock
--host-cache-size=0
--skip-name-resolve
--pid-file=/run/mysqld/mysqld.pid
--basedir=/usr
--expire_logs_days=10
--character-set-server=utf8mb4
--character-set-collations=utf8mb4=uca1400_ai_ci
--ssl=1
--ssl-cert=/etc/mysql/conf.d/cert.pem
--ssl-key=/etc/mysql/conf.d/key.pem

ssl-ca may help make the connection resolveable - https://mariadb.com/kb/en/ssltls-system-variables/#ssl_ca

I haven't yet found and client and/server option(s) that used in combination will allow a connection to a temp server started with --skip-networking --skip-grant-tables. Even something as brute force as:

diff --git a/11.4/docker-entrypoint.sh b/11.4/docker-entrypoint.sh
index e5b4aa9..e05a58d 100755
--- a/11.4/docker-entrypoint.sh
+++ b/11.4/docker-entrypoint.sh
@@ -121,7 +121,7 @@ mysql_get_config() {
 docker_temp_server_start() {
        "$@" --skip-networking --default-time-zone=SYSTEM --socket="${SOCKET}" --wsrep_on=OFF \
                --expire-logs-days=0 \
-               --loose-innodb_buffer_pool_load_at_startup=0 &
+               --loose-innodb_buffer_pool_load_at_startup=0 --skip-ssl &
        declare -g MARIADB_PID
        MARIADB_PID=$!
        mysql_note "Waiting for server startup"
@@ -133,7 +133,7 @@ docker_temp_server_start() {
        fi
        local i
        for i in {30..0}; do
-               if docker_process_sql "${extraArgs[@]}" --database=mysql <<<'SELECT 1' &> /dev/null; then
+               if docker_process_sql "${extraArgs[@]}" --database=mysql --skip-ssl --skip-ssl-verify-server-cert  <<<'SELECT 1' &> /dev/null; then
                        break
                fi
                sleep 1

Appears to fail even without certificates specified.

Here is what I have done:

$ podman run --name cnt-mariadb --rm -v ~/data/container/mariadb:/etc/mysql/conf.d:z -v m104:/var/lib/mysql --env MARIADB_ROOT_PASSWORD=bob --env MARIADB_AUTO_UPGRADE=1 docker.io/mariadb:10.11 my_print_defaults --mariadbd
--socket=/run/mysqld/mysqld.sock
--host-cache-size=0
--skip-name-resolve
--pid-file=/run/mysqld/mysqld.pid
--basedir=/usr
--expire_logs_days=10
--character-set-server=utf8mb4
--collation-server=utf8mb4_general_ci
--ssl_cert=/etc/mysql/conf.d/server-cert.pem
--ssl_key=/etc/mysql/conf.d/server-key.pem
--ssl_ca=/etc/mysql/conf.d/ca.pem

while using the following files in my config dir: 99-tls.cnf <- config ca.pem <- intermediate CA cert rootca.pem <- root CA cert server-cert.pem <- server cert server-key.pem <- server key

99-tls.cnf

[mariadb]
ssl_cert = /etc/mysql/conf.d/server-cert.pem
ssl_key = /etc/mysql/conf.d/server-key.pem
ssl_ca = /etc/mysql/conf.d/ca.pem

[client-mariadb]
ssl_ca = /etc/mysql/conf.d/rootca.pem
ssl-verify-server-cert

... and as I didn't mentioned this before, my certs have the x509v3

grooverdan commented 3 weeks ago

Thanks.

For testing quay.io/danielgblack/mariadb-test:11.4 - based on #594

Seems to fail for < 11.4. 11.4+ tests are failing on timeout. I'm thinking the rest of the functionality is ok.

egabosh commented 3 weeks ago

Thanks, but same error again with quay.io/danielgblack/mariadb-test:

$ docker run --user mysql --rm -v ./:/etc/mysql/conf.d:z --env MARIADB_ROOT_PASSWORD=bob -v ./dbdata:/var/lib/mysql --env MARIADB_AUTO_UPGRADE=1 quay.io/danielgblack/mariadb-test:11.4

Tried to use lets-encrypt-r3.pem and isrgrootx1.pem from https://letsencrypt.org/certificates/. Didn't help.

[mariadbd]
ssl=1
ssl_cert=/etc/mysql/conf.d/cert.pem
ssl_key=/etc/mysql/conf.d/key.pem
ssl_ca=/etc/mysql/conf.d/lets-encrypt-r3.pem

[client-mariadb]
ssl_ca = /etc/mysql/conf.d/isrgrootx1.pem
ssl-verify-server-cert

But I think/hope that the LetsEncrypt Chain should be verified from the default Linux Truststore (/etc/ssl/certs or where ever), right?

egabosh commented 3 weeks ago

Hmm... my cert is created for CN: mariadb.mydomain.example.com Connections from "outside" are working using this domain. Maybe the problem is, that internally with mariadb-upgrade "localhost" or something else is used?

grooverdan commented 3 weeks ago

I also made a mistake, suspended laptop while it was still uploading the image quay.io/danielgblack/mariadb-test:11.4 :facepalm:

grooverdan commented 3 weeks ago

I have confirmed:

grooverdan commented 3 weeks ago

Looks like skipping tls on sockets was intended, just not implemented.

Now skipping TLS, client and server on temp_server starts. TLS authentication also disable in 11.4/11.5 healthcheck.sh.

for testing:

quay.io/danielgblack/mariadb-test:11.4 (tested it finished pushing this time).

will submit Docker Official Images after archive.mariadb.org recovers from some rate limiting event.

grooverdan commented 3 weeks ago

included in https://github.com/docker-library/official-images/pull/16944

awaiting fix https://github.com/docker-library/official-images/pull/16945 too.

egabosh commented 3 weeks ago

Thanks for the new version! Unfortunately the same error: output.txt

grooverdan commented 3 weeks ago

repull the image quay.io/danielgblack/mariadb-test:11.4 image - logs don't contain these changes: https://github.com/MariaDB/mariadb-docker/pull/594/files#diff-797c4fdb036a8060304c41d292293103907b58ec36f49b63940fd2c8e6019a78

Docker Official Image release hasn't happened yet (not until US daytime)

egabosh commented 3 weeks ago

Thank you, after another repull the error is gone! :-)

egabosh commented 3 weeks ago

@crysknife: Can you confirm the fix too? I think then this issue can be closed.

grooverdan commented 3 weeks ago

note also includes a healthcheck.sh fix if the ssl-ca isn't specified. But yes, was going to close this when the Docker Official Image is released.

crysknife commented 3 weeks ago

@crysknife: Can you confirm the fix too? I think then this issue can be closed.

I can confirm that the MariaDB upgrade works now if I don'T have any client-mariadb options set. However, when I try to use the cli tools the issue remains despite I changed the ca.pem file for the ssl-ca to include the intermediate AND root. Have to check to add the root to the trust store though.

grooverdan commented 3 weeks ago

Have to check to add the root to the trust store though.

So for cli tools it would be:

[client]
loose-ssl_ca = /etc/mysql/conf.d/ca.pem

Which I think defines the trust for for it. Probably the root only is sufficient here.

The verification disable can be:

[client]
loose-skip-ssl-verify-server-cert

Anyway, new Docker Official Image is in the build pipeline now and should be out in a few hours.