search

openstad / openstad-kubernetes

The Kubenernetes YAML files and the Helm chart for automated full stack deployment
https://www.open-stad.amsterdam/projecten/#
MIT License
4 stars 7 forks source link

Should the charts declare a MySQL root password? #17

Open avbentem opened 4 years ago

avbentem commented 4 years ago

Not a common use case, but after some deleting and re-installing, I ran into MySQL not starting. Enabling debug logging showed that the initialization phase failed:

[Note] Access denied for user 'root'@'localhost' (using password: YES)
mysql_upgrade: Got error: 1045: Access denied for user 'root'@'localhost' (using password: YES) while connecting to the MySQL server
Upgrade process encountered error and will not continue.

I think this is caused by Helm auto-generating passwords when not specified, yielding a different password when installing again, which does not match what is already in the persistent storage?

See helm/charts#5167, Random passwords change on upgrade. Create and document charts solution. And according to Bitnami's documentation, though maybe for a different reason:

It's necessary to set the root.password parameter when upgrading for readiness/liveness probes to work properly. When you install this chart for the first time, some notes will be displayed providing the credentials you must use under the 'Administrator credentials' section. Please note down the password and run the command below to upgrade your chart:

$ helm upgrade my-release bitnami/mysql --set root.password=[ROOT_PASSWORD]

When after helm delete openstad -n openstad I also wipe the PersistentVolumeClaims, then after a new installation the master starts successfully. It then shows:

[Warning] root@localhost is created with an empty password ! Please consider switching off the --initialize-insecure 
...
create user 'root'@'%' identified by 'Hb9mRwPmSG';
...
create  user 'openstad'@'%' identified by 'examplePass2146';
...
create user 'replicator'@'%' identified with 'mysql_native_password' by 'THETFkl8mC';

Repeating this indeed shows different passwords for root and replicator.

However, the slave keeps showing me the access denied error. But in this case initialization actually seems to complete fine, while the access denied is reported along with readiness probe failed. (As mentioned in the quote from the Bitnami documentation. Still investigating.)

Tested on latest MicroK8s, K8s 1.18.3, Helm 3.2.3. Just in case it helps, many more details below.

Details ```text $ kubectl get pods -n openstad NAME READY STATUS RESTARTS AGE ... openstad-mysql-master-0 0/1 Error 1 29s openstad-mysql-slave-0 0/1 Error 1 29s ``` No error messages in `kubectl describe`, but it does show an exit code: ```text $ kubectl describe pod openstad-mysql-master-0 -n openstad ... Last State: Terminated Reason: Error Exit Code: 11 ... ``` No errors at all in the logs: ```text $ kubectl logs openstad-mysql-master-0 -n openstad Welcome to the Bitnami mysql container Subscribe to project updates by watching https://github.com/bitnami/bitnami-docker-mysql Submit issues and feature requests at https://github.com/bitnami/bitnami-docker-mysql/issues INFO ==> ** Starting MySQL setup ** INFO ==> Validating settings in MYSQL_*/MARIADB_* env vars INFO ==> Initializing mysql database DEBUG ==> Ensuring expected directories/files exist WARN ==> The mysql configuration file '/opt/bitnami/mysql/conf/my.cnf' is not writable. Configurations based on environment variables will not be applied for this file. INFO ==> Using persisted data INFO ==> Running mysql_upgrade INFO ==> Starting mysql in background INFO ==> Stopping mysql ``` Added `debug` to `values.yaml`: ```yaml ## Settings for MySQL mysql: image: # Docker Image tag: 5.7.30 # Docker Tag: 5.7.30 for compatibility debug: true ``` Now, re-installing showed many more details, including `Access denied for user 'root'@'localhost' (using password: YES)`. Timestamps removed for readability: ```text $ kubectl logs openstad-mysql-master-0 -n openstad Welcome to the Bitnami mysql container Subscribe to project updates by watching https://github.com/bitnami/bitnami-docker-mysql Submit issues and feature requests at https://github.com/bitnami/bitnami-docker-mysql/issues INFO ==> ** Starting MySQL setup ** INFO ==> Validating settings in MYSQL_*/MARIADB_* env vars INFO ==> Initializing mysql database DEBUG ==> Ensuring expected directories/files exist WARN ==> The mysql configuration file '/opt/bitnami/mysql/conf/my.cnf' is not writable. Configurations based on environment be applied for this file. INFO ==> Using persisted data INFO ==> Running mysql_upgrade INFO ==> Starting mysql in background [Note] --secure-file-priv is set to NULL. Operations related to importing and exporting data are disabled [Note] /opt/bitnami/mysql/bin/mysqld (mysqld 5.7.30-log) starting as process 45 ... [Note] InnoDB: PUNCH HOLE support available [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins [Note] InnoDB: Uses event mutexes [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier [Note] InnoDB: Compressed tables use zlib 1.2.11 [Note] InnoDB: Using Linux native AIO [Note] InnoDB: Number of pools: 1 [Note] InnoDB: Using CPU crc32 instructions [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M [Note] InnoDB: Completed initialization of buffer pool [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). [Note] InnoDB: Highest supported file format is Barracuda. [Note] InnoDB: Creating shared tablespace for temporary tables [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ... [Note] InnoDB: File './ibtmp1' size is now 12 MB. [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active. [Note] InnoDB: 32 non-redo rollback segment(s) are active. [Note] InnoDB: Waiting for purge to start [Note] InnoDB: 5.7.30 started; log sequence number 2730039 [Note] InnoDB: Loading buffer pool(s) from /bitnami/mysql/data/ib_buffer_pool [Note] Plugin 'FEDERATED' is disabled. [Note] InnoDB: Buffer pool(s) load completed at 200614 19:59:43 [Note] Found ca.pem, server-cert.pem and server-key.pem in data directory. Trying to enable SSL support using them. [Note] Skipping generation of SSL certificates as certificate files are present in data directory. [Warning] CA certificate ca.pem is self signed. [Note] Skipping generation of RSA key pair as key files are present in data directory. [Note] Server hostname (bind-address): '127.0.0.1'; port: 3306 [Note] - '127.0.0.1' resolves to '127.0.0.1'; [Note] Server socket created on IP: '127.0.0.1'. [Note] Event Scheduler: Loaded 0 events [Note] /opt/bitnami/mysql/bin/mysqld: ready for connections. Version: '5.7.30-log' socket: '/opt/bitnami/mysql/tmp/mysql.sock' port: 3306 MySQL Community Server (GPL) mysql_upgrade: [Warning] Using a password on the command line interface can be insecure. [Note] Access denied for user 'root'@'localhost' (using password: YES) mysql_upgrade: Got error: 1045: Access denied for user 'root'@'localhost' (using password: YES) while connecting to the MySQL server Upgrade process encountered error and will not continue. INFO ==> Stopping mysql [Note] Giving 0 client threads a chance to die gracefully [Note] Shutting down slave threads [Note] Forcefully disconnecting 0 remaining clients [Note] Event Scheduler: Purging the queue. 0 events [Note] Binlog end [Note] Shutting down plugin 'ngram' [Note] Shutting down plugin 'partition' [Note] Shutting down plugin 'BLACKHOLE' [Note] Shutting down plugin 'ARCHIVE' [Note] Shutting down plugin 'PERFORMANCE_SCHEMA' [Note] Shutting down plugin 'MRG_MYISAM' [Note] Shutting down plugin 'MyISAM' [Note] Shutting down plugin 'INNODB_SYS_VIRTUAL' [Note] Shutting down plugin 'INNODB_SYS_DATAFILES' [Note] Shutting down plugin 'INNODB_SYS_TABLESPACES' [Note] Shutting down plugin 'INNODB_SYS_FOREIGN_COLS' [Note] Shutting down plugin 'INNODB_SYS_FOREIGN' [Note] Shutting down plugin 'INNODB_SYS_FIELDS' [Note] Shutting down plugin 'INNODB_SYS_COLUMNS' [Note] Shutting down plugin 'INNODB_SYS_INDEXES' [Note] Shutting down plugin 'INNODB_SYS_TABLESTATS' [Note] Shutting down plugin 'INNODB_SYS_TABLES' [Note] Shutting down plugin 'INNODB_FT_INDEX_TABLE' [Note] Shutting down plugin 'INNODB_FT_INDEX_CACHE' [Note] Shutting down plugin 'INNODB_FT_CONFIG' [Note] Shutting down plugin 'INNODB_FT_BEING_DELETED' [Note] Shutting down plugin 'INNODB_FT_DELETED' [Note] Shutting down plugin 'INNODB_FT_DEFAULT_STOPWORD' [Note] Shutting down plugin 'INNODB_METRICS' [Note] Shutting down plugin 'INNODB_TEMP_TABLE_INFO' [Note] Shutting down plugin 'INNODB_BUFFER_POOL_STATS' [Note] Shutting down plugin 'INNODB_BUFFER_PAGE_LRU' [Note] Shutting down plugin 'INNODB_BUFFER_PAGE' [Note] Shutting down plugin 'INNODB_CMP_PER_INDEX_RESET' [Note] Shutting down plugin 'INNODB_CMP_PER_INDEX' [Note] Shutting down plugin 'INNODB_CMPMEM_RESET' [Note] Shutting down plugin 'INNODB_CMPMEM' [Note] Shutting down plugin 'INNODB_CMP_RESET' [Note] Shutting down plugin 'INNODB_CMP' [Note] Shutting down plugin 'INNODB_LOCK_WAITS' [Note] Shutting down plugin 'INNODB_LOCKS' [Note] Shutting down plugin 'INNODB_TRX' [Note] Shutting down plugin 'InnoDB' [Note] InnoDB: FTS optimize thread exiting. [Note] InnoDB: Starting shutdown... [Note] InnoDB: Dumping buffer pool(s) to /bitnami/mysql/data/ib_buffer_pool [Note] InnoDB: Buffer pool(s) dump completed at 200614 19:59:44 [Note] InnoDB: Shutdown completed; log sequence number 2730058 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1" [Note] Shutting down plugin 'MEMORY' [Note] Shutting down plugin 'CSV' [Note] Shutting down plugin 'sha256_password' [Note] Shutting down plugin 'mysql_native_password' [Note] Shutting down plugin 'binlog' [Note] /opt/bitnami/mysql/bin/mysqld: Shutdown complete ``` Uninstalling, deleting the PersistentVolumeClaims, and re-installing: ```text helm delete openstad -n openstad kubectl delete pvc data-openstad-mysql-master-0 -n openstad kubectl delete pvc data-openstad-mysql-slave-0 -n openstad helm install --values custom-values.yaml openstad . -n openstad ``` After this, the master starts successfully. But the slave is still failing, with a similar error, but only after the initialization completes successfully? ```text $ kubectl logs openstad-mysql-slave-0 -n openstad Welcome to the Bitnami mysql container Subscribe to project updates by watching https://github.com/bitnami/bitnami-docker-mysql Submit issues and feature requests at https://github.com/bitnami/bitnami-docker-mysql/issues INFO ==> ** Starting MySQL setup ** INFO ==> Validating settings in MYSQL_*/MARIADB_* env vars INFO ==> Initializing mysql database WARN ==> The mysql configuration file '/opt/bitnami/mysql/conf/my.cnf' is not writable. Configurations based on environment variables will not be applied for this file. INFO ==> Installing database INFO ==> Starting mysql in background INFO ==> Configuring authentication INFO ==> Configuring replication in slave node INFO ==> Running mysql_upgrade INFO ==> Stopping mysql INFO ==> ** MySQL setup finished! ** INFO ==> ** Starting MySQL ** [Note] --secure-file-priv is set to NULL. Operations related to importing and exporting data are disabled [Note] /opt/bitnami/mysql/bin/mysqld (mysqld 5.7.30-log) starting as process 1 ... [Note] InnoDB: PUNCH HOLE support available [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins [Note] InnoDB: Uses event mutexes [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier [Note] InnoDB: Compressed tables use zlib 1.2.11 [Note] InnoDB: Using Linux native AIO [Note] InnoDB: Number of pools: 1 [Note] InnoDB: Using CPU crc32 instructions [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M [Note] InnoDB: Completed initialization of buffer pool [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). [Note] InnoDB: Highest supported file format is Barracuda. [Note] InnoDB: Creating shared tablespace for temporary tables [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ... [Note] InnoDB: File './ibtmp1' size is now 12 MB. [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active. [Note] InnoDB: 32 non-redo rollback segment(s) are active. [Note] InnoDB: Waiting for purge to start [Note] InnoDB: 5.7.30 started; log sequence number 2738366 [Note] Plugin 'FEDERATED' is disabled. [Note] InnoDB: Loading buffer pool(s) from /bitnami/mysql/data/ib_buffer_pool [Note] InnoDB: Buffer pool(s) load completed at 200614 22:25:03 [Note] Found ca.pem, server-cert.pem and server-key.pem in data directory. Trying to enable SSL support using them. [Note] Skipping generation of SSL certificates as certificate files are present in data directory. [Warning] CA certificate ca.pem is self signed. [Note] Skipping generation of RSA key pair as key files are present in data directory. [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306 [Note] - '0.0.0.0' resolves to '0.0.0.0'; [Note] Server socket created on IP: '0.0.0.0'. [Warning] Storing MySQL user name or password information in the master info repository is not secure and is therefore not recommended. Please consider using the USER and PASSWORD connection options for START SLAVE; see the 'START SLAVE Syntax' in the MySQL Manual for more information. [Note] Slave I/O thread for channel '': connected to master 'replicator@openstad-mysql:3306',replication started in [Note] Event Scheduler: Loaded 0 events [Note] /opt/bitnami/mysql/bin/mysqld: ready for connections. '/opt/bitnami/mysql/tmp/mysql.sock' port: 3306 MySQL Community Server (GPL) [Note] Access denied for user 'root'@'localhost' (using password: YES) [Note] Slave SQL thread for channel '' initialized, starting replication in log 'FIRST' at position 0, relay log './mysql-relay-bin.000001' position: 4 ``` And in `kubectl describe`: ```text $ kubectl describe pod openstad-mysql-slave-0 -n openstad ... Liveness: exec [sh -c password_aux="${MYSQL_MASTER_ROOT_PASSWORD:-}" if [ -f "${MYSQL_MASTER_ROOT_PASSWORD_FILE:-}" ]; then password_aux=$(cat $MYSQL_MASTER_ROOT_PASSWORD_FILE) fi mysqladmin status -uroot -p$password_aux ] delay=120s timeout=1s period=10s #success=1 #failure=3 Readiness: exec [sh -c password_aux="${MYSQL_MASTER_ROOT_PASSWORD:-}" if [ -f "${MYSQL_MASTER_ROOT_PASSWORD_FILE:-}" ]; then password_aux=$(cat $MYSQL_MASTER_ROOT_PASSWORD_FILE) fi mysqladmin status -uroot -p$password_aux ] delay=30s timeout=1s period=10s #success=1 #failure=3 ... Events: ... Readiness probe failed: mysqladmin: [Warning] Using a password on the command line interface can be insecure. mysqladmin: connect to server at 'localhost' failed error: 'Access denied for user 'root'@'localhost' (using password: YES)' ``` ```text kubectl get pods -n openstad NAME READY STATUS RESTARTS AGE openstad-adminer-684f797998-ddqvb 1/1 Running 0 37m openstad-api-64df844d65-c77dd 1/1 Running 0 37m openstad-auth-754b4d44-hlmh4 1/1 Running 0 37m openstad-frontend-89f4478dd-mxqp7 1/1 Running 0 37m openstad-image-c8b6d878d-7wg6s 1/1 Running 0 37m openstad-mongodb-7f578f944-7kp9r 1/1 Running 0 37m openstad-mysql-master-0 1/1 Running 0 37m openstad-mysql-slave-0 0/1 CrashLoopBackOff 11 37m ```
Debug logging for successful startup of master ```text Welcome to the Bitnami mysql container Subscribe to project updates by watching https://github.com/bitnami/bitnami-docker-mysql Submit issues and feature requests at https://github.com/bitnami/bitnami-docker-mysql/issues INFO ==> ** Starting MySQL setup ** INFO ==> Validating settings in MYSQL_*/MARIADB_* env vars INFO ==> Initializing mysql database DEBUG ==> Ensuring expected directories/files exist WARN ==> The mysql configuration file '/opt/bitnami/mysql/conf/my.cnf' is not writable. Configurations based on environment variables will not be applied for this file. DEBUG ==> Cleaning data directory to ensure successfully initialization INFO ==> Installing database [Warning] InnoDB: New log files created, LSN=45790 [Warning] InnoDB: Creating foreign key constraint system tables. [Warning] No existing UUID has been found, so we assume that this is the first time that this server has been : 5a363d11-ae81-11ea-b692-b6d9b58ff9e4. [Warning] Gtid table is not ready to be used. Table 'mysql.gtid_executed' cannot be opened. [Warning] CA certificate ca.pem is self signed. [Warning] root@localhost is created with an empty password ! Please consider switching off the --initialize-insecure INFO ==> Starting mysql in background [Note] --secure-file-priv is set to NULL. Operations related to importing and exporting data are disabled [Note] /opt/bitnami/mysql/bin/mysqld (mysqld 5.7.30-log) starting as process 69 ... [Note] InnoDB: PUNCH HOLE support available [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins [Note] InnoDB: Uses event mutexes [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier [Note] InnoDB: Compressed tables use zlib 1.2.11 [Note] InnoDB: Using Linux native AIO [Note] InnoDB: Number of pools: 1 [Note] InnoDB: Using CPU crc32 instructions [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M [Note] InnoDB: Completed initialization of buffer pool [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). [Note] InnoDB: Highest supported file format is Barracuda. [Note] InnoDB: Creating shared tablespace for temporary tables [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ... [Note] InnoDB: File './ibtmp1' size is now 12 MB. [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active. [Note] InnoDB: 32 non-redo rollback segment(s) are active. [Note] InnoDB: Waiting for purge to start [Note] InnoDB: 5.7.30 started; log sequence number 2630882 [Note] Plugin 'FEDERATED' is disabled. [Note] InnoDB: Loading buffer pool(s) from /bitnami/mysql/data/ib_buffer_pool [Note] InnoDB: Buffer pool(s) load completed at 200614 20:55:34 [Note] Found ca.pem, server-cert.pem and server-key.pem in data directory. Trying to enable SSL support using them. [Note] Skipping generation of SSL certificates as certificate files are present in data directory. [Warning] CA certificate ca.pem is self signed. [Note] Skipping generation of RSA key pair as key files are present in data directory. [Note] Server hostname (bind-address): '127.0.0.1'; port: 3306 [Note] - '127.0.0.1' resolves to '127.0.0.1'; [Note] Server socket created on IP: '127.0.0.1'. [Note] Event Scheduler: Loaded 0 events DEBUG ==> Executing SQL command: [Note] /opt/bitnami/mysql/bin/mysqld: ready for connections. Version: '5.7.30-log' socket: '/opt/bitnami/mysql/tmp/mysql.sock' port: 3306 MySQL Community Server (GPL) INFO ==> Configuring authentication DEBUG ==> Executing SQL command: DELETE FROM mysql.user WHERE user not in ('mysql.sys','mariadb.sys'); DEBUG ==> Starting replication DEBUG ==> Configuring root user credentials DEBUG ==> Executing SQL command: -- create admin user create user 'root'@'%' identified by 'Hb9mRwPmSG'; grant all on *.* to 'root'@'%' with grant option; flush privileges; DEBUG ==> removing the unknown user DEBUG ==> Executing SQL command: select Host from user where User=''; mysql: [Warning] Using a password on the command line interface can be insecure. DEBUG ==> creating database user \'openstad\' DEBUG ==> Executing SQL command: create user 'openstad'@'%' identified by 'examplePass2146'; mysql: [Warning] Using a password on the command line interface can be insecure. DEBUG ==> Removing all other hosts for the user DEBUG ==> Executing SQL command: select Host from user where User='openstad' and Host!='%'; mysql: [Warning] Using a password on the command line interface can be insecure. DEBUG ==> Creating database image DEBUG ==> Executing SQL command: create database if not exists `image` ; mysql: [Warning] Using a password on the command line interface can be insecure. DEBUG ==> Providing privileges to username openstad on database image DEBUG ==> Executing SQL command: grant all on `image`.* to 'openstad'@'%'; mysql: [Warning] Using a password on the command line interface can be insecure. INFO ==> Configuring replication in master node DEBUG ==> Configure replication user credentials DEBUG ==> Executing SQL command: create user 'replicator'@'%' identified with 'mysql_native_password' by 'THETFkl8mC'; mysql: [Warning] Using a password on the command line interface can be insecure. DEBUG ==> Executing SQL command: grant REPLICATION SLAVE on *.* to 'replicator'@'%' with grant option; flush privileges; mysql: [Warning] Using a password on the command line interface can be insecure. INFO ==> Running mysql_upgrade mysql_upgrade: [Warning] Using a password on the command line interface can be insecure. Checking server version. Running queries to upgrade MySQL server. Checking system database. mysql.columns_priv OK mysql.db OK mysql.engine_cost OK mysql.event OK mysql.func OK mysql.general_log OK mysql.gtid_executed OK mysql.help_category OK mysql.help_keyword OK mysql.help_relation OK mysql.help_topic OK mysql.innodb_index_stats OK mysql.innodb_table_stats OK mysql.ndb_binlog_index OK mysql.plugin OK mysql.proc OK mysql.procs_priv OK mysql.proxies_priv OK mysql.server_cost OK mysql.servers OK mysql.slave_master_info OK mysql.slave_relay_log_info OK mysql.slave_worker_info OK mysql.slow_log OK mysql.tables_priv OK mysql.time_zone OK mysql.time_zone_leap_second OK mysql.time_zone_name OK mysql.time_zone_transition OK mysql.time_zone_transition_type OK mysql.user OK The sys schema is already up to date (version 1.5.2). Checking databases. sys.sys_config OK Upgrade process completed successfully. Checking if update is needed. INFO ==> Loading user's custom files from /docker-entrypoint-initdb.d DEBUG ==> Executing /docker-entrypoint-initdb.d/extra_dbs.sql DEBUG ==> Executing SQL command: select 1 mysql: [Warning] Using a password on the command line interface can be insecure. 1 DEBUG ==> Executing SQL command: CREATE DATABASE IF NOT EXISTS api DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL PRIVILEGES ON api.* TO 'openstad'@'%'; CREATE DATABASE IF NOT EXISTS auth DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL PRIVILEGES ON auth.* TO 'openstad'@'%'; mysql: [Warning] Using a password on the command line interface can be insecure. INFO ==> Stopping mysql [Note] Giving 0 client threads a chance to die gracefully [Note] Shutting down slave threads [Note] Forcefully disconnecting 0 remaining clients [Note] Event Scheduler: Purging the queue. 0 events [Note] Binlog end [Note] Shutting down plugin 'ngram' [Note] Shutting down plugin 'partition' [Note] Shutting down plugin 'BLACKHOLE' [Note] Shutting down plugin 'ARCHIVE' [Note] Shutting down plugin 'PERFORMANCE_SCHEMA' [Note] Shutting down plugin 'MRG_MYISAM' [Note] Shutting down plugin 'MyISAM' [Note] Shutting down plugin 'INNODB_SYS_VIRTUAL' [Note] Shutting down plugin 'INNODB_SYS_DATAFILES' [Note] Shutting down plugin 'INNODB_SYS_TABLESPACES' [Note] Shutting down plugin 'INNODB_SYS_FOREIGN_COLS' [Note] Shutting down plugin 'INNODB_SYS_FOREIGN' [Note] Shutting down plugin 'INNODB_SYS_FIELDS' [Note] Shutting down plugin 'INNODB_SYS_COLUMNS' [Note] Shutting down plugin 'INNODB_SYS_INDEXES' [Note] Shutting down plugin 'INNODB_SYS_TABLESTATS' [Note] Shutting down plugin 'INNODB_SYS_TABLES' [Note] Shutting down plugin 'INNODB_FT_INDEX_TABLE' [Note] Shutting down plugin 'INNODB_FT_INDEX_CACHE' [Note] Shutting down plugin 'INNODB_FT_CONFIG' [Note] Shutting down plugin 'INNODB_FT_BEING_DELETED' [Note] Shutting down plugin 'INNODB_FT_DELETED' [Note] Shutting down plugin 'INNODB_FT_DEFAULT_STOPWORD' [Note] Shutting down plugin 'INNODB_METRICS' [Note] Shutting down plugin 'INNODB_TEMP_TABLE_INFO' [Note] Shutting down plugin 'INNODB_BUFFER_POOL_STATS' [Note] Shutting down plugin 'INNODB_BUFFER_PAGE_LRU' [Note] Shutting down plugin 'INNODB_BUFFER_PAGE' [Note] Shutting down plugin 'INNODB_CMP_PER_INDEX_RESET' [Note] Shutting down plugin 'INNODB_CMP_PER_INDEX' [Note] Shutting down plugin 'INNODB_CMPMEM_RESET' [Note] Shutting down plugin 'INNODB_CMPMEM' [Note] Shutting down plugin 'INNODB_CMP_RESET' [Note] Shutting down plugin 'INNODB_CMP' [Note] Shutting down plugin 'INNODB_LOCK_WAITS' [Note] Shutting down plugin 'INNODB_LOCKS' [Note] Shutting down plugin 'INNODB_TRX' [Note] Shutting down plugin 'InnoDB' [Note] InnoDB: FTS optimize thread exiting. [Note] InnoDB: Starting shutdown... [Note] InnoDB: Dumping buffer pool(s) to /bitnami/mysql/data/ib_buffer_pool [Note] InnoDB: Buffer pool(s) dump completed at 200614 20:55:48 [Note] InnoDB: Shutdown completed; log sequence number 2729852 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1" [Note] Shutting down plugin 'MEMORY' [Note] Shutting down plugin 'CSV' [Note] Shutting down plugin 'sha256_password' [Note] Shutting down plugin 'mysql_native_password' [Note] Shutting down plugin 'binlog' [Note] /opt/bitnami/mysql/bin/mysqld: Shutdown complete INFO ==> ** MySQL setup finished! ** INFO ==> ** Starting MySQL ** [Note] --secure-file-priv is set to NULL. Operations related to importing and exporting data are disabled [Note] /opt/bitnami/mysql/bin/mysqld (mysqld 5.7.30-log) starting as process 1 ... [Note] InnoDB: PUNCH HOLE support available [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins [Note] InnoDB: Uses event mutexes [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier [Note] InnoDB: Compressed tables use zlib 1.2.11 [Note] InnoDB: Using Linux native AIO [Note] InnoDB: Number of pools: 1 [Note] InnoDB: Using CPU crc32 instructions [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M [Note] InnoDB: Completed initialization of buffer pool [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). [Note] InnoDB: Highest supported file format is Barracuda. [Note] InnoDB: Creating shared tablespace for temporary tables [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ... [Note] InnoDB: File './ibtmp1' size is now 12 MB. [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active. [Note] InnoDB: 32 non-redo rollback segment(s) are active. [Note] InnoDB: Waiting for purge to start [Note] InnoDB: 5.7.30 started; log sequence number 2729852 [Note] Plugin 'FEDERATED' is disabled. [Note] InnoDB: Loading buffer pool(s) from /bitnami/mysql/data/ib_buffer_pool [Note] InnoDB: Buffer pool(s) load completed at 200614 20:55:52 [Note] Found ca.pem, server-cert.pem and server-key.pem in data directory. Trying to enable SSL support using them. [Note] Skipping generation of SSL certificates as certificate files are present in data directory. [Warning] CA certificate ca.pem is self signed. [Note] Skipping generation of RSA key pair as key files are present in data directory. [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306 [Note] - '0.0.0.0' resolves to '0.0.0.0'; [Note] Server socket created on IP: '0.0.0.0'. [Note] Failed to start slave threads for channel '' [Note] Event Scheduler: Loaded 0 events [Note] /opt/bitnami/mysql/bin/mysqld: ready for connections. Version: '5.7.30-log' socket: '/opt/bitnami/mysql/tmp/mysql.sock' port: 3306 MySQL Community Server (GPL) ```
jaapgorjup commented 4 years ago

Thanks for reporting this. We had some issues with the re-install of the Helm chart of which some we can fix in pre-hooks. But for others we will adjust the installation documentation and add some guidance on this. Will keep this issue open for now in case we encounter this again.

ghost commented 4 years ago

We encountered this exact problem recently when we were trying to deploy a customized instance. We ended up revamping the secrets a bit and implemented pre-install hooks in addition to the resource policy of keep. See 0134856f36c3f82e22673983ee67e7c099703012.

Currently, I have also used this method for secrets in other projects and haven't reached this problem anymore. Can you confirm whether you still have this issue in the latest version?

Edit: confirmed to work on our Hetzner Cloud setup