search

lando / pantheon

The Official Lando Pantheon plugin.
https://docs.lando.dev/pantheon
GNU General Public License v3.0
11 stars 18 forks source link

The Service "Database" Failed Its Healthcheck #14

Closed digisavvy closed 2 years ago

digisavvy commented 3 years ago

Tell us about your setup MacOS, Big Sur 11.4

What is your lando version and operating system? (note that the older your version the less likely we are to reply) v3.3.0

Tell us about your .lando.yml

name: members
recipe: pantheon
config:
  framework: wordpress
  site: members
  id: c72335e1-49d7-49b6-b73c-28a65e6c3675

Tell us about the command you were running

I restarted my computer, upon restarting I noticed that the database service was waiting for several minutes.

I run lando rebuild and still had the same issue.

I noticed a newer version of Lando was available so I installed.

I ran lando poweroff and lando start the issue persists

Eventually this error displays ERROR ==> database reported as unhealthy.

Tell us more Output from lando rebuild -s database

database_1         | userperms 16:06:07.28 WARN  ==> Only the root user can reset permissions! This is probably ok though...
database_1         | loadkeys 16:06:07.29 WARN  ==> Only the root user can load ssh keys! This is probably ok though...
database_1         | lando 16:06:07.30 INFO  ==> Lando handing off to: /launch.sh
database_1         | lando 16:06:07.30 DEBUG ==> Running command with exec...
database_1         | mariadb 16:06:07.34 
database_1         | mariadb 16:06:07.34 Welcome to the Bitnami mariadb container
database_1         | mariadb 16:06:07.34 Subscribe to project updates by watching https://github.com/bitnami/bitnami-docker-mariadb
database_1         | mariadb 16:06:07.34 Submit issues and feature requests at https://github.com/bitnami/bitnami-docker-mariadb/issues
database_1         | mariadb 16:06:07.34 
database_1         | mariadb 16:06:07.35 INFO  ==> ** Starting MariaDB setup **
database_1         | mariadb 16:06:07.37 INFO  ==> Validating settings in MYSQL_*/MARIADB_* env vars
database_1         | mariadb 16:06:07.37 WARN  ==> You set the environment variable ALLOW_EMPTY_PASSWORD=yes. For safety reasons, do not use this flag in a production environment.
database_1         | mariadb 16:06:07.38 INFO  ==> Initializing mariadb database
database_1         | mariadb 16:06:07.39 DEBUG ==> Ensuring expected directories/files exist
database_1         | mariadb 16:06:07.40 INFO  ==> Updating 'my.cnf' with custom configuration
database_1         | mariadb 16:06:07.41 DEBUG ==> Injecting custom configuration from my_custom.conf
database_1         | mariadb 16:06:07.42 INFO  ==> Setting user option
database_1         | mariadb 16:06:07.43 DEBUG ==> Setting user to 'pantheon' in mariadb configuration file /opt/bitnami/mariadb/conf/my.cnf
database_1         | mariadb 16:06:07.46 INFO  ==> Injecting custom configuration 'my_custom.cnf'
database_1         | mariadb 16:06:07.46 INFO  ==> Using persisted data
database_1         | mariadb 16:06:07.50 INFO  ==> Running mysql_upgrade
database_1         | mariadb 16:06:07.51 INFO  ==> Starting mariadb in background
database_1         | 2021-09-02 16:06:07 139774275657536 [Note] /opt/bitnami/mariadb/sbin/mysqld (mysqld 10.1.47-MariaDB) starting as process 76 ...
database_1         | 2021-09-02 16:06:07 139774275657536 [ERROR] Could not open mysql.plugin table. Some plugins may be not loaded
database_1         | 2021-09-02 16:06:07 139774275657536 [Note] InnoDB: Using mutexes to ref count buffer pool pages
database_1         | 2021-09-02 16:06:07 139774275657536 [Note] InnoDB: The InnoDB memory heap is disabled
database_1         | 2021-09-02 16:06:07 139774275657536 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
database_1         | 2021-09-02 16:06:07 139774275657536 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
database_1         | 2021-09-02 16:06:07 139774275657536 [Note] InnoDB: Compressed tables use zlib 1.2.11
database_1         | 2021-09-02 16:06:07 139774275657536 [Note] InnoDB: Using Linux native AIO
database_1         | 2021-09-02 16:06:07 139774275657536 [Note] InnoDB: Using SSE crc32 instructions
database_1         | 2021-09-02 16:06:07 139774275657536 [Note] InnoDB: Initializing buffer pool, size = 384.0M
database_1         | 2021-09-02 16:06:07 139774275657536 [Note] InnoDB: Completed initialization of buffer pool
database_1         | 2021-09-02 16:06:07 139774275657536 [Note] InnoDB: Highest supported file format is Barracuda.
database_1         | 2021-09-02 16:06:07 139774275657536 [Note] InnoDB: The log sequence number 0 in ibdata file do not match the log sequence number 1616753 in the ib_logfiles!
database_1         | 2021-09-02 16:06:07 139774275657536 [Note] InnoDB: Restoring possible half-written data pages from the doublewrite buffer...
database_1         | 2021-09-02 16:06:07 139774275657536 [Note] InnoDB: 128 rollback segment(s) are active.
database_1         | 2021-09-02 16:06:07 139774275657536 [Note] InnoDB: Waiting for purge to start
database_1         | 2021-09-02 16:06:07 139774275657536 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.49-89.0 started; log sequence number 1616753
database_1         | 2021-09-02 16:06:07 139774275657536 [Note] Plugin 'FEEDBACK' is disabled.
database_1         | 2021-09-02 16:06:07 139772662118144 [Note] InnoDB: Dumping buffer pool(s) not yet started
database_1         | 2021-09-02 16:06:07 139774275657536 [Note] Recovering after a crash using tc.log
database_1         | 2021-09-02 16:06:07 139774275657536 [Note] Starting crash recovery...
database_1         | 2021-09-02 16:06:07 139774275657536 [Note] Crash recovery finished.
database_1         | 2021-09-02 16:06:08 139774275657536 [ERROR] Can't open and lock privilege tables: Incorrect file format 'servers'
database_1         | 2021-09-02 16:06:08 139774275657536 [Note] Server socket created on IP: '127.0.0.1'.
database_1         | 2021-09-02 16:06:08 139774275657536 [ERROR] Fatal error: Can't open and lock privilege tables: Incorrect file format 'user'
database_1         | mariadb 16:16:08.04 ERROR ==> MySQL failed to start

Could better documentation have prevented this issue? if so what should we change? Does this bug prevent you from using lando?

reynoldsalec commented 2 years ago

@digisavvy did you manage to get around this?

reynoldsalec commented 2 years ago

@digisavvy LMK if you want to chat about this in the future...looking back at your logs, I'm curious if maybe you had upgraded a major version of MariaDB accidentally somehow (Can't open and lock privilege tables: Incorrect file format 'servers') and that was what was causing the problem.