Closed paboman closed 4 months ago
WayneManion
commented
5 months ago I don't have all the log data to back me up, but after performing the latest MariaDB update my Home Assistant setup starts up, works for five minutes, and then stops responding.
Whippie72
commented
5 months ago Similar issues on my end. After updating to 2.7.0 the Database wont start properly.
[05:20:32] INFO: Starting MariaDB 2024-04-16 5:20:32 0 [Note] Starting MariaDB 10.11.6-MariaDB source revision fecd78b83785d5ae96f2c6ff340375be803cd299 as process 27980 2024-04-16 5:20:32 0 [Note] InnoDB: Compressed tables use zlib 1.3.1 2024-04-16 5:20:32 0 [Note] InnoDB: Number of transaction pools: 1 2024-04-16 5:20:32 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions 2024-04-16 5:20:32 0 [Note] mysqld: O_TMPFILE is not supported on /var/tmp (disabling future attempts) 2024-04-16 5:20:32 0 [Note] InnoDB: Using Linux native AIO 2024-04-16 5:20:32 0 [Note] InnoDB: Initializing buffer pool, total size = 128.000MiB, chunk size = 2.000MiB 2024-04-16 5:20:32 0 [Note] InnoDB: Completed initialization of buffer pool 2024-04-16 5:20:32 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes) 2024-04-16 5:20:32 0 [ERROR] InnoDB: Upgrade after a crash is not supported. The redo log was created with MariaDB 10.6.8. You must start up and shut down MariaDB 10.7 or earlier. 2024-04-16 5:20:32 0 [ERROR] InnoDB: Plugin initialization aborted with error Generic error 2024-04-16 5:20:32 0 [Note] InnoDB: Starting shutdown... 2024-04-16 5:20:33 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed. 2024-04-16 5:20:33 0 [Note] Plugin 'FEEDBACK' is disabled. 2024-04-16 5:20:33 0 [ERROR] Unknown/unsupported storage engine: InnoDB 2024-04-16 5:20:33 0 [ERROR] Aborting [05:20:33] INFO: Service mariadb exited with code 1 (by signal 0)
agners
commented
5 months ago Hm, Alpine 3.16 (as used in add-on before the 2.7.0 upgrade) and Alpine 3.17 come with MariaDB 10.6.16, Alpine 3.18 and 3.19 come with 10.11.6. So there is no Alpine release with 10.7 :thinking:
agners
commented
5 months ago Ok, reading this again, the problem seems to be the add-on shutdown before the upgrade was not graceful in your case:
2024-04-16 5:20:32 0 [ERROR] InnoDB: Upgrade after a crash is not supported. The redo log was created with MariaDB 10.6.8. You must start up and shut down MariaDB 10.7 or earlier.Maybe just downgrading and upgrading again helps?
If not (or in any case), you can use ha host logs to get the logs of previous add-on runs. It would be interesting to see the logs of the shutdown before the first (failed) startup attempt:
ha host logs --identifier addon_core_mariadb -n 10000
paboman
commented
5 months ago Ok, reading this again, the problem seems to be the add-on shutdown before the upgrade was not graceful in your case:
2024-04-16 5:20:32 0 [ERROR] InnoDB: Upgrade after a crash is not supported. The redo log was created with MariaDB 10.6.8. You must start up and shut down MariaDB 10.7 or earlier.Maybe just downgrading and upgrading again helps?
If not (or in any case), you can use
ha host logsto get the logs of previous add-on runs. It would be interesting to see the logs of the shutdown before the first (failed) startup attempt:ha host logs --identifier addon_core_mariadb -n 10000
in the HA terminal I cant scroll up enough to see the reason
agners
commented
5 months ago in the HA terminal I cant scroll up enough to see the reason
Note, I am talking about the reason the previous shutdown failed, not the startup...
If you can see the reason of the shutdown failure, can you please share that part of the log as well? This way we can improve the add-on to correctly shutdown also in your case.
FWIW, my guess is a timeout of some sort. From what I understand we wait up to 18s for MariaDB to shutdown. But maybe it took longer in your case? :thinking:
paboman
commented
5 months ago now I rolled back to 2.6.1 should I stop the MariaDB addon first and then do the update to 2.7.0 ?
agners
commented
5 months ago Yeah a manual stop and share the logs to the end of the stop would be interesting.
agners
commented
5 months ago For me, a graceful shutdown looks like this:
s6-rc: info: service legacy-services: stopping
s6-rc: info: service legacy-services successfully stopped
s6-rc: info: service mariadb-post: stopping
[09:29:05] INFO: Deleting service information from Home Assistant
s6-rc: info: service mariadb-post successfully stopped
s6-rc: info: service mariadb-core: stopping
2024-04-16 9:29:05 0 [Note] mysqld (initiated by: unknown): Normal shutdown
2024-04-16 9:29:05 0 [Note] InnoDB: FTS optimize thread exiting.
2024-04-16 9:29:05 0 [Note] InnoDB: Starting shutdown...
2024-04-16 9:29:05 0 [Note] InnoDB: Dumping buffer pool(s) to /data/databases/ib_buffer_pool
2024-04-16 9:29:05 0 [Note] InnoDB: Restricted to 2016 pages due to innodb_buf_pool_dump_pct=25
2024-04-16 9:29:05 0 [Note] InnoDB: Buffer pool(s) dump completed at 240416 9:29:05
2024-04-16 9:29:05 0 [Note] InnoDB: Removed temporary tablespace data file: "./ibtmp1"
2024-04-16 9:29:05 0 [Note] InnoDB: Shutdown completed; log sequence number 738796972612; transaction id 141268245
2024-04-16 9:29:05 0 [Note] mysqld: Shutdown complete
[09:29:06] INFO: Service mariadb exited with code 0 (by signal 0)
s6-rc: info: service mariadb-core successfully stopped
s6-rc: info: service mariadb-pre: stopping
s6-rc: info: service mariadb-pre successfully stopped
s6-rc: info: service legacy-cont-init: stopping
s6-rc: info: service legacy-cont-init successfully stopped
s6-rc: info: service fix-attrs: stopping
s6-rc: info: service fix-attrs successfully stopped
s6-rc: info: service s6rc-oneshot-runner: stopping
s6-rc: info: service s6rc-oneshot-runner successfully stopped
Acey-yo
commented
5 months ago I have same trouble. 🙈
Whippie72
commented
5 months ago now I rolled back to 2.6.1 should I stop the MariaDB addon first and then do the update to 2.7.0 ?
I haven't stopped the addon while restoring the backup. I'm using "Home Assistant Google Drive Backup" as an addon to backup and restore my HA :)
Boia11
commented
5 months ago Same issue after update from 2.6.1 to 2.7.0
demonaxxx
commented
5 months ago Same issue after update from 2.6.1 to 2.7.0
agners
commented
5 months ago Can someone provide the logs from the shutdown before the upgrade (see https://github.com/home-assistant/addons/issues/3566#issuecomment-2058387438).
EvgenyBash
commented
5 months ago Can someone provide the logs from the shutdown before the upgrade (see #3566 (comment)).
s6-rc: info: service legacy-services: stopping s6-rc: info: service legacy-services successfully stopped s6-rc: info: service mariadb-post: stopping [14:02:18] INFO: Deleting service information from Home Assistant
s6-rc: info: service mariadb-post successfully stopped s6-rc: info: service mariadb-core: stopping 2024-04-16 14:02:18 0 [Note] mysqld (initiated by: unknown): Normal shutdown 2024-04-16 14:02:18 0 [Note] InnoDB: FTS optimize thread exiting. 2024-04-16 14:02:18 0 [Note] InnoDB: Starting shutdown... 2024-04-16 14:02:18 0 [Note] InnoDB: Dumping buffer pool(s) to /data/databases/ib_buffer_pool 2024-04-16 14:02:18 0 [Note] InnoDB: Restricted to 2028 pages due to innodb_buf_pool_dump_pct=25 2024-04-16 14:02:18 0 [Note] InnoDB: Buffer pool(s) dump completed at 240416 14:02:18 2024-04-16 14:02:20 0 [Note] InnoDB: Removed temporary tablespace data file: "./ibtmp1" 2024-04-16 14:02:20 0 [Note] InnoDB: Shutdown completed; log sequence number 35521245713; transaction id 2243519 2024-04-16 14:02:20 0 [Note] mysqld: Shutdown complete
[14:02:20] INFO: Service mariadb exited with code 0 (by signal 0) s6-rc: info: service mariadb-core successfully stopped s6-rc: info: service mariadb-pre: stopping s6-rc: info: service mariadb-pre successfully stopped s6-rc: info: service legacy-cont-init: stopping s6-rc: info: service legacy-cont-init successfully stopped s6-rc: info: service fix-attrs: stopping s6-rc: info: service fix-attrs successfully stopped s6-rc: info: service s6rc-oneshot-runner: stopping s6-rc: info: service s6rc-oneshot-runner successfully stopped
NYZack
commented
5 months ago After having the problem described here, I had success by restoring 2.6.1, stopping the add-on, doing the update to 2.7, and then starting the add-on.
paboman
commented
5 months ago After having the problem described here, I had success by restoring 2.6.1, stopping the add-on, doing the update to 2.7, and then starting the add-on.
did you tried to restart HA to check if MariaBD 2.7.0 start as it should ?
NYZack
commented
5 months ago did you tried to restart HA to check if MariaBD 2.7.0 start as it should ?
Three scenarios:
WayneManion
commented
5 months ago I am running the Nextcloud add-on alongside MariaDB. After upgrading MariaDB I can see in the logs that Nextcloud starts without issue but there is not a word in the logs about any homeassistant tables.
nextcloud.oc_mail_messages OK
nextcloud.oc_mail_messages_retention OK
nextcloud.oc_mail_messages_snoozed OK
nextcloud.oc_mail_provisionings OK
nextcloud.oc_mail_recipients OK
nextcloud.oc_mail_smime_certificates OK
nextcloud.oc_mail_tags OK
nextcloud.oc_mail_trusted_senders OK
nextcloud.oc_migrations OK
nextcloud.oc_mimetypes OK
nextcloud.oc_mounts OK
nextcloud.oc_notes_meta OK
nextcloud.oc_notifications OK
nextcloud.oc_notifications_pushhash OK
nextcloud.oc_notifications_settings OK
nextcloud.oc_oauth2_access_tokens OK
nextcloud.oc_oauth2_clients OK
nextcloud.oc_open_local_editor OK
nextcloud.oc_photos_albums OK
nextcloud.oc_photos_albums_collabs OK
nextcloud.oc_photos_albums_files OK
nextcloud.oc_preferences OK
nextcloud.oc_privacy_admins OK
nextcloud.oc_profile_config OK
nextcloud.oc_properties OK
nextcloud.oc_ratelimit_entries OK
nextcloud.oc_reactions OK
nextcloud.oc_recent_contact OK
nextcloud.oc_richdocuments_assets OK
nextcloud.oc_richdocuments_direct OK
nextcloud.oc_richdocuments_template OK
nextcloud.oc_richdocuments_wopi OK
nextcloud.oc_schedulingobjects OK
nextcloud.oc_share OK
nextcloud.oc_share_external OK
nextcloud.oc_storages OK
nextcloud.oc_storages_credentials OK
nextcloud.oc_systemtag OK
nextcloud.oc_systemtag_group OK
nextcloud.oc_systemtag_object_mapping OK
nextcloud.oc_talk_attachments OK
nextcloud.oc_talk_attendees OK
nextcloud.oc_talk_bots_conversation OK
nextcloud.oc_talk_bots_server OK
nextcloud.oc_talk_bridges OK
nextcloud.oc_talk_commands OK
nextcloud.oc_talk_consent OK
nextcloud.oc_talk_internalsignaling OK
nextcloud.oc_talk_invitations OK
nextcloud.oc_talk_poll_votes OK
nextcloud.oc_talk_polls OK
nextcloud.oc_talk_reminders OK
nextcloud.oc_talk_rooms OK
nextcloud.oc_talk_sessions OK
nextcloud.oc_text2image_tasks OK
nextcloud.oc_text_documents OK
nextcloud.oc_text_sessions OK
nextcloud.oc_text_steps OK
nextcloud.oc_textprocessing_tasks OK
nextcloud.oc_trusted_servers OK
nextcloud.oc_twofactor_backupcodes OK
nextcloud.oc_twofactor_providers OK
nextcloud.oc_user_status OK
nextcloud.oc_user_transfer_owner OK
nextcloud.oc_users OK
nextcloud.oc_vcategory OK
nextcloud.oc_vcategory_to_object OK
nextcloud.oc_webauthn OK
nextcloud.oc_whats_new OK
nginxproxymanager.access_list OK
nginxproxymanager.access_list_auth OK
nginxproxymanager.access_list_client OK
nginxproxymanager.audit_log OK
nginxproxymanager.auth OK
nginxproxymanager.certificate OK
nginxproxymanager.dead_host OK
nginxproxymanager.migrations OK
nginxproxymanager.migrations_lock OK
nginxproxymanager.proxy_host OK
nginxproxymanager.redirection_host OK
nginxproxymanager.setting OK
nginxproxymanager.stream OK
nginxproxymanager.user OK
nginxproxymanager.user_permission OK
sys.sys_config OK
[07:21:13] INFO: Ensuring internal database upgrades are performed
[07:21:13] INFO: Ensure databases exists
[07:21:13] INFO: Create database homeassistant
[07:21:13] INFO: Create database nextcloud
[07:21:13] INFO: Ensure users exists and are updated
[07:21:13] INFO: Update user homeassistant
[07:21:13] INFO: Update user service
[07:21:13] INFO: Init/Update rights
[07:21:13] INFO: Granting all privileges to homeassistant on homeassistant
[07:21:13] INFO: Granting all privileges to nextcloud on nextcloud
[07:21:13] INFO: Sending service information to Home Assistant
s6-rc: info: service mariadb-post successfully started
s6-rc: info: service legacy-services: starting
s6-rc: info: service legacy-services successfully started```
N3rdix
commented
5 months ago Can someone provide the logs from the shutdown before the upgrade (see #3566 (comment)).
Just gave it a try on my test instance and do not see an issue for far on this machine (HAOS 12.2 on a Pi4) although there are some strange errors concerning incorrect definitions of table mqsql.column_stats.
Full upgrade log including stop, install, start:
s6-rc: info: service mariadb-lock-core: starting
[14:48:42] INFO: Start MariaDB client (to lock tables for backups)
s6-rc: info: service mariadb-lock-core successfully started
s6-rc: info: service mariadb-lock-post: starting
[14:48:42] INFO: MariaDB tables locked
s6-rc: info: service mariadb-lock-post successfully started
s6-rc: info: service mariadb-lock-post: stopping
[14:48:54] INFO: MariaDB tables unlocked
s6-rc: info: service mariadb-lock-post successfully stopped
s6-rc: info: service mariadb-lock-core: stopping
[14:48:54] INFO: MariaDB client (to lock tables for backups) stopped
s6-rc: info: service mariadb-lock-core successfully stopped
s6-rc: info: service legacy-services: stopping
s6-rc: info: service legacy-services successfully stopped
s6-rc: info: service mariadb-post: stopping
[14:49:16] INFO: Deleting service information from Home Assistant
s6-rc: info: service mariadb-post successfully stopped
s6-rc: info: service mariadb-core: stopping
2024-04-16 14:49:16 0 [Note] mysqld (initiated by: unknown): Normal shutdown
2024-04-16 14:49:16 0 [Note] InnoDB: FTS optimize thread exiting.
2024-04-16 14:49:16 0 [Note] InnoDB: Starting shutdown...
2024-04-16 14:49:16 0 [Note] InnoDB: Dumping buffer pool(s) to /data/databases/ib_buffer_pool
2024-04-16 14:49:16 0 [Note] InnoDB: Restricted to 2028 pages due to innodb_buf_pool_dump_pct=25
2024-04-16 14:49:16 0 [Note] InnoDB: Buffer pool(s) dump completed at 240416 14:49:16
2024-04-16 14:49:16 0 [Note] InnoDB: Removed temporary tablespace data file: "./ibtmp1"
2024-04-16 14:49:16 0 [Note] InnoDB: Shutdown completed; log sequence number 129651300584; transaction id 27096757
2024-04-16 14:49:16 0 [Note] mysqld: Shutdown complete
[14:49:16] INFO: Service mariadb exited with code 0 (by signal 0)
s6-rc: info: service mariadb-core successfully stopped
s6-rc: info: service mariadb-pre: stopping
s6-rc: info: service mariadb-pre successfully stopped
s6-rc: info: service legacy-cont-init: stopping
s6-rc: info: service legacy-cont-init successfully stopped
s6-rc: info: service fix-attrs: stopping
s6-rc: info: service fix-attrs successfully stopped
s6-rc: info: service s6rc-oneshot-runner: stopping
s6-rc: info: service s6rc-oneshot-runner successfully stopped
s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service mariadb-pre: starting
[14:49:23] INFO: Using existing mariadb
s6-rc: info: service mariadb-pre successfully started
s6-rc: info: service mariadb-core: starting
s6-rc: info: service mariadb-core successfully started
s6-rc: info: service mariadb-post: starting
[14:49:23] INFO: Starting MariaDB
2024-04-16 14:49:23 0 [Note] Starting MariaDB 10.11.6-MariaDB source revision fecd78b83785d5ae96f2c6ff340375be803cd299 as process 67
2024-04-16 14:49:23 0 [Note] InnoDB: Compressed tables use zlib 1.3.1
2024-04-16 14:49:23 0 [Note] InnoDB: Number of transaction pools: 1
2024-04-16 14:49:23 0 [Note] InnoDB: Using ARMv8 crc32 instructions
2024-04-16 14:49:23 0 [Note] mysqld: O_TMPFILE is not supported on /var/tmp (disabling future attempts)
2024-04-16 14:49:23 0 [Note] InnoDB: Using Linux native AIO
2024-04-16 14:49:23 0 [Note] InnoDB: Initializing buffer pool, total size = 128.000MiB, chunk size = 2.000MiB
2024-04-16 14:49:23 0 [Note] InnoDB: Completed initialization of buffer pool
2024-04-16 14:49:23 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes)
2024-04-16 14:49:23 0 [Note] InnoDB: Upgrading redo log: 48.000MiB; LSN=129651300584
2024-04-16 14:49:23 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes)
2024-04-16 14:49:24 0 [Note] InnoDB: 128 rollback segments are active.
2024-04-16 14:49:24 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ...
2024-04-16 14:49:24 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB.
2024-04-16 14:49:24 0 [Note] InnoDB: log sequence number 129651300584; transaction id 27096758
2024-04-16 14:49:24 0 [Note] Plugin 'FEEDBACK' is disabled.
2024-04-16 14:49:24 0 [Note] InnoDB: Loading buffer pool(s) from /data/databases/ib_buffer_pool
2024-04-16 14:49:24 0 [Note] Server socket created on IP: '0.0.0.0'.
2024-04-16 14:49:24 0 [Note] Server socket created on IP: '::'.
2024-04-16 14:49:24 0 [Note] mysqld: ready for connections.
Version: '10.11.6-MariaDB' socket: '/run/mysqld/mysqld.sock' port: 3306 Alpine Linux
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 0 [Note] InnoDB: Buffer pool(s) load completed at 240416 14:49:24
[14:49:24] INFO: MariaDB started
2024-04-16 14:49:24 5 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 5 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 5 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 5 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 14:49:24 5 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 5 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
[14:49:24] INFO: Check data integrity and fix corruptions
mysql.column_stats OK
mysql.columns_priv OK
mysql.db OK
mysql.event OK
mysql.func OK
mysql.global_priv OK
mysql.gtid_slave_pos OK
mysql.help_category OK
mysql.help_keyword OK
mysql.help_relation OK
mysql.help_topic OK
mysql.index_stats OK
mysql.innodb_index_stats OK
mysql.innodb_table_stats OK
mysql.plugin OK
mysql.proc OK
mysql.procs_priv OK
mysql.proxies_priv OK
mysql.roles_mapping OK
mysql.servers OK
mysql.table_stats OK
mysql.tables_priv OK
mysql.time_zone OK
mysql.time_zone_leap_second OK
mysql.time_zone_name OK
mysql.time_zone_transition OK
mysql.time_zone_transition_type OK
mysql.transaction_registry OK
homeassistant.event_data OK
homeassistant.event_types OK
homeassistant.events OK
homeassistant.migration_changes OK
homeassistant.recorder_runs OK
homeassistant.schema_changes OK
homeassistant.state_attributes OK
homeassistant.states OK
homeassistant.states_meta OK
homeassistant.statistics OK
homeassistant.statistics_meta OK
homeassistant.statistics_runs OK
homeassistant.statistics_short_term OK
nginxproxymanager.access_list OK
nginxproxymanager.access_list_auth OK
nginxproxymanager.access_list_client OK
nginxproxymanager.audit_log OK
nginxproxymanager.auth OK
nginxproxymanager.certificate OK
nginxproxymanager.dead_host OK
nginxproxymanager.migrations OK
nginxproxymanager.migrations_lock OK
nginxproxymanager.proxy_host OK
nginxproxymanager.redirection_host OK
nginxproxymanager.setting OK
nginxproxymanager.stream OK
nginxproxymanager.user OK
nginxproxymanager.user_permission OK
phpmyadmin.pma__bookmark OK
phpmyadmin.pma__central_columns OK
phpmyadmin.pma__column_info OK
phpmyadmin.pma__designer_settings OK
phpmyadmin.pma__export_templates OK
phpmyadmin.pma__favorite OK
phpmyadmin.pma__history OK
phpmyadmin.pma__navigationhiding OK
phpmyadmin.pma__pdf_pages OK
phpmyadmin.pma__recent OK
phpmyadmin.pma__relation OK
phpmyadmin.pma__savedsearches OK
phpmyadmin.pma__table_coords OK
phpmyadmin.pma__table_info OK
phpmyadmin.pma__table_uiprefs OK
phpmyadmin.pma__tracking OK
phpmyadmin.pma__userconfig OK
phpmyadmin.pma__usergroups OK
phpmyadmin.pma__users OK
sys.sys_config OK
[14:49:25] INFO: Ensuring internal database upgrades are performed
[14:49:30] INFO: Ensure databases exists
[14:49:30] INFO: Create database homeassistant
[14:49:30] INFO: Ensure users exists and are updated
[14:49:30] INFO: Update user hassio
[14:49:30] INFO: Init/Update rights
[14:49:30] INFO: Granting all privileges to hassio on homeassistant
[14:49:30] INFO: Sending service information to Home Assistant
s6-rc: info: service mariadb-post successfully started
s6-rc: info: service legacy-services: starting
s6-rc: info: service legacy-services successfully started@N3rdix I actually noticed this errors in my MariaDB too:
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 14:49:24 3 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).I think the problem here is that just the upgrade step gets run afterwards:
...
[14:49:25] INFO: Ensuring internal database upgrades are performedReading MariaDB documentation this kinda seems expected (see https://mariadb.com/kb/en/upgrading-between-major-mariadb-versions/#step-by-step-instructions-for-upgrades).
Do they appear for you when restarting the addon?
paboman
commented
5 months ago did you tried to restart HA to check if MariaBD 2.7.0 start as it should ?
Three scenarios:
- Did upgrade to 2.7 without stopping MariaDB. Multiple errors as described in this thread, despite restarting HA and the computer it runs on.
- Restored 2.6.1, stopped the add-on, upgraded to 2.7, started the add-on. Everything functions as it should.
- After doing (2), I tried rebooting HA, and everything continued to function. I then restarted the computer HA runs on, and everything continued to function.
your method works, thank you !
N3rdix
commented
5 months ago Do they appear for you when restarting the addon?
No, seems to have been a one-time error. Log looks clean at restart:
s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service mariadb-pre: starting
[16:53:34] INFO: Using existing mariadb
s6-rc: info: service mariadb-pre successfully started
s6-rc: info: service mariadb-core: starting
s6-rc: info: service mariadb-core successfully started
s6-rc: info: service mariadb-post: starting
[16:53:35] INFO: Starting MariaDB
2024-04-16 16:53:35 0 [Note] Starting MariaDB 10.11.6-MariaDB source revision fecd78b83785d5ae96f2c6ff340375be803cd299 as process 67
2024-04-16 16:53:35 0 [Note] InnoDB: Compressed tables use zlib 1.3.1
2024-04-16 16:53:35 0 [Note] InnoDB: Number of transaction pools: 1
2024-04-16 16:53:35 0 [Note] InnoDB: Using ARMv8 crc32 instructions
2024-04-16 16:53:35 0 [Note] mysqld: O_TMPFILE is not supported on /var/tmp (disabling future attempts)
2024-04-16 16:53:35 0 [Note] InnoDB: Using Linux native AIO
2024-04-16 16:53:35 0 [Note] InnoDB: Initializing buffer pool, total size = 128.000MiB, chunk size = 2.000MiB
2024-04-16 16:53:35 0 [Note] InnoDB: Completed initialization of buffer pool
2024-04-16 16:53:35 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes)
2024-04-16 16:53:35 0 [Note] InnoDB: End of log at LSN=129652326089
2024-04-16 16:53:35 0 [Note] InnoDB: 128 rollback segments are active.
2024-04-16 16:53:35 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ...
2024-04-16 16:53:35 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB.
2024-04-16 16:53:35 0 [Note] InnoDB: log sequence number 129652326089; transaction id 27097331
2024-04-16 16:53:35 0 [Note] Plugin 'FEEDBACK' is disabled.
2024-04-16 16:53:35 0 [Note] InnoDB: Loading buffer pool(s) from /data/databases/ib_buffer_pool
2024-04-16 16:53:35 0 [Note] Server socket created on IP: '0.0.0.0'.
2024-04-16 16:53:35 0 [Note] Server socket created on IP: '::'.
2024-04-16 16:53:35 0 [Note] mysqld: ready for connections.
Version: '10.11.6-MariaDB' socket: '/run/mysqld/mysqld.sock' port: 3306 Alpine Linux
2024-04-16 16:53:35 0 [Note] InnoDB: Buffer pool(s) load completed at 240416 16:53:35
[16:53:36] INFO: MariaDB started
[16:53:36] INFO: Check data integrity and fix corruptions
mysql.column_stats OK
mysql.columns_priv OK
mysql.db OK
mysql.event OK
mysql.func OK
mysql.global_priv OK
mysql.gtid_slave_pos OK
mysql.help_category OK
mysql.help_keyword OK
mysql.help_relation OK
mysql.help_topic OK
mysql.index_stats OK
mysql.innodb_index_stats OK
mysql.innodb_table_stats OK
mysql.plugin OK
mysql.proc OK
mysql.procs_priv OK
mysql.proxies_priv OK
mysql.roles_mapping OK
mysql.servers OK
mysql.table_stats OK
mysql.tables_priv OK
mysql.time_zone OK
mysql.time_zone_leap_second OK
mysql.time_zone_name OK
mysql.time_zone_transition OK
mysql.time_zone_transition_type OK
mysql.transaction_registry OK
homeassistant.event_data OK
homeassistant.event_types OK
homeassistant.events OK
homeassistant.migration_changes OK
homeassistant.recorder_runs OK
homeassistant.schema_changes OK
homeassistant.state_attributes OK
homeassistant.states OK
homeassistant.states_meta OK
homeassistant.statistics OK
homeassistant.statistics_meta OK
homeassistant.statistics_runs OK
homeassistant.statistics_short_term OK
nginxproxymanager.access_list OK
nginxproxymanager.access_list_auth OK
nginxproxymanager.access_list_client OK
nginxproxymanager.audit_log OK
nginxproxymanager.auth OK
nginxproxymanager.certificate OK
nginxproxymanager.dead_host OK
nginxproxymanager.migrations OK
nginxproxymanager.migrations_lock OK
nginxproxymanager.proxy_host OK
nginxproxymanager.redirection_host OK
nginxproxymanager.setting OK
nginxproxymanager.stream OK
nginxproxymanager.user OK
nginxproxymanager.user_permission OK
phpmyadmin.pma__bookmark OK
phpmyadmin.pma__central_columns OK
phpmyadmin.pma__column_info OK
phpmyadmin.pma__designer_settings OK
phpmyadmin.pma__export_templates OK
phpmyadmin.pma__favorite OK
phpmyadmin.pma__history OK
phpmyadmin.pma__navigationhiding OK
phpmyadmin.pma__pdf_pages OK
phpmyadmin.pma__recent OK
phpmyadmin.pma__relation OK
phpmyadmin.pma__savedsearches OK
phpmyadmin.pma__table_coords OK
phpmyadmin.pma__table_info OK
phpmyadmin.pma__table_uiprefs OK
phpmyadmin.pma__tracking OK
phpmyadmin.pma__userconfig OK
phpmyadmin.pma__usergroups OK
phpmyadmin.pma__users OK
sys.sys_config OK
[16:53:36] INFO: Ensuring internal database upgrades are performed
[16:53:36] INFO: Ensure databases exists
[16:53:36] INFO: Create database homeassistant
[16:53:36] INFO: Ensure users exists and are updated
[16:53:36] INFO: Update user hassio
[16:53:36] INFO: Init/Update rights
[16:53:36] INFO: Granting all privileges to hassio on homeassistant
[16:53:37] INFO: Sending service information to Home Assistant
s6-rc: info: service mariadb-post successfully started
s6-rc: info: service legacy-services: starting
s6-rc: info: service legacy-services successfully started@N3rdix ok thanks for confirming. So I think these errors are are really not problematic (and not really relevant for the issue here).
@EvgenyBash so your shutdown log looks good. Is that really from the last shutdown before the upgrade? And you see the same error on startup as the original poster saw?
agners
commented
5 months ago I just realized, the Supervisor logs OP shared actually show the problem we run into: The MariaDB took longer than 30s to shutdown. It got killed by Docker, so that then caused problem for the new version:
2024-04-16 00:03:21.486 INFO (SyncWorker_2) [supervisor.docker.manager] Stopping addon_core_mariadb application
2024-04-16 00:03:51.108 INFO (SyncWorker_2) [supervisor.docker.manager] Cleaning addon_core_mariadb application
2@paboman what is the size of your database? Is each stop of MariaDB taking that long for you? :thinking:
EvgenyBash
commented
5 months ago @agners This is the log already after the 2.6.1 restore. Most likely MariaDB really took a long time to shut down.
EvgenyBash
commented
5 months ago After update to 2.7.0 log:
mysql.help_relation OK
mysql.help_topic OK
mysql.index_stats OK
mysql.innodb_index_stats OK
mysql.innodb_table_stats OK
mysql.plugin OK
mysql.proc OK
mysql.procs_priv OK
mysql.proxies_priv OK
mysql.roles_mapping OK
mysql.servers OK
mysql.table_stats OK
mysql.tables_priv OK
mysql.time_zone OK
mysql.time_zone_leap_second OK
mysql.time_zone_name OK
mysql.time_zone_transition OK
mysql.time_zone_transition_type OK
mysql.transaction_registry OK
2024-04-16 19:10:55 6 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 19:10:55 6 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 19:10:55 10 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 19:10:55 10 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 19:10:55 10 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 19:10:55 10 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 19:10:55 10 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 19:10:55 10 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 19:10:55 10 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 19:10:55 10 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 19:10:55 10 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 19:10:55 10 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 19:10:55 10 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 19:10:55 10 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 19:10:55 10 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 19:10:55 10 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 19:10:55 10 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 19:10:55 10 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 19:10:55 10 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 19:10:55 10 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 19:10:55 10 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 19:10:55 10 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 19:10:56 10 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 19:10:56 10 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 19:10:56 10 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 19:10:56 10 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 19:10:56 10 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 19:10:56 10 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 19:10:56 10 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 19:10:56 10 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 19:10:56 10 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 19:10:56 10 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 19:10:56 10 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 19:10:56 10 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-16 19:10:56 10 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-16 19:10:56 10 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
homeassistant.event_data OK
homeassistant.event_types OK
homeassistant.events OK
homeassistant.migration_changes OK
homeassistant.recorder_runs OK
homeassistant.schema_changes OK
homeassistant.state_attributes OK
homeassistant.states OK
homeassistant.states_meta OK
homeassistant.statistics OK
homeassistant.statistics_meta OK
homeassistant.statistics_runs OK
homeassistant.statistics_short_term OK
phpmyadmin.pma__bookmark OK
phpmyadmin.pma__central_columns OK
phpmyadmin.pma__column_info OK
phpmyadmin.pma__designer_settings OK
phpmyadmin.pma__export_templates OK
phpmyadmin.pma__favorite OK
phpmyadmin.pma__history OK
phpmyadmin.pma__navigationhiding OK
phpmyadmin.pma__pdf_pages OK
phpmyadmin.pma__recent OK
phpmyadmin.pma__relation OK
phpmyadmin.pma__savedsearches OK
phpmyadmin.pma__table_coords OK
phpmyadmin.pma__table_info OK
phpmyadmin.pma__table_uiprefs OK
phpmyadmin.pma__tracking OK
phpmyadmin.pma__userconfig OK
phpmyadmin.pma__usergroups OK
phpmyadmin.pma__users OK
sys.sys_config OK
[19:10:56] INFO: Ensuring internal database upgrades are performed
[19:12:39] INFO: Ensure databases exists
[19:12:39] INFO: Create database homeassistant
[19:12:39] INFO: Ensure users exists and are updated
[19:12:40] INFO: Update user hass
[19:12:40] INFO: Init/Update rights
[19:12:40] INFO: Granting all privileges to hass on homeassistant
[19:12:41] INFO: Sending service information to Home Assistant
s6-rc: info: service mariadb-post successfully started
s6-rc: info: service legacy-services: starting
s6-rc: info: service legacy-services successfully started
- Restored 2.6.1, stopped the add-on, upgraded to 2.7, started the add-on. Everything functions as it should.
It seems this stop right after a restore just took not as long then. :thinking:
I guess the stop on upgrade took much longer as the add-on has been started much longer, and had a lot of data in page cache and whatnot. So stop took longer that time around.
I guess the fix here is to just increase the shutdown timeout (currently 18 seconds, plus some more until kill kicks in).
kakou-fr
commented
5 months ago same problem ... with a 14go db on rpi4 (ssd) trying to restore my last backup ...
lmagyar
commented
5 months ago I guess the fix here is to just increase the shutdown timeout (currently 18 seconds, plus some more until kill kicks in).
S6_SERVICES_GRACETIME=18000 is for the legacy services.d, so it has no effect in the current rc.d structure and can be deleted (if I understand the S6 docs correctly: https://github.com/just-containers/s6-overlay).
As I understand S6 waits for the rc.d services as long as they need to shut down in an orderly fashion.
Isn't this 30s a Supervisor timeout?
cortado
commented
5 months ago Same problem here. RPi4, HassOSm latest everything.
AFter update and after removing and reinstalling plugin, I end up with the same every time, this in an endless loop:
[08:27:01] INFO: Starting MariaDB 2024-04-17 8:27:02 0 [Note] Starting MariaDB 10.11.6-MariaDB source revision fecd78b83785d5ae96f2c6ff340375be803cd299 as process 3344 2024-04-17 8:27:02 0 [ERROR] mysqld: Got error 'Size of control file is smaller than expected' when trying to use aria control file '/data/databases/aria_log_control' 2024-04-17 8:27:02 0 [ERROR] Plugin 'Aria' registration as a STORAGE ENGINE failed. 2024-04-17 8:27:02 0 [Note] InnoDB: Compressed tables use zlib 1.3.1 2024-04-17 8:27:02 0 [Note] InnoDB: Number of transaction pools: 1 2024-04-17 8:27:02 0 [Note] InnoDB: Using ARMv8 crc32 instructions 2024-04-17 8:27:02 0 [Note] mysqld: O_TMPFILE is not supported on /var/tmp (disabling future attempts) 2024-04-17 8:27:02 0 [Note] InnoDB: Using Linux native AIO 2024-04-17 8:27:02 0 [Note] InnoDB: Initializing buffer pool, total size = 128.000MiB, chunk size = 2.000MiB 2024-04-17 8:27:02 0 [Note] InnoDB: Completed initialization of buffer pool 2024-04-17 8:27:02 0 [ERROR] InnoDB: The Auto-extending data file './ibdata1' is of a different size 0 pages than specified by innodb_data_file_path 2024-04-17 8:27:02 0 [ERROR] InnoDB: Plugin initialization aborted with error Generic error 2024-04-17 8:27:02 0 [Note] InnoDB: Starting shutdown... 2024-04-17 8:27:02 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed. 2024-04-17 8:27:02 0 [Note] Plugin 'FEEDBACK' is disabled. 2024-04-17 8:27:02 0 [ERROR] Could not open mysql.plugin table: "Table 'mysql.plugin' doesn't exist". Some plugins may be not loaded 2024-04-17 8:27:02 0 [ERROR] Failed to initialize plugins. 2024-04-17 8:27:02 0 [ERROR] Aborting [08:27:02] INFO: Service mariadb exited with code 1 (by signal 0)
agners
commented
5 months ago
S6_SERVICES_GRACETIME=18000is for the legacy services.d, so it has no effect in the current rc.d structure and can be deleted (if I understand the S6 docs correctly: https://github.com/just-containers/s6-overlay).
Hm, right, from that documentation it appears so :thinking:
As I understand S6 waits for the rc.d services as long as they need to shut down in an orderly fashion.
From https://skarnet.org/software/s6/servicedir.html it seems that services are not killed by default by s6:
An optional regular file named timeout-kill. If such a file exists, it must only contain an unsigned integer t. If t is nonzero, then on receipt of an s6-svc -d command, which sends a SIGTERM (by default, see down-signal below) and a SIGCONT to the service, a timeout of t milliseconds is set; and if the service is still not dead after t milliseconds, then it is sent a SIGKILL. If timeout-kill does not exist, or contains 0 or an invalid value, then the service is never forcibly killed (unless, of course, an s6-svc -k command is sent).
Isn't this 30s a Supervisor timeout?
The Supervisor passes the add-on configuration to Docker's stop command. From what I understand, this timeout defines the time by when a SIGKILL is being sent to all the services in the container. The MariaDB add-on defines a 20s timeout.
At this point, it is not clear to me why in the OP's log there were exactly 30s between stop and cleanup :thinking:
agners
commented
5 months ago I've tried to simulate a non-cooperating MariaDB (or one which has longer than 20s), and at least on my system it took exactly 20s as specified:
...
Apr 17 07:24:10 ha-fujitsu hassio_supervisor[465]: 2024-04-17 09:24:10.016 INFO (SyncWorker_4) [supervisor.docker.manager] Stopping addon_core_mariadb application
...
Apr 17 07:24:30 ha-fujitsu dockerd[465]: time="2024-04-17T07:24:30.044440250Z" level=info msg="Container failed to exit within 20s of signal 15 - using the force" container=e271a635e141d499272589c290db41b35f8ce3a50b4df94e6d44012b8bd6a780
...
Apr 17 07:24:30 ha-fujitsu hassio_supervisor[465]: 2024-04-17 09:24:30.329 INFO (SyncWorker_4) [supervisor.docker.manager] Cleaning addon_core_mariadb applicationI am assuming OPs system was just overwhelmed in general, which lead these two operation to take a bit longer. The fact that it almost exactly took 30s seems to be just coincidence.
Anyhow, since a couple of users run into this problem, increasing the timeout makes sense. I went ahead and increased it to 55s with #3570.
@lmagyar review welcome :smile:
agners
commented
5 months ago Anyone which runs into
2024-04-16 0:31:13 0 [ERROR] InnoDB: Upgrade after a crash is not supported. The redo log was created with MariaDB 10.6.8. You must start up and shut down MariaDB 10.7 or earlier.Please restore the 2.6.1 backup and upgrade again. This second upgrade should typically work as shutdown is much quicker after a fresh restore/start of the old add-on version.
As this is a problem of 2.6.x not shutting down properly, we can't really fix it with an update. The fix https://github.com/home-assistant/addons/pull/3570 will only address this problem for future upgrades.
agners
commented
5 months ago @cortado your case looks a bit different:
2024-04-17 8:27:02 0 [ERROR] InnoDB: The Auto-extending data file './ibdata1' is of a different size 0 pages than specified by innodb_data_file_pathMaybe still try restore the old version and upgrade again, it might be shutdown related to. If the problem persists, please open a new issue.
thijsdejong
commented
5 months ago Just ran into this problem, updated from the command line, so no backup. Lost all my historical data from the past 2 years 😢
Downgrading to 2.5.0 (the last backup I had from 2 years ago) works fine. Stopping the addon and then updating results in the exact same error as before.
JohannCR
commented
4 months ago Same here. After 3 unsuccessful tries, I finally managed to get it working. The key to update from a previous (restored) version is waiting time. For all unsuccessful tries I either did not wait long enough or interrupted the process (restore/stop/update) at some point... You need to let a lot of time pass between steps so the addon has time to check and fix the database. Probably takes a lot of time when it's a big database.
agners
commented
4 months ago Just ran into this problem, updated from the command line, so no backup. Lost all my historical data from the past 2 years 😢
Ugh. Yeah unfortunately you can't downgrade without having a backup.
What you could do in this case is take a backup of the upgraded/non-working state (as the data is still there, it is just the new version of MariaDB cannot recover it :cry: ), download that backup, open the tar (I think 7-Zip can do that on Windows) edit backup.json, specifically change the version field to 2.6.1 for core_mariadb. Recreate the tar, upload the backup and restore it. This then should install MariaDB 2.6.1 again which should be able to recover the issue. Then make sure it gracefully shuts down, and upgrade again.
rmkew377
commented
4 months ago Hello, my HA is currently running MariaDB 2.6.1 An update to 2.7.1 is offered. A “restart” immediately before the update is recommended. Is this a safe method or can you update even if the MariaDB add-on is stopped? Sorry,, but I'm a bit worried about the update ;)
agners
commented
4 months ago Is this a safe method or can you update even if the MariaDB add-on is stopped?
The important thing here is that the stop happened cleanly. Check the add-on logs after the stop to learn if the stop worked cleanly. It should contain something this line:
[14:02:20] INFO: Service mariadb exited with code 0 (by signal 0)In any case, a backup is always a good idea :smile:
rmkew377
commented
4 months ago Thank you for this quick answer! I'll try it!
miga97
commented
4 months ago hi, I'm trying to update, but my container always shuts down with code 137 from version 2.6.1 to 2.7.1 obviously I tried to do a restart of mariadb first
s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service mariadb-pre: starting
s6-rc: info: service mariadb-pre successfully started
[16:03:26] INFO: Using existing mariadb
s6-rc: info: service mariadb-core: starting
s6-rc: info: service mariadb-core successfully started
s6-rc: info: service mariadb-post: starting
[16:03:26] INFO: Starting MariaDB
2024-04-22 16:03:26 0 [Note] Starting MariaDB 10.6.12-MariaDB source revision 4c79e15cc3716f69c044d4287ad2160da8101cdc as process 66
2024-04-22 16:03:26 0 [Note] mysqld: Aria engine: starting recovery
recovered pages: 0% 20% 41% 53% 69% 87% 100% (0.0 seconds); tables to flush: 2 1 0
(5.3 seconds);
2024-04-22 16:03:35 0 [Note] mysqld: Aria engine: recovery done
2024-04-22 16:03:35 0 [Note] InnoDB: Compressed tables use zlib 1.2.12
2024-04-22 16:03:35 0 [Note] InnoDB: Number of pools: 1
2024-04-22 16:03:35 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
2024-04-22 16:03:35 0 [Note] InnoDB: Using Linux native AIO
2024-04-22 16:03:35 0 [Note] mysqld: O_TMPFILE is not supported on /var/tmp (disabling future attempts)
2024-04-22 16:03:35 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728
2024-04-22 16:03:35 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=289402935541,289402935541
2024-04-22 16:03:35 0 [Note] InnoDB: Completed initialization of buffer pool
2024-04-22 16:03:40 0 [Note] InnoDB: Starting final batch to recover 2168 pages from redo log.
2024-04-22 16:03:46 0 [Note] InnoDB: 128 rollback segments are active.
2024-04-22 16:03:46 0 [Note] InnoDB: Removed temporary tablespace data file: "./ibtmp1"
2024-04-22 16:03:46 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2024-04-22 16:03:46 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2024-04-22 16:03:46 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2024-04-22 16:03:46 0 [Note] InnoDB: 10.6.12 started; log sequence number 289410969812; transaction id 55799888
2024-04-22 16:03:46 0 [Note] Plugin 'FEEDBACK' is disabled.
2024-04-22 16:03:46 0 [Note] InnoDB: Loading buffer pool(s) from /data/databases/ib_buffer_pool
2024-04-22 16:03:49 0 [Note] InnoDB: Buffer pool(s) load completed at 240422 16:03:49
2024-04-22 16:03:52 0 [Note] Server socket created on IP: '0.0.0.0'.
2024-04-22 16:03:52 0 [Note] Server socket created on IP: '::'.
2024-04-22 16:03:58 0 [Note] mysqld: ready for connections.
Version: '10.6.12-MariaDB' socket: '/run/mysqld/mysqld.sock' port: 3306 MariaDB Server
[16:03:58] INFO: MariaDB started
[16:03:59] INFO: Check data integrity and fix corruptions
mysql.column_stats OK
mysql.columns_priv OK
mysql.db OK
mysql.event OK
mysql.global_priv OK
mysql.func OK
mysql.gtid_slave_pos OK
mysql.help_keyword OK
mysql.help_category OK
mysql.help_relation OK
mysql.help_topic OK
mysql.index_stats OK
mysql.innodb_index_stats OK
mysql.innodb_table_stats OK
mysql.plugin OK
mysql.proc OK
mysql.proxies_priv OK
mysql.procs_priv OK
mysql.roles_mapping OK
mysql.table_stats OK
mysql.servers OK
mysql.tables_priv OK
mysql.time_zone OK
mysql.time_zone_leap_second OK
mysql.time_zone_name OK
mysql.time_zone_transition_type OK
mysql.time_zone_transition OK
mysql.transaction_registry OK
assiodb.event_types OK
assiodb.event_data OK
assiodb.migration_changes OK
assiodb.events OK
assiodb.recorder_runs OK
assiodb.schema_changes OK
assiodb.state_attributes OK
assiodb.states OK
assiodb.states_meta OK
assiodb.statistics OK
assiodb.statistics_meta OK
assiodb.statistics_runs OK
assiodb.statistics_short_term OK
phpmyadmin.pma__central_columns OK
phpmyadmin.pma__bookmark OK
phpmyadmin.pma__column_info OK
phpmyadmin.pma__designer_settings OK
phpmyadmin.pma__export_templates OK
phpmyadmin.pma__history OK
phpmyadmin.pma__favorite OK
phpmyadmin.pma__navigationhiding OK
phpmyadmin.pma__pdf_pages OK
phpmyadmin.pma__recent OK
phpmyadmin.pma__relation OK
phpmyadmin.pma__savedsearches OK
phpmyadmin.pma__table_coords OK
phpmyadmin.pma__table_uiprefs OK
phpmyadmin.pma__table_info OK
phpmyadmin.pma__tracking OK
phpmyadmin.pma__usergroups OK
phpmyadmin.pma__userconfig OK
phpmyadmin.pma__users OK
sys.sys_config OK
[16:04:02] INFO: Ensuring internal database upgrades are performed
[16:04:03] INFO: Ensure databases exists
[16:04:03] INFO: Create database assiodb
[16:04:03] INFO: Ensure users exists and are updated
[16:04:05] INFO: Update user assio
[16:04:06] INFO: Update user grafana
[16:04:06] INFO: Init/Update rights
[16:04:07] INFO: Granting all privileges to assio on assiodb
[16:04:09] INFO: Granting all privileges to grafana on assiodb
[16:04:11] INFO: Sending service information to Home Assistant
s6-rc: info: service mariadb-post successfully started
s6-rc: info: service legacy-services: starting
s6-rc: info: service legacy-services successfully started
s6-rc: info: service legacy-services: stopping
s6-rc: info: service legacy-services successfully stopped
s6-rc: info: service mariadb-post: stopping
[16:56:41] INFO: Deleting service information from Home Assistant
s6-rc: info: service mariadb-post successfully stopped
s6-rc: info: service mariadb-core: stopping
2024-04-22 16:56:43 0 [Note] mysqld (initiated by: unknown): Normal shutdown
2024-04-22 16:56:43 0 [Note] InnoDB: FTS optimize thread exiting.
2024-04-22 16:56:44 0 [Note] InnoDB: Starting shutdown...
2024-04-22 16:56:44 0 [Note] InnoDB: Dumping buffer pool(s) to /data/databases/ib_buffer_pool
2024-04-22 16:56:44 0 [Note] InnoDB: Restricted to 2028 pages due to innodb_buf_pool_dump_pct=25
2024-04-22 16:56:44 0 [Note] InnoDB: Buffer pool(s) dump completed at 240422 16:56:44and obviously if I try to update this happens:
s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service mariadb-pre: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service mariadb-pre successfully started
[17:15:46] INFO: Using existing mariadb
s6-rc: info: service mariadb-core: starting
s6-rc: info: service mariadb-core successfully started
s6-rc: info: service mariadb-post: starting
[17:15:46] INFO: Starting MariaDB
2024-04-22 17:15:46 0 [Note] Starting MariaDB 10.11.6-MariaDB source revision fecd78b83785d5ae96f2c6ff340375be803cd299 as process 67
2024-04-22 17:15:47 0 [Note] mysqld: Aria engine: starting recovery
recovered pages: 0% 20% 41% 53% 69% 87% 100% (0.0 seconds); tables to flush: 2 1 0
(2.0 seconds);
2024-04-22 17:15:51 0 [Note] mysqld: Aria engine: recovery done
2024-04-22 17:15:54 0 [Note] InnoDB: Compressed tables use zlib 1.3.1
2024-04-22 17:15:54 0 [Note] InnoDB: Number of transaction pools: 1
2024-04-22 17:15:54 0 [Note] mysqld: O_TMPFILE is not supported on /var/tmp (disabling future attempts)
2024-04-22 17:15:54 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
2024-04-22 17:15:54 0 [Note] InnoDB: Using Linux native AIO
2024-04-22 17:15:54 0 [Note] InnoDB: Completed initialization of buffer pool
2024-04-22 17:15:54 0 [Note] InnoDB: Initializing buffer pool, total size = 128.000MiB, chunk size = 2.000MiB
2024-04-22 17:15:54 0 [ERROR] InnoDB: Upgrade after a crash is not supported. The redo log was created with MariaDB 10.6.8. You must start up and shut down MariaDB 10.7 or earlier.
2024-04-22 17:15:54 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes)
2024-04-22 17:15:54 0 [ERROR] InnoDB: Plugin initialization aborted with error Generic error
2024-04-22 17:15:54 0 [Note] InnoDB: Starting shutdown...
2024-04-22 17:15:54 0 [Note] Plugin 'FEEDBACK' is disabled.
2024-04-22 17:15:54 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2024-04-22 17:15:54 0 [ERROR] Unknown/unsupported storage engine: InnoDB
2024-04-22 17:15:54 0 [ERROR] Aborting
[17:15:59] INFO: Starting MariaDB
[17:15:57] INFO: Service mariadb exited with code 1 (by signal 0)
2024-04-22 17:15:59 0 [Note] Starting MariaDB 10.11.6-MariaDB source revision fecd78b83785d5ae96f2c6ff340375be803cd299 as process 117
2024-04-22 17:16:01 0 [Note] InnoDB: Compressed tables use zlib 1.3.1
and so on ...I don't know what to do, fortunately restore version 2.6.1, it always works
agners
commented
4 months ago Can you restore 2.6.1 again, then try the following command to stop the add-on before updating in the Advanced SSH & Web Terminal (from the community store)
docker stop --time 300 addon_core_mariadbCan you restore 2.6.1 again, then try the following command to stop the add-on before updating in the Advanced SSH & Web Terminal (from the community store)
docker stop --time 300 addon_core_mariadb
yes, it works, it took 42 seconds
maybe it should be added in the supervisor that it has to wait that time only when updating. I imagine that it cannot be set as a default for things such as that the system must be able to shut down in an emergency situation such as when the power goes out and you are on UPS.
but I don't know how it works, I just wanted to say my opinion and thank you so much for your help
Even if it works, errors have come up repeatedly, I don't know if it helps:
2024-04-22 20:29:55 4 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
2024-04-22 20:29:55 4 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
[20:29:55] INFO: MariaDB started
[20:29:56] INFO: Check data integrity and fix corruptions
[20:29:59] INFO: Ensuring internal database upgrades are performed
2024-04-22 20:29:55 4 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
2024-04-22 20:29:55 4 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).Even if it works, errors have come up repeatedly
If you restart the 2.7.1 version, these errors should disappear: during first startup after the upgrade, mariadb has issues with these statistics data (20:29:55 4 [ERROR]), but a few seconds later a script runs that updates these schema ([20:29:59] INFO: Ensuring internal database upgrades are performed).
RubenKelevra
commented
4 months ago Same issue here.
I've never have had a power failure or something like that in the past 2 years, just installed every update and it bricked itself.
@agners how about shipping the old binary in an update next to the current one, and if there this failure to start it will spin up the old version automatically? 🤔
So it would repair the damage and then restart the container cleanly.
I mean I don't know how many people are affected, but it would prefer not to loose my history or tinker with the files of the container to get the current database state back.
I'm sure many other users are not even confident in doing something like this at all.
guice
commented
4 months ago Today, after updating HomeAssistant, I ran into this issue. I thought I was running successfully on MariaDB 2.7.1 since the update and since 2.7.x. I did see something about needing to restart before updating to 2.7.1 - I verified I was on 2.7.0, so I updated to .1 without question (i.e. since I "thought" I was running on 2.7.0 just fine, it meant I was okay).
Anyway, after updating HomeAssistant (supervisord), MariaDB now will not start giving me the above error. And for some reason, MariaDB isn't in my backups (probably because it was crashing ....), I'm left without a way to downgrade.
How do I manually tell Supervisor to use 2.6.1? Or how can I just download the right Docker image and run it myself? I just need the full docker cli, and I run it myself as a temporary solution. I can't seen to find that.
I've tried this, but no go:
docker run --rm --network hassio --name addon_core_mariadb --env MARIADB_USER=homeassistant --env MARIADB_PASSWORD=password --env MARIADB_DATABASE=homeassistant --env MARIADB_ROOT_PASSWORD=password homeassistant/amd64-addon-mariadb:2.6.1@guice what you can do (but it is a hack) just download the other version and tag it as the latest:
docker pull homeassistant/aarch64-addon-mariadb:2.6.1
docker tag homeassistant/aarch64-addon-mariadb:2.6.1 homeassistant/aarch64-addon-mariadb:2.7.1Then restart the add-on via UI. Then try stopping it, and check for successful stop in the logs.
Don't forget to restore the original container image after 2.6.1 startup/stop succeeded.
docker pull homeassistant/aarch64-addon-mariadb:2.6.1Then restart the container. Note this is a hack, and only really works as long as there is no incompatible config change (which is the case between these two versions).
agners
commented
4 months ago @agners how about shipping the old binary in an update next to the current one, and if there this failure to start it will spin up the old version automatically? 🤔
Yeah it is not all that easy, as we get the binaries from Alpine directly. Afaik, it isn't possible to deploy two mariadb packages (from different Alpine versions) into one container.
In the end, it seems that only so many people are affected, and there are a couple of workarounds documented here.
RubenKelevra
commented
4 months ago @agners how about shipping the old binary in an update next to the current one, and if there this failure to start it will spin up the old version automatically? 🤔
Yeah it is not all that easy, as we get the binaries from Alpine directly. Afaik, it isn't possible to deploy two mariadb packages (from different Alpine versions) into one container.
In the end, it seems that only so many people are affected, and there are a couple of workarounds documented here.
Well I've tried your backup -> modify the json -> upload backup method.
I could be wrong, but in 7zip it appears like the json file is in a "." folder.
If extracted under windows with 7zip the folder gets dropped and the json is plain in the root folder of the extraction. And you can't create a "." folder under windows.
Just modifying the tar with 7zip does not work either, as the folder is deleted as soon as you delete the json file.
Creating a dot folder in 7zip does not work.
I've tried to just pack the json in the tar without a folder but either I did something wrong or it doesn't work that way: The backup does not show up in HA.
In addition I discovered that trying to backing up mariadb in the broken state does not work: You end up with a 10 KB file which only contains the json.
Only after stopping the add on the backup completes correctly.
Describe the issue you are experiencing
what should i do ?
What type of installation are you running?
Home Assistant OS
Which operating system are you running on?
Home Assistant Operating System
Which add-on are you reporting an issue with?
MariaDB
What is the version of the add-on?
2.7.0
Steps to reproduce the issue
1.update from 2.6.1 to 2.7.0
System Health information
System Information
Home Assistant Community Store
GitHub API | ok -- | -- GitHub Content | ok GitHub Web | ok GitHub API Calls Remaining | 4885 Installed Version | 1.34.0 Stage | running Available Repositories | 1402 Downloaded Repositories | 17Home Assistant Cloud
logged_in | false -- | -- can_reach_cert_server | ok can_reach_cloud_auth | ok can_reach_cloud | okHome Assistant Supervisor
host_os | Home Assistant OS 12.2 -- | -- update_channel | stable supervisor_version | supervisor-2024.04.0 agent_version | 1.6.0 docker_version | 25.0.5 disk_total | 30.8 GB disk_used | 7.4 GB healthy | true supported | true board | ova supervisor_api | ok version_api | ok installed_addons | File editor (5.8.0), Terminal & SSH (9.13.0), Mosquitto broker (6.4.0), Zigbee2MQTT (1.36.1-1), MariaDB (2.7.0), Samba share (12.3.1), Ring-MQTT with Video Streaming (5.6.3), Studio Code Server (5.15.0), Node-RED (17.0.11)Dashboards
dashboards | 2 -- | -- resources | 8 views | 9 mode | storageSonoff
version | 3.6.0 (8dd8af9) -- | -- cloud_online | 1 / 2 local_online | 1 / 1Anything in the Supervisor logs that might be useful for us?
Anything in the add-on logs that might be useful for us?
Additional information
No response