colinmollenhour / mariadb-galera-swarm

MariaDb Galera Cluster container based on official mariadb image which can auto-bootstrap and recover cluster state.
https://hub.docker.com/r/colinmollenhour/mariadb-galera-swarm
Apache License 2.0
216 stars 101 forks source link

Aborted connection 11 to db: 'unconnected' user #100

Closed csib closed 3 years ago

csib commented 3 years ago

Hi,

I've just copy-paste your example from here: https://github.com/colinmollenhour/mariadb-galera-swarm/tree/master/examples/swarm

And I've received: Aborted connection 11 to db: 'unconnected' user: 'system' host: '127.0.0.1' (Got timeout reading communication packets)

Full log from the seed (node is not started yet):

`...------======------... MariaDB Galera Start Script ...------======------... Got NODE_ADDRESS=10.0.6.45 Generating cluster bootstrap script... /usr/local/bin/start.sh: ignoring /docker-entrypoint-initdb.d/*

Starting seed node ===|mysqld.sh|===: Starting a new cluster... ===|mysqld.sh|===: Initializing MariaDb data directory... Tailing /tmp/mysql-console/fifo... 2020-08-27 20:36:38 140629995533632 [Warning] You need to use --log-bin to make --binlog-format work.

PLEASE REMEMBER TO SET A PASSWORD FOR THE MariaDB root USER ! To do so, start the server, then issue the following commands:

'/usr/bin/mysqladmin' -u root password 'new-password' '/usr/bin/mysqladmin' -u root -h password 'new-password'

Alternatively you can run: '/usr/bin/mysql_secure_installation'

which will also give you the option of removing the test databases and anonymous user created by default. This is strongly recommended for production servers.

See the MariaDB Knowledgebase at http://mariadb.com/kb or the MySQL manual for more instructions.

Please report any problems at http://mariadb.org/jira

The latest information about MariaDB is available at http://mariadb.org/. You can find additional information about the MySQL part at: http://dev.mysql.com Consider joining MariaDB's strong and vibrant community: https://mariadb.org/get-involved/

===|mysqld.sh|===: Loading timezone info... 2020-08-27 20:36:40 140032752497984 [Note] mysqld (mysqld 10.2.23-MariaDB-1:10.2.23+maria~bionic) starting as process 105 ... 2020-08-27 20:36:40 140032752497984 [Warning] You need to use --log-bin to make --binlog-format work. 2020-08-27 20:36:40 140032752497984 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2020-08-27 20:36:40 140032752497984 [Note] InnoDB: Uses event mutexes 2020-08-27 20:36:40 140032752497984 [Note] InnoDB: Compressed tables use zlib 1.2.11 2020-08-27 20:36:40 140032752497984 [Note] InnoDB: Using Linux native AIO 2020-08-27 20:36:40 140032752497984 [Note] InnoDB: Number of pools: 1 2020-08-27 20:36:40 140032752497984 [Note] InnoDB: Using SSE2 crc32 instructions 2020-08-27 20:36:40 140032752497984 [Note] InnoDB: Initializing buffer pool, total size = 256M, instances = 1, chunk size = 128M 2020-08-27 20:36:40 140032752497984 [Note] InnoDB: Completed initialization of buffer pool 2020-08-27 20:36:40 140032024303360 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). 2020-08-27 20:36:40 140032752497984 [Note] InnoDB: Highest supported file format is Barracuda. 2020-08-27 20:36:40 140032752497984 [Note] InnoDB: 128 out of 128 rollback segments are active. 2020-08-27 20:36:40 140032752497984 [Note] InnoDB: Creating shared tablespace for temporary tables 2020-08-27 20:36:40 140032752497984 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ... 2020-08-27 20:36:40 140032752497984 [Note] InnoDB: File './ibtmp1' size is now 12 MB. 2020-08-27 20:36:40 140032752497984 [Note] InnoDB: Waiting for purge to start 2020-08-27 20:36:40 140032752497984 [Note] InnoDB: 5.7.25 started; log sequence number 1619987 2020-08-27 20:36:40 140031863793408 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool 2020-08-27 20:36:40 140031863793408 [Note] InnoDB: Buffer pool(s) load completed at 200827 20:36:40 2020-08-27 20:36:40 140032752497984 [Note] Plugin 'FEEDBACK' is disabled. 2020-08-27 20:36:40 140032752497984 [Note] Reading of all Master_info entries succeded 2020-08-27 20:36:40 140032752497984 [Note] Added new Master_info '' to hash table 2020-08-27 20:36:40 140032752497984 [Note] mysqld: ready for connections. Version: '10.2.23-MariaDB-1:10.2.23+maria~bionic' socket: '/tmp/mysql.sock' port: 0 mariadb.org binary distribution /usr/local/bin/run-upgrades.sh: waiting for server to become available... Warning: Unable to load '/usr/share/zoneinfo/leap-seconds.list' as time zone. Skipping it. 2020-08-27 20:36:45 140032579364608 [Note] mysqld (initiated by: unknown): Normal shutdown 2020-08-27 20:36:45 140031905756928 [Note] InnoDB: FTS optimize thread exiting. 2020-08-27 20:36:45 140032579364608 [Note] InnoDB: Starting shutdown... 2020-08-27 20:36:45 140031863793408 [Note] InnoDB: Dumping buffer pool(s) to /var/lib/mysql/ib_buffer_pool 2020-08-27 20:36:45 140031863793408 [Note] InnoDB: Buffer pool(s) dump completed at 200827 20:36:45 2020-08-27 20:36:46 140032579364608 [Note] InnoDB: Shutdown completed; log sequence number 1620015 2020-08-27 20:36:46 140032579364608 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1" 2020-08-27 20:36:46 140032579364608 [Note] mysqld: Shutdown complete

===|mysqld.sh|===: --------------------------------------------------------------- ===|mysqld.sh|===: Starting with options: --console --wsrep_sst_auth=xtrabackup:ZbgylfKamsLVrN63KJukHAL6AVjqsL+ikdF2vZddbgc= --init-file=/tmp/bootstrap.sql --wsrep-on=ON --wsrep-new-cluster --wsrep-sst-method=mariabackup --wsrep_cluster_name=cluster --wsrep_cluster_address=gcomm:// --wsrep_node_address=10.0.6.45:4567 --default-time-zone=+00:00 2020-08-27 20:36:46 139787412055360 [Note] mysqld (mysqld 10.2.23-MariaDB-1:10.2.23+maria~bionic) starting as process 34 ... 2020-08-27 20:36:46 139787412055360 [Note] WSREP: Read nil XID from storage engines, skipping position init 2020-08-27 20:36:46 139787412055360 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/galera/libgalera_smm.so' 2020-08-27 20:36:46 139787412055360 [Note] WSREP: wsrep_load(): Galera 25.3.26(r3857) by Codership Oy info@codership.com loaded successfully. 2020-08-27 20:36:46 139787412055360 [Note] WSREP: CRC-32C: using hardware acceleration. 2020-08-27 20:36:46 139787412055360 [Warning] WSREP: Could not open state file for reading: '/var/lib/mysql//grastate.dat' 2020-08-27 20:36:46 139787412055360 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1, safe_to_bootstrap: 1 2020-08-27 20:36:46 139787412055360 [Note] WSREP: Skipped GCache ring buffer recovery: could not determine history UUID. 2020-08-27 20:36:46 139787412055360 [Note] WSREP: Passing config to GCS: base_dir = /var/lib/mysql/; base_host = 10.0.6.45; base_port = 4567; cert.log_conflicts = no; cert.optimistic_pa = yes; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 1024M; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 128M; gcache.recover = yes; gcache.size = 2048M; gcomm.thread_prio = ; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT 2020-08-27 20:36:46 139787412055360 [Note] WSREP: GCache history reset: 00000000-0000-0000-0000-000000000000:0 -> 00000000-0000-0000-0000-000000000000:-1 2020-08-27 20:36:46 139787412055360 [Note] WSREP: Assign initial position for certification: -1, protocol version: -1 2020-08-27 20:36:46 139787412055360 [Note] WSREP: wsrep_sst_grab() 2020-08-27 20:36:46 139787412055360 [Note] WSREP: Start replication 2020-08-27 20:36:46 139787412055360 [Note] WSREP: 'wsrep-new-cluster' option used, bootstrapping the cluster 2020-08-27 20:36:46 139787412055360 [Note] WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:-1 2020-08-27 20:36:46 139787412055360 [Note] WSREP: protonet asio version 0 2020-08-27 20:36:46 139787412055360 [Note] WSREP: Using CRC-32C for message checksums. 2020-08-27 20:36:46 139787412055360 [Note] WSREP: backend: asio 2020-08-27 20:36:46 139787412055360 [Note] WSREP: gcomm thread scheduling priority set to other:0 2020-08-27 20:36:46 139787412055360 [Warning] WSREP: access file(/var/lib/mysql//gvwstate.dat) failed(No such file or directory) 2020-08-27 20:36:46 139787412055360 [Note] WSREP: restore pc from disk failed 2020-08-27 20:36:46 139787412055360 [Note] WSREP: GMCast version 0 2020-08-27 20:36:46 139787412055360 [Note] WSREP: (06a15c68, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567 2020-08-27 20:36:46 139787412055360 [Note] WSREP: (06a15c68, 'tcp://0.0.0.0:4567') multicast: , ttl: 1 2020-08-27 20:36:46 139787412055360 [Note] WSREP: EVS version 0 2020-08-27 20:36:46 139787412055360 [Note] WSREP: gcomm: bootstrapping new group 'cluster' 2020-08-27 20:36:46 139787412055360 [Note] WSREP: start_prim is enabled, turn off pc_recovery 2020-08-27 20:36:46 139787412055360 [Note] WSREP: Node 06a15c68 state prim 2020-08-27 20:36:46 139787412055360 [Note] WSREP: view(view_id(PRIM,06a15c68,1) memb { 06a15c68,0 } joined { } left { } partitioned { }) 2020-08-27 20:36:46 139787412055360 [Note] WSREP: save pc into disk 2020-08-27 20:36:46 139787412055360 [Note] WSREP: gcomm: connected 2020-08-27 20:36:46 139787412055360 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636 2020-08-27 20:36:46 139787412055360 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0) 2020-08-27 20:36:46 139787412055360 [Note] WSREP: Opened channel 'cluster' 2020-08-27 20:36:46 139787412055360 [Note] WSREP: Waiting for SST to complete. 2020-08-27 20:36:46 139787238209280 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 1 2020-08-27 20:36:46 139787238209280 [Note] WSREP: Starting new group from scratch: 06a1cf11-e8a5-11ea-978e-127b511c09c5 2020-08-27 20:36:46 139787238209280 [Note] WSREP: STATE_EXCHANGE: sent state UUID: 06a1d2e1-e8a5-11ea-a99f-7e9dd7c3ed16 2020-08-27 20:36:46 139787238209280 [Note] WSREP: STATE EXCHANGE: sent state msg: 06a1d2e1-e8a5-11ea-a99f-7e9dd7c3ed16 2020-08-27 20:36:46 139787238209280 [Note] WSREP: STATE EXCHANGE: got state msg: 06a1d2e1-e8a5-11ea-a99f-7e9dd7c3ed16 from 0 (4496fb739852) 2020-08-27 20:36:46 139787238209280 [Note] WSREP: Quorum results: version = 4, component = PRIMARY, conf_id = 0, members = 1/1 (joined/total), act_id = 0, last_appl. = -1, protocols = 0/9/3 (gcs/repl/appl), group UUID = 06a1cf11-e8a5-11ea-978e-127b511c09c5 2020-08-27 20:36:46 139787238209280 [Note] WSREP: Flow-control interval: [16, 16] 2020-08-27 20:36:46 139787238209280 [Note] WSREP: Trying to continue unpaused monitor 2020-08-27 20:36:46 139787238209280 [Note] WSREP: Restored state OPEN -> JOINED (0) 2020-08-27 20:36:46 139787238209280 [Note] WSREP: Member 0.0 (4496fb739852) synced with group. 2020-08-27 20:36:46 139787238209280 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 0) 2020-08-27 20:36:46 139787410409216 [Note] WSREP: New cluster view: global state: 06a1cf11-e8a5-11ea-978e-127b511c09c5:0, view# 1: Primary, number of nodes: 1, my index: 0, protocol version 3 2020-08-27 20:36:46 139787412055360 [Note] WSREP: SST complete, seqno: 0 2020-08-27 20:36:46 139787412055360 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2020-08-27 20:36:46 139787412055360 [Note] InnoDB: Uses event mutexes 2020-08-27 20:36:46 139787412055360 [Note] InnoDB: Compressed tables use zlib 1.2.11 2020-08-27 20:36:46 139787412055360 [Note] InnoDB: Using Linux native AIO 2020-08-27 20:36:46 139787412055360 [Note] InnoDB: Number of pools: 1 2020-08-27 20:36:46 139787412055360 [Note] InnoDB: Using SSE2 crc32 instructions 2020-08-27 20:36:46 139787412055360 [Note] InnoDB: Initializing buffer pool, total size = 256M, instances = 1, chunk size = 128M 2020-08-27 20:36:46 139787412055360 [Note] InnoDB: Completed initialization of buffer pool 2020-08-27 20:36:46 139784199689984 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). 2020-08-27 20:36:46 139787412055360 [Note] InnoDB: Highest supported file format is Barracuda. 2020-08-27 20:36:46 139787412055360 [Note] InnoDB: 128 out of 128 rollback segments are active. 2020-08-27 20:36:46 139787412055360 [Note] InnoDB: Creating shared tablespace for temporary tables 2020-08-27 20:36:46 139787412055360 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ... 2020-08-27 20:36:46 139787412055360 [Note] InnoDB: File './ibtmp1' size is now 12 MB. 2020-08-27 20:36:46 139787412055360 [Note] InnoDB: 5.7.25 started; log sequence number 1620015 2020-08-27 20:36:46 139784039151360 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool 2020-08-27 20:36:46 139784039151360 [Note] InnoDB: Buffer pool(s) load completed at 200827 20:36:46 2020-08-27 20:36:46 139787412055360 [Note] Plugin 'FEEDBACK' is disabled. 2020-08-27 20:36:46 139787412055360 [Note] Server socket created on IP: '0.0.0.0'. 2020-08-27 20:36:46 139787412055360 [Warning] 'user' entry 'root@4496fb739852' ignored in --skip-name-resolve mode. 2020-08-27 20:36:46 139787412055360 [Warning] 'proxies_priv' entry '@% root@4496fb739852' ignored in --skip-name-resolve mode. 2020-08-27 20:36:46 139787412055360 [Note] Reading of all Master_info entries succeded 2020-08-27 20:36:46 139787410409216 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2020-08-27 20:36:46 139787410409216 [Note] WSREP: REPL Protocols: 9 (4, 2) 2020-08-27 20:36:46 139787410409216 [Note] WSREP: Assign initial position for certification: 0, protocol version: 4 2020-08-27 20:36:46 139787412055360 [Note] Added new Master_info '' to hash table 2020-08-27 20:36:46 139787280172800 [Note] WSREP: Service thread queue flushed. 2020-08-27 20:36:46 139787410409216 [Note] WSREP: GCache history reset: 00000000-0000-0000-0000-000000000000:0 -> 06a1cf11-e8a5-11ea-978e-127b511c09c5:0 2020-08-27 20:36:46 139787410409216 [Note] WSREP: Synchronized with group, ready for connections 2020-08-27 20:36:46 139787410409216 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2020-08-27 20:36:46 139784955029248 [Warning] 'user' entry 'root@4496fb739852' ignored in --skip-name-resolve mode. 2020-08-27 20:36:46 139784955029248 [Warning] 'proxies_priv' entry '@% root@4496fb739852' ignored in --skip-name-resolve mode. 2020-08-27 20:36:46 139787412055360 [Note] mysqld: ready for connections. Version: '10.2.23-MariaDB-1:10.2.23+maria~bionic' socket: '/var/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution 2020-08-27 20:46:50 139784945846016 [Warning] Access denied for user 'root'@'10.0.6.49' (using password: YES) 2020-08-27 20:46:53 139784955029248 [Warning] Aborted connection 11 to db: 'unconnected' user: 'system' host: '127.0.0.1' (Got timeout reading communication packets)`

csib commented 3 years ago

After that, I've tried it without secrets. My config is:

version: '3.4'

services:
  seed:
    image: colinmollenhour/mariadb-galera-swarm:10.4.11-2020-01-06
    environment:
      - XTRABACKUP_PASSWORD=asd
      - SYSTEM_PASSWORD=asd
      - MYSQL_USER=joomla
      - MYSQL_PASSWORD=asd
      - MYSQL_DATABASE=db_new
      - MYSQL_ROOT_PASSWORD=asd
      - MYSQL_ROOT_HOST=%
      - DEFAULT_TIME_ZONE=Europe/Budapest
      - NODE_ADDRESS=^10.0.*.*
    networks:
      - db_network
    command: seed
    volumes:
      - mysql-data:/var/lib/mysql

  node:
    image: colinmollenhour/mariadb-galera-swarm:10.4.11-2020-01-06
    environment:
      - XTRABACKUP_PASSWORD=asd
      - SYSTEM_PASSWORD=asd
      - NODE_ADDRESS=^10.0.*.*
      - HEALTHY_WHILE_BOOTING=1
      - DEFAULT_TIME_ZONE=Europe/Budapest
    networks:
      -db_network
    command: node tasks.seed,tasks.node
    volumes:
      - mysql-data:/var/lib/mysql
    deploy:
      replicas: 0

[..]

The seed is now looks good, started to continue the proccess. Set node to 2 after they are healthy seed 0, then node 3.

Tried to load contents into the cluster through phpmyadmin.

Node log: `removed directory '/var/lib/mysql/mysql'

removed '/var/lib/mysql/multi-master.info'

removed '/var/lib/mysql/ib_logfile1'

WSREP_SST: [INFO] Waiting for SST streaming to complete! (20200827 21:36:52.575)

/usr/local/bin/run-upgrades.sh: waiting for server to become available...

2020-08-27 21:36:54 0 [Note] WSREP: (6ad6fded, 'tcp://0.0.0.0:4567') turning message relay requesting off

joiner: => Rate:[0.00 B/s] Avg:[0.00 B/s] Elapsed:0:00:10 Bytes: 0.00 B /usr/local/bin/run-upgrades.sh: waiting for server to become available...

joiner: => Rate:[1.84MiB/s] Avg:[1.84MiB/s] Elapsed:0:00:12 Bytes: 22.9MiB

2020-08-27 21:37:04 0 [Note] WSREP: 0.0 (ef3561e5ac3e): State transfer to 2.0 (fbb69ae535b4) complete.

2020-08-27 21:37:04 0 [Note] WSREP: Member 0.0 (ef3561e5ac3e) synced with group.

WSREP_SST: [INFO] Preparing the backup at /var/lib/mysql//.sst (20200827 21:37:05.029)

WSREP_SST: [INFO] Evaluating /usr//bin/mariabackup --innobackupex $INNOEXTRA --apply-log $rebuildcmd ${DATA} &> /var/lib/mysql//mariabackup.prepare.log (20200827 21:37:05.031)

WSREP_SST: [INFO] Moving the backup to /var/lib/mysql/ (20200827 21:37:06.641)

WSREP_SST: [INFO] Evaluating /usr//bin/mariabackup --innobackupex --move-back --force-non-empty-directories ${DATA} &> /var/lib/mysql//mariabackup.move.log (20200827 21:37:06.643)

WSREP_SST: [INFO] Move successful, removing /var/lib/mysql//.sst (20200827 21:37:06.659)

WSREP_SST: [INFO] Galera co-ords from recovery: 00663b45-e8ad-11ea-b0cb-778ed96c8853:5 (20200827 21:37:06.665)

WSREP_SST: [INFO] Total time on joiner: 0 seconds (20200827 21:37:06.669)

WSREP_SST: [INFO] Removing the sst_in_progress file (20200827 21:37:06.672)

2020-08-27 21:37:06 3 [Note] WSREP: SST received

2020-08-27 21:37:06 3 [Note] WSREP: Server status change joiner -> initializing

2020-08-27 21:37:06 3 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.

2020-08-27 21:37:06 0 [Note] InnoDB: Using Linux native AIO

2020-08-27 21:37:06 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins

2020-08-27 21:37:06 0 [Note] InnoDB: Uses event mutexes

2020-08-27 21:37:06 0 [Note] InnoDB: Compressed tables use zlib 1.2.11

2020-08-27 21:37:06 0 [Note] InnoDB: Number of pools: 1

2020-08-27 21:37:06 0 [Note] InnoDB: Using SSE2 crc32 instructions

2020-08-27 21:37:06 0 [Note] mysqld: O_TMPFILE is not supported on /tmp (disabling future attempts)

2020-08-27 21:37:06 0 [Note] InnoDB: Initializing buffer pool, total size = 256M, instances = 1, chunk size = 128M

2020-08-27 21:37:06 0 [Note] InnoDB: Completed initialization of buffer pool

2020-08-27 21:37:06 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().

2020-08-27 21:37:06 0 [Note] InnoDB: Setting log file ./ib_logfile101 size to 50331648 bytes

2020-08-27 21:37:06 0 [Note] InnoDB: Setting log file ./ib_logfile1 size to 50331648 bytes

2020-08-27 21:37:06 0 [Note] InnoDB: Renaming log file ./ib_logfile101 to ./ib_logfile0

2020-08-27 21:37:06 0 [Note] InnoDB: New log files created, LSN=19902733

2020-08-27 21:37:06 0 [Note] InnoDB: 128 out of 128 rollback segments are active.

2020-08-27 21:37:06 0 [Note] InnoDB: Creating shared tablespace for temporary tables

2020-08-27 21:37:06 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...

2020-08-27 21:37:06 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.

2020-08-27 21:37:06 0 [Note] InnoDB: Waiting for purge to start

2020-08-27 21:37:06 0 [Note] InnoDB: 10.4.11 started; log sequence number 19902988; transaction id 14668

2020-08-27 21:37:06 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool

2020-08-27 21:37:06 0 [Note] Plugin 'FEEDBACK' is disabled.

2020-08-27 21:37:06 0 [Note] Server socket created on IP: '0.0.0.0'.

2020-08-27 21:37:06 0 [Warning] 'user' entry '@5d913d5611dc' ignored in --skip-name-resolve mode.

2020-08-27 21:37:06 0 [Warning] 'proxies_priv' entry '@% root@5d913d5611dc' ignored in --skip-name-resolve mode.

2020-08-27 21:37:06 0 [Note] InnoDB: Buffer pool(s) load completed at 200827 21:37:06

2020-08-27 21:37:06 0 [Note] WSREP: wsrep_init_schema_and_SR (nil)

2020-08-27 21:37:06 0 [Note] WSREP: Server initialized

2020-08-27 21:37:06 0 [Note] WSREP: Server status change initializing -> initialized

2020-08-27 21:37:06 0 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.

2020-08-27 21:37:06 3 [Note] WSREP: Server status change initialized -> joined

2020-08-27 21:37:06 3 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.

2020-08-27 21:37:06 3 [Note] WSREP: Recovered position from storage: 00663b45-e8ad-11ea-b0cb-778ed96c8853:5

2020-08-27 21:37:06 3 [Note] WSREP: Recovered view from SST:

id: 00663b45-e8ad-11ea-b0cb-778ed96c8853:5

status: primary

protocol_version: 4

capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO

final: no

own_index: 2

members(3):

0: 1ebf9e8e-e8ad-11ea-b2ba-1ff1f9609aa3, ef3561e5ac3e

1: 21a824a1-e8ad-11ea-b19c-c2321b5235ce, 84b94bb55f55

2: 6ad6fded-e8ad-11ea-9493-e7e8fe31037a, fbb69ae535b4

2020-08-27 21:37:06 3 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.

2020-08-27 21:37:06 10 [Note] WSREP: Recovered cluster id 00663b45-e8ad-11ea-b0cb-778ed96c8853

2020-08-27 21:37:06 3 [Note] WSREP: SST received: 00663b45-e8ad-11ea-b0cb-778ed96c8853:5

2020-08-27 21:37:06 2 [Note] WSREP: Installed new state from SST: 00663b45-e8ad-11ea-b0cb-778ed96c8853:5

2020-08-27 21:37:06 0 [Note] Reading of all Master_info entries succeeded

2020-08-27 21:37:06 0 [Note] Added new Master_info '' to hash table

2020-08-27 21:37:06 0 [Note] mysqld: ready for connections.

Version: '10.4.11-MariaDB-1:10.4.11+maria~bionic' socket: '/var/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution

2020-08-27 21:37:06 2 [Note] WSREP: Cert. index preload up to 5

2020-08-27 21:37:06 0 [Note] WSREP: ####### IST applying starts with 6

2020-08-27 21:37:06 0 [Note] WSREP: ####### IST current seqno initialized to 5

2020-08-27 21:37:06 0 [Note] WSREP: Receiving IST... 0.0% (0/1 events) complete.

2020-08-27 21:37:06 0 [Note] WSREP: IST preload starting at 5

2020-08-27 21:37:06 0 [Note] WSREP: REPL Protocols: 10 (5, 3)

2020-08-27 21:37:06 0 [Note] WSREP: Service thread queue flushed.

2020-08-27 21:37:06 0 [Note] WSREP: ####### Assign initial position for certification: 00663b45-e8ad-11ea-b0cb-778ed96c8853:4, protocol version: 5

2020-08-27 21:37:06 0 [Note] WSREP: REPL Protocols: 10 (5, 3)

2020-08-27 21:37:06 0 [Note] WSREP: ####### Adjusting cert position: 4 -> 5

2020-08-27 21:37:06 0 [Note] WSREP: Service thread queue flushed.

2020-08-27 21:37:06 0 [Note] WSREP: Lowest cert indnex boundary for CC from preload: 5

2020-08-27 21:37:06 0 [Note] WSREP: Min available from gcache for CC from preload: 5

2020-08-27 21:37:06 0 [Note] WSREP: Receiving IST...100.0% (1/1 events) complete.

2020-08-27 21:37:06 2 [Note] WSREP: IST received: 00663b45-e8ad-11ea-b0cb-778ed96c8853:5

2020-08-27 21:37:06 2 [Note] WSREP: Lowest cert indnex boundary for CC from group: 5

2020-08-27 21:37:06 2 [Note] WSREP: Min available from gcache for CC from group: 5

2020-08-27 21:37:06 0 [Note] WSREP: 2.0 (fbb69ae535b4): State transfer from 0.0 (ef3561e5ac3e) complete.

2020-08-27 21:37:06 0 [Note] WSREP: Shifting JOINER -> JOINED (TO: 5)

2020-08-27 21:37:06 0 [Note] WSREP: Member 2.0 (fbb69ae535b4) synced with group.

2020-08-27 21:37:06 0 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 5)

2020-08-27 21:37:06 2 [Note] WSREP: Server fbb69ae535b4 synced with group

2020-08-27 21:37:06 2 [Note] WSREP: Server status change joined -> synced

2020-08-27 21:37:06 2 [Note] WSREP: Synchronized with group, ready for connections

2020-08-27 21:37:06 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.

2020-08-27 21:47:12 14 [Warning] Aborted connection 14 to db: 'unconnected' user: 'system' host: '127.0.0.1' (Got timeout reading communication packets)`

The problem still persists. I've realised that with this error the data is not propagated to other nodes. The loaded database only persist on one node.

95 - seems the same problem (also from me), maybe it's issue duplicationn btw.

csib commented 3 years ago

More info:

Overview after my existing database is restored into the cluster: image

Refresh the page and check again: image

As you see the Rows are in the tables are changed. The schema is exists on all cluster member but the data itself is not.

csib commented 3 years ago

Ohh, got it. The replication is not working for MyISAM tables, only for InnoDB.

After change every table to InnoDB the problem disappeared.

Is it normal that MyISAM replication is not working?

colinmollenhour commented 3 years ago

Yes, Galera does not support MyISAM in any way.