setup three galera nodes with "systemctl start mariadb", one of them restarts with every 5 seconds, with loop

[indicolite]

I saw a bug with below link: https://bugzilla.redhat.com/show_bug.cgi?id=1335849 but I cannot reproduce it normally, Has anyone encounter the same issue? I cannot see any useful info in mariadb.log.

Thanks.

[indicolite]

part of the log:

WSREP_SST: [INFO] Waiting for SST streaming to complete! (20170505 19:28:47.379) 2017-05-05 19:28:49 139670121867008 [Note] WSREP: (008fd450, 'tcp://0.0.0.0:4567') turning message relay requesting off xbstream: Can't create/write to file './ibdata1' (Errcode: 17 - File exists) xbstream: failed to create file. 2017/05/05 19:28:57 socat[60374] E write(1, 0x2283340, 8192): Broken pipe WSREP_SST: [ERROR] Error while getting data from donor node: exit codes: 1 1 (20170505 19:28:57.756) WSREP_SST: [ERROR] Cleanup after exit with status:32 (20170505 19:28:57.758) 2017-05-05 19:28:57 139670105081600 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup-v2 --role 'joiner' --address '10.127.2.63' --datadir '/var/lib/mysql/' --parent '60079' '' : 32 (Broken pipe) 2017-05-05 19:28:57 139670105081600 [ERROR] WSREP: Failed to read uuid:seqno and wsrep_gtid_domain_id from joiner script. 2017-05-05 19:28:57 139687499147392 [ERROR] WSREP: SST failed: 32 (Broken pipe) 2017-05-05 19:28:57 139687499147392 [ERROR] Aborting

2017-05-05 19:28:57 139670113474304 [Warning] WSREP: 1.0 (test-compute1): State transfer to 0.0 (test-neutron1) failed: -22 (Invalid argument) 2017-05-05 19:28:57 139670113474304 [ERROR] WSREP: gcs/src/gcs_group.cpp:gcs_group_handle_join_msg():731: Will never receive state. Need to abort. 2017-05-05 19:29:03 140353217464448 [Note] WSREP: Read nil XID from storage engines, skipping position init 2017-05-05 19:29:03 140353217464448 [Note] WSREP: wsrep_load(): loading provider library 'none' 2017-05-05 19:29:03 7fa687ab8880 InnoDB: Warning: Using innodb_locks_unsafe_for_binlog is DEPRECATED. This option may be removed in future releases. Please use READ COMMITTED transaction isolation level instead, see http://dev.mysql.com/doc/refman/5.6/en/set-transaction.html. 2017-05-05 19:29:03 140353217464448 [Note] InnoDB: Using mutexes to ref count buffer pool pages 2017-05-05 19:29:03 140353217464448 [Note] InnoDB: The InnoDB memory heap is disabled 2017-05-05 19:29:03 140353217464448 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2017-05-05 19:29:03 140353217464448 [Note] InnoDB: Memory barrier is not used 2017-05-05 19:29:03 140353217464448 [Note] InnoDB: Compressed tables use zlib 1.2.7 2017-05-05 19:29:03 140353217464448 [Note] InnoDB: Using Linux native AIO 2017-05-05 19:29:03 140353217464448 [Note] InnoDB: Using SSE crc32 instructions 2017-05-05 19:29:03 140353217464448 [Note] InnoDB: Initializing buffer pool, size = 30.0G 2017-05-05 19:29:05 140353217464448 [Note] InnoDB: Completed initialization of buffer pool 2017-05-05 19:29:07 140353217464448 [Note] InnoDB: The first specified data file ./ibdata1 did not exist: a new database to be created! 2017-05-05 19:29:07 140353217464448 [Note] InnoDB: Setting file ./ibdata1 size to 12 MB 2017-05-05 19:29:07 140353217464448 [Note] InnoDB: Database physically writes the file full: wait... 2017-05-05 19:29:07 140353217464448 [Note] InnoDB: Setting log file ./ib_logfile101 size to 1 MB 2017-05-05 19:29:07 140353217464448 [Note] InnoDB: Setting log file ./ib_logfile1 size to 1 MB 2017-05-05 19:29:07 140353217464448 [Note] InnoDB: Renaming log file ./ib_logfile101 to ./ib_logfile0 2017-05-05 19:29:07 140353217464448 [Warning] InnoDB: New log files created, LSN=45883 2017-05-05 19:29:07 140353217464448 [Note] InnoDB: Doublewrite buffer not found: creating new 2017-05-05 19:29:07 140353217464448 [Note] InnoDB: Doublewrite buffer created 2017-05-05 19:29:07 140353217464448 [Note] InnoDB: 128 rollback segment(s) are active. 2017-05-05 19:29:07 140353217464448 [Warning] InnoDB: Creating foreign key constraint system tables. 2017-05-05 19:29:07 140353217464448 [Note] InnoDB: Foreign key constraint system tables created 2017-05-05 19:29:07 140353217464448 [Note] InnoDB: Creating tablespace and datafile system tables. 2017-05-05 19:29:07 140353217464448 [Note] InnoDB: Tablespace and datafile system tables created. 2017-05-05 19:29:07 140353217464448 [Note] InnoDB: Waiting for purge to start 2017-05-05 19:29:07 140353217464448 [Note] InnoDB: Percona XtraDB (http://www.percona.com) 5.6.28-76.1 started; log sequence number 0 2017-05-05 19:29:07 140312613742336 [Note] InnoDB: Dumping buffer pool(s) not yet started 2017-05-05 19:29:12 140070750480512 [Note] WSREP: Read nil XID from storage engines, skipping position init 2017-05-05 19:29:12 140070750480512 [Note] WSREP: wsrep_load(): loading provider library 'none' 2017-05-05 19:29:12 7f64c3536880 InnoDB: Warning: Using innodb_locks_unsafe_for_binlog is DEPRECATED. This option may be removed in future releases. Please use READ COMMITTED transaction isolation level instead, see http://dev.mysql.com/doc/refman/5.6/en/set-transaction.html. 2017-05-05 19:29:12 140070750480512 [Note] InnoDB: Using mutexes to ref count buffer pool pages 2017-05-05 19:29:12 140070750480512 [Note] InnoDB: The InnoDB memory heap is disabled 2017-05-05 19:29:12 140070750480512 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2017-05-05 19:29:12 140070750480512 [Note] InnoDB: Memory barrier is not used 2017-05-05 19:29:12 140070750480512 [Note] InnoDB: Compressed tables use zlib 1.2.7 2017-05-05 19:29:12 140070750480512 [Note] InnoDB: Using Linux native AIO 2017-05-05 19:29:12 140070750480512 [Note] InnoDB: Using SSE crc32 instructions 2017-05-05 19:29:12 140070750480512 [Note] InnoDB: Initializing buffer pool, size = 30.0G 2017-05-05 19:29:14 140070750480512 [Note] InnoDB: Completed initialization of buffer pool 2017-05-05 19:29:16 140070750480512 [Note] InnoDB: Highest supported file format is Barracuda. 2017-05-05 19:29:16 140070750480512 [Note] InnoDB: 128 rollback segment(s) are active. 2017-05-05 19:29:16 140070750480512 [Note] InnoDB: Waiting for purge to start 2017-05-05 19:29:16 140070750480512 [Note] InnoDB: Percona XtraDB (http://www.percona.com) 5.6.28-76.1 started; log sequence number 1616799 2017-05-05 19:29:16 140030790051584 [Note] InnoDB: Dumping buffer pool(s) not yet started 2017-05-05 19:29:21 140521278163072 [Note] WSREP: Read nil XID from storage engines, skipping position init 2017-05-05 19:29:21 140521278163072 [Note] WSREP: wsrep_load(): loading provider library 'none' 2017-05-05 19:29:21 7fcda8de4880 InnoDB: Warning: Using innodb_locks_unsafe_for_binlog is DEPRECATED. This option may be removed in future releases. Please use READ COMMITTED transaction isolation level instead, see http://dev.mysql.com/doc/refman/5.6/en/set-transaction.html. 2017-05-05 19:29:21 140521278163072 [Note] InnoDB: Using mutexes to ref count buffer pool pages 2017-05-05 19:29:21 140521278163072 [Note] InnoDB: The InnoDB memory heap is disabled 2017-05-05 19:29:21 140521278163072 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2017-05-05 19:29:21 140521278163072 [Note] InnoDB: Memory barrier is not used 2017-05-05 19:29:21 140521278163072 [Note] InnoDB: Compressed tables use zlib 1.2.7 2017-05-05 19:29:21 140521278163072 [Note] InnoDB: Using Linux native AIO 2017-05-05 19:29:21 140521278163072 [Note] InnoDB: Using SSE crc32 instructions 2017-05-05 19:29:21 140521278163072 [Note] InnoDB: Initializing buffer pool, size = 30.0G 2017-05-05 19:29:23 140521278163072 [Note] InnoDB: Completed initialization of buffer pool 2017-05-05 19:29:25 140521278163072 [Note] InnoDB: Highest supported file format is Barracuda. 2017-05-05 19:29:25 140521278163072 [Note] InnoDB: 128 rollback segment(s) are active. 2017-05-05 19:29:25 140521278163072 [Note] InnoDB: Waiting for purge to start 2017-05-05 19:29:26 140521278163072 [Note] InnoDB: Percona XtraDB (http://www.percona.com) 5.6.28-76.1 started; log sequence number 1616809 2017-05-05 19:29:26 140481308641024 [Note] InnoDB: Dumping buffer pool(s) not yet started 2017-05-05 19:29:30 140337064814720 [Note] WSREP: Read nil XID from storage engines, skipping position init 2017-05-05 19:29:30 140337064814720 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib64/galera/libgalera_smm.so' 2017-05-05 19:29:30 140337064814720 [Note] WSREP: wsrep_load(): Galera 3.12(r9921e73) by Codership Oy info@codership.com loaded successfully. 2017-05-05 19:29:30 140337064814720 [Note] WSREP: CRC-32C: using hardware acceleration. 2017-05-05 19:29:30 140337064814720 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1 2017-05-05 19:29:30 140337064814720 [Note] WSREP: Passing config to GCS: base_dir = /var/lib/mysql/; base_host = 10.127.2.63; base_port = 4567; cert.log_conflicts = YES; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 16G; gcache.size = 16G; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 1024; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.checksum = false; pc.ignore_quorum = false; pc.ignore_sb = false; 2017-05-05 19:29:30 140336972293888 [Note] WSREP: Service thread queue flushed. 2017-05-05 19:29:30 140337064814720 [Note] WSREP: Assign initial position for certification: -1, protocol version: -1 2017-05-05 19:29:30 140337064814720 [Note] WSREP: wsrep_sst_grab() 2017-05-05 19:29:30 140337064814720 [Note] WSREP: Start replication 2017-05-05 19:29:30 140337064814720 [Note] WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:-1 2017-05-05 19:29:30 140337064814720 [Note] WSREP: protonet asio version 0 2017-05-05 19:29:30 140337064814720 [Note] WSREP: Using CRC-32C for message checksums. 2017-05-05 19:29:30 140337064814720 [Note] WSREP: backend: asio 2017-05-05 19:29:30 140337064814720 [Warning] WSREP: access file(/var/lib/mysql//gvwstate.dat) failed(No such file or directory) 2017-05-05 19:29:30 140337064814720 [Note] WSREP: restore pc from disk failed 2017-05-05 19:29:30 140337064814720 [Note] WSREP: GMCast version 0 2017-05-05 19:29:30 140337064814720 [Note] WSREP: (1b0b4c5d, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567 2017-05-05 19:29:30 140337064814720 [Note] WSREP: (1b0b4c5d, 'tcp://0.0.0.0:4567') multicast: , ttl: 1 2017-05-05 19:29:30 140337064814720 [Note] WSREP: EVS version 0 2017-05-05 19:29:30 140337064814720 [Note] WSREP: gcomm: connecting to group 'fh_wsrep_cluster', peer '10.127.2.60:,10.127.2.65:' 2017-05-05 19:29:30 140337064814720 [Note] WSREP: (1b0b4c5d, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers: 2017-05-05 19:29:30 140337064814720 [Note] WSREP: declaring 02d46a25 at tcp://10.127.2.65:4567 stable 2017-05-05 19:29:30 140337064814720 [Note] WSREP: declaring 1a7de35a at tcp://10.127.2.60:4567 stable 2017-05-05 19:29:30 140337064814720 [Note] WSREP: Node 02d46a25 state prim 2017-05-05 19:29:30 140337064814720 [Note] WSREP: view(view_id(PRIM,02d46a25,398) memb { 02d46a25,0 1a7de35a,0 1b0b4c5d,0 } joined { } left { } partitioned { }) 2017-05-05 19:29:30 140337064814720 [Note] WSREP: save pc into disk 2017-05-05 19:29:31 140337064814720 [Note] WSREP: gcomm: connected 2017-05-05 19:29:31 140337064814720 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636 2017-05-05 19:29:31 140337064814720 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0) 2017-05-05 19:29:31 140337064814720 [Note] WSREP: Opened channel 'fh_wsrep_cluster' 2017-05-05 19:29:31 140336930330368 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 2, memb_num = 3 2017-05-05 19:29:31 140336930330368 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID. 2017-05-05 19:29:31 140336930330368 [Note] WSREP: STATE EXCHANGE: sent state msg: 1b180211-3186-11e7-94db-f2719815e9ea 2017-05-05 19:29:31 140336930330368 [Note] WSREP: STATE EXCHANGE: got state msg: 1b180211-3186-11e7-94db-f2719815e9ea from 0 (test-compute1) 2017-05-05 19:29:31 140336930330368 [Note] WSREP: STATE EXCHANGE: got state msg: 1b180211-3186-11e7-94db-f2719815e9ea from 1 (test-controller1) 2017-05-05 19:29:31 140337064814720 [Note] WSREP: Waiting for SST to complete. 2017-05-05 19:29:31 140336930330368 [Note] WSREP: STATE EXCHANGE: got state msg: 1b180211-3186-11e7-94db-f2719815e9ea from 2 (test-neutron1) 2017-05-05 19:29:31 140336930330368 [Note] WSREP: Quorum results: version = 3, component = PRIMARY, conf_id = 396, members = 2/3 (joined/total), act_id = 28221, last_appl. = -1, protocols = 0/7/3 (gcs/repl/appl), group UUID = f9b78904-3171-11e7-86f5-03aad28c2a6a 2017-05-05 19:29:31 140336930330368 [Note] WSREP: Flow-control interval: [1774, 1774] 2017-05-05 19:29:31 140336930330368 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 28221) 2017-05-05 19:29:31 140337063312128 [Note] WSREP: State transfer required: Group state: f9b78904-3171-11e7-86f5-03aad28c2a6a:28221 Local state: 00000000-0000-0000-0000-000000000000:-1 2017-05-05 19:29:31 140337063312128 [Note] WSREP: New cluster view: global state: f9b78904-3171-11e7-86f5-03aad28c2a6a:28221, view# 397: Primary, number of nodes: 3, my index: 2, protocol version 3 2017-05-05 19:29:31 140337063312128 [Warning] WSREP: Gap in state sequence. Need state transfer. 2017-05-05 19:29:31 140319668561664 [Note] WSREP: Running: 'wsrep_sst_xtrabackup-v2 --role 'joiner' --address '10.127.2.63' --datadir '/var/lib/mysql/' --parent '65402' '' ' WSREP_SST: [INFO] Streaming with xbstream (20170505 19:29:31.357) WSREP_SST: [INFO] Using socat as streamer (20170505 19:29:31.359) WSREP_SST: [INFO] Stale sst_in_progress file: /var/lib/mysql//sst_in_progress (20170505 19:29:31.362) WSREP_SST: [INFO] Evaluating timeout -k 110 100 socat -u TCP-LISTEN:4444,reuseaddr stdio | xbstream -x; RC=( ${PIPESTATUS[@]} ) (20170505 19:29:31.385) 2017-05-05 19:29:31 140337063312128 [Note] WSREP: Prepared SST request: xtrabackup-v2|10.127.2.63:4444/xtrabackup_sst//1 2017-05-05 19:29:31 140337063312128 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2017-05-05 19:29:31 140337063312128 [Note] WSREP: REPL Protocols: 7 (3, 2) 2017-05-05 19:29:31 140336972293888 [Note] WSREP: Service thread queue flushed. 2017-05-05 19:29:31 140337063312128 [Note] WSREP: Assign initial position for certification: 28221, protocol version: 3 2017-05-05 19:29:31 140336972293888 [Note] WSREP: Service thread queue flushed. 2017-05-05 19:29:31 140337063312128 [Warning] WSREP: Failed to prepare for incremental state transfer: Local state UUID (00000000-0000-0000-0000-000000000000) does not match group state UUID (f9b78904-3171-11e7-86f5-03aad28c2a6a): 1 (Operation not permitted) at galera/src/replicator_str.cpp:prepare_for_IST():482. IST will be unavailable. 2017-05-05 19:29:31 140336930330368 [Note] WSREP: Member 2.0 (test-neutron1) requested state transfer from 'any'. Selected 0.0 (test-compute1)(SYNCED) as donor. 2017-05-05 19:29:31 140336930330368 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 28221) 2017-05-05 19:29:31 140337063312128 [Note] WSREP: Requesting state transfer: success, donor: 0 WSREP_SST: [INFO] WARNING: Stale temporary SST directory: /var/lib/mysql//.sst from previous state transfer (20170505 19:29:31.794) WSREP_SST: [INFO] Proceeding with SST (20170505 19:29:31.797) WSREP_SST: [INFO] Evaluating socat -u TCP-LISTEN:4444,reuseaddr stdio | xbstream -x; RC=( ${PIPESTATUS[@]} ) (20170505 19:29:31.798) WSREP_SST: [INFO] Cleaning the existing datadir and innodb-data/log directories (20170505 19:29:31.799) removed ‘/var/lib/mysql/mysql_upgrade_info’ removed ‘/var/lib/mysql/ib_logfile0’ removed ‘/var/lib/mysql/aria_log.00000001’ removed ‘/var/lib/mysql/performance_schema/db.opt’ removed directory: ‘/var/lib/mysql/performance_schema’ removed ‘/var/lib/mysql/mysql/innodb_table_stats.frm’ removed ‘/var/lib/mysql/mysql/help_relation.MYD’ removed ‘/var/lib/mysql/mysql/proc.MYI’ removed ‘/var/lib/mysql/mysql/columns_priv.MYD’ removed ‘/var/lib/mysql/mysql/index_stats.frm’ removed ‘/var/lib/mysql/mysql/index_stats.MYI’ removed ‘/var/lib/mysql/mysql/procs_priv.MYI’ removed ‘/var/lib/mysql/mysql/column_stats.MYI’ removed ‘/var/lib/mysql/mysql/db.MYD’ removed ‘/var/lib/mysql/mysql/table_stats.frm’ removed ‘/var/lib/mysql/mysql/host.MYD’ removed ‘/var/lib/mysql/mysql/help_topic.MYD’ removed ‘/var/lib/mysql/mysql/proxies_priv.MYD’ removed ‘/var/lib/mysql/mysql/proxies_priv.frm’ removed ‘/var/lib/mysql/mysql/time_zone_transition.MYI’ removed ‘/var/lib/mysql/mysql/time_zone_leap_second.MYD’ removed ‘/var/lib/mysql/mysql/host.frm’ removed ‘/var/lib/mysql/mysql/columns_priv.MYI’ removed ‘/var/lib/mysql/mysql/servers.frm’ removed ‘/var/lib/mysql/mysql/func.frm’ removed ‘/var/lib/mysql/mysql/time_zone_transition_type.MYD’ removed ‘/var/lib/mysql/mysql/func.MYI’ removed ‘/var/lib/mysql/mysql/plugin.MYD’ removed ‘/var/lib/mysql/mysql/column_stats.MYD’ removed ‘/var/lib/mysql/mysql/time_zone_transition_type.frm’ removed ‘/var/lib/mysql/mysql/table_stats.MYD’ removed ‘/var/lib/mysql/mysql/time_zone.frm’ removed ‘/var/lib/mysql/mysql/general_log.CSV’ removed ‘/var/lib/mysql/mysql/help_topic.MYI’ removed ‘/var/lib/mysql/mysql/time_zone_name.MYI’ removed ‘/var/lib/mysql/mysql/general_log.CSM’ removed ‘/var/lib/mysql/mysql/time_zone_leap_second.frm’ removed ‘/var/lib/mysql/mysql/user.MYI’ removed ‘/var/lib/mysql/mysql/time_zone.MYD’ removed ‘/var/lib/mysql/mysql/gtid_slave_pos.frm’ removed ‘/var/lib/mysql/mysql/tables_priv.MYI’ removed ‘/var/lib/mysql/mysql/proc.frm’ removed ‘/var/lib/mysql/mysql/event.MYD’ removed ‘/var/lib/mysql/mysql/servers.MYI’ removed ‘/var/lib/mysql/mysql/slow_log.CSV’ removed ‘/var/lib/mysql/mysql/host.MYI’ removed ‘/var/lib/mysql/mysql/innodb_table_stats.ibd’ removed ‘/var/lib/mysql/mysql/help_category.MYI’ removed ‘/var/lib/mysql/mysql/column_stats.frm’ removed ‘/var/lib/mysql/mysql/slow_log.CSM’ removed ‘/var/lib/mysql/mysql/innodb_index_stats.frm’ removed ‘/var/lib/mysql/mysql/index_stats.MYD’ removed ‘/var/lib/mysql/mysql/tables_priv.frm’ removed ‘/var/lib/mysql/mysql/help_keyword.MYD’ removed ‘/var/lib/mysql/mysql/user.MYD’ removed ‘/var/lib/mysql/mysql/roles_mapping.frm’ removed ‘/var/lib/mysql/mysql/roles_mapping.MYD’ removed ‘/var/lib/mysql/mysql/time_zone_transition.MYD’ removed ‘/var/lib/mysql/mysql/help_relation.frm’ removed ‘/var/lib/mysql/mysql/tables_priv.MYD’ removed ‘/var/lib/mysql/mysql/general_log.frm’ removed ‘/var/lib/mysql/mysql/help_keyword.frm’ removed ‘/var/lib/mysql/mysql/procs_priv.frm’ removed ‘/var/lib/mysql/mysql/db.MYI’ removed ‘/var/lib/mysql/mysql/table_stats.MYI’ removed ‘/var/lib/mysql/mysql/servers.MYD’ removed ‘/var/lib/mysql/mysql/proc.MYD’ removed ‘/var/lib/mysql/mysql/time_zone_transition.frm’ removed ‘/var/lib/mysql/mysql/time_zone_transition_type.MYI’ removed ‘/var/lib/mysql/mysql/procs_priv.MYD’ removed ‘/var/lib/mysql/mysql/time_zone.MYI’ removed ‘/var/lib/mysql/mysql/help_topic.frm’ removed ‘/var/lib/mysql/mysql/help_category.MYD’ removed ‘/var/lib/mysql/mysql/db.frm’ removed ‘/var/lib/mysql/mysql/time_zone_name.MYD’ removed ‘/var/lib/mysql/mysql/time_zone_name.frm’ removed ‘/var/lib/mysql/mysql/help_category.frm’ removed ‘/var/lib/mysql/mysql/proxies_priv.MYI’ removed ‘/var/lib/mysql/mysql/user.frm’ removed ‘/var/lib/mysql/mysql/func.MYD’ removed ‘/var/lib/mysql/mysql/help_relation.MYI’ removed ‘/var/lib/mysql/mysql/roles_mapping.MYI’ removed ‘/var/lib/mysql/mysql/plugin.frm’ removed ‘/var/lib/mysql/mysql/gtid_slave_pos.ibd’ removed ‘/var/lib/mysql/mysql/event.frm’ removed ‘/var/lib/mysql/mysql/event.MYI’ removed ‘/var/lib/mysql/mysql/columns_priv.frm’ removed ‘/var/lib/mysql/mysql/help_keyword.MYI’ removed ‘/var/lib/mysql/mysql/innodb_index_stats.ibd’ removed ‘/var/lib/mysql/mysql/plugin.MYI’ removed ‘/var/lib/mysql/mysql/slow_log.frm’ removed ‘/var/lib/mysql/mysql/time_zone_leap_second.MYI’ removed directory: ‘/var/lib/mysql/mysql’ removed directory: ‘/var/lib/mysql/test’ removed ‘/var/lib/mysql/aria_log_control’ removed ‘/var/lib/mysql/ibdata1’ removed ‘/var/lib/mysql/ib_logfile1’ WSREP_SST: [INFO] Waiting for SST streaming to complete! (20170505 19:29:31.811) 2017-05-05 19:29:34 140336938723072 [Note] WSREP: (1b0b4c5d, 'tcp://0.0.0.0:4567') turning message relay requesting off xbstream: Can't create/write to file './ibdata1' (Errcode: 17 - File exists) xbstream: failed to create file. 2017/05/05 19:29:42 socat[464] E write(1, 0xe5d340, 8192): Broken pipe WSREP_SST: [ERROR] Error while getting data from donor node: exit codes: 1 1 (20170505 19:29:42.190) WSREP_SST: [ERROR] Cleanup after exit with status:32 (20170505 19:29:42.191) 2017-05-05 19:29:42 140319668561664 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup-v2 --role 'joiner' --address '10.127.2.63' --datadir '/var/lib/mysql/' --parent '65402' '' : 32 (Broken pipe) 2017-05-05 19:29:42 140319668561664 [ERROR] WSREP: Failed to read uuid:seqno and wsrep_gtid_domain_id from joiner script. 2017-05-05 19:29:42 140337064814720 [ERROR] WSREP: SST failed: 32 (Broken pipe) 2017-05-05 19:29:42 140337064814720 [ERROR] Aborting

2017-05-05 19:29:42 140336930330368 [Warning] WSREP: 0.0 (test-compute1): State transfer to 2.0 (test-neutron1) failed: -22 (Invalid argument) 2017-05-05 19:29:42 140336930330368 [ERROR] WSREP: gcs/src/gcs_group.cpp:gcs_group_handle_join_msg():731: Will never receive state. Need to abort. 2017-05-05 19:29:45 140583762925696 [Note] WSREP: Read nil XID from storage engines, skipping position init 2017-05-05 19:29:45 140583762925696 [Note] WSREP: wsrep_load(): loading provider library 'none' 2017-05-05 19:29:45 7fdc35401880 InnoDB: Warning: Using innodb_locks_unsafe_for_binlog is DEPRECATED. This option may be removed in future releases. Please use READ COMMITTED transaction isolation level instead, see http://dev.mysql.com/doc/refman/5.6/en/set-transaction.html. 2017-05-05 19:29:45 140583762925696 [Note] InnoDB: Using mutexes to ref count buffer pool pages 2017-05-05 19:29:45 140583762925696 [Note] InnoDB: The InnoDB memory heap is disabled 2017-05-05 19:29:45 140583762925696 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2017-05-05 19:29:45 140583762925696 [Note] InnoDB: Memory barrier is not used 2017-05-05 19:29:45 140583762925696 [Note] InnoDB: Compressed tables use zlib 1.2.7 2017-05-05 19:29:45 140583762925696 [Note] InnoDB: Using Linux native AIO 2017-05-05 19:29:45 140583762925696 [Note] InnoDB: Using SSE crc32 instructions 2017-05-05 19:29:45 140583762925696 [Note] InnoDB: Initializing buffer pool, size = 30.0G 2017-05-05 19:29:47 140583762925696 [Note] InnoDB: Completed initialization of buffer pool 2017-05-05 19:29:49 140583762925696 [Note] InnoDB: The first specified data file ./ibdata1 did not exist: a new database to be created! 2017-05-05 19:29:49 140583762925696 [Note] InnoDB: Setting file ./ibdata1 size to 12 MB 2017-05-05 19:29:49 140583762925696 [Note] InnoDB: Database physically writes the file full: wait... 2017-05-05 19:29:49 140583762925696 [Note] InnoDB: Setting log file ./ib_logfile101 size to 1 MB 2017-05-05 19:29:49 140583762925696 [Note] InnoDB: Setting log file ./ib_logfile1 size to 1 MB 2017-05-05 19:29:49 140583762925696 [Note] InnoDB: Renaming log file ./ib_logfile101 to ./ib_logfile0 2017-05-05 19:29:49 140583762925696 [Warning] InnoDB: New log files created, LSN=45883 2017-05-05 19:29:49 140583762925696 [Note] InnoDB: Doublewrite buffer not found: creating new 2017-05-05 19:29:49 140583762925696 [Note] InnoDB: Doublewrite buffer created 2017-05-05 19:29:49 140583762925696 [Note] InnoDB: 128 rollback segment(s) are active. 2017-05-05 19:29:49 140583762925696 [Warning] InnoDB: Creating foreign key constraint system tables. 2017-05-05 19:29:49 140583762925696 [Note] InnoDB: Foreign key constraint system tables created 2017-05-05 19:29:49 140583762925696 [Note] InnoDB: Creating tablespace and datafile system tables. 2017-05-05 19:29:49 140583762925696 [Note] InnoDB: Tablespace and datafile system tables created. 2017-05-05 19:29:49 140583762925696 [Note] InnoDB: Waiting for purge to start 2017-05-05 19:29:49 140583762925696 [Note] InnoDB: Percona XtraDB (http://www.percona.com) 5.6.28-76.1 started; log sequence number 0 2017-05-05 19:29:49 140543229888256 [Note] InnoDB: Dumping buffer pool(s) not yet started 2017-05-05 19:29:54 139970383452288 [Note] WSREP: Read nil XID from storage engines, skipping position init 2017-05-05 19:29:54 139970383452288 [Note] WSREP: wsrep_load(): loading provider library 'none' 2017-05-05 19:29:54 7f4d64fc1880 InnoDB: Warning: Using innodb_locks_unsafe_for_binlog is DEPRECATED. This option may be removed in future releases. Please use READ COMMITTED transaction isolation level instead, see http://dev.mysql.com/doc/refman/5.6/en/set-transaction.html. 2017-05-05 19:29:54 139970383452288 [Note] InnoDB: Using mutexes to ref count buffer pool pages 2017-05-05 19:29:54 139970383452288 [Note] InnoDB: The InnoDB memory heap is disabled 2017-05-05 19:29:54 139970383452288 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2017-05-05 19:29:54 139970383452288 [Note] InnoDB: Memory barrier is not used 2017-05-05 19:29:54 139970383452288 [Note] InnoDB: Compressed tables use zlib 1.2.7 2017-05-05 19:29:54 139970383452288 [Note] InnoDB: Using Linux native AIO 2017-05-05 19:29:54 139970383452288 [Note] InnoDB: Using SSE crc32 instructions 2017-05-05 19:29:54 139970383452288 [Note] InnoDB: Initializing buffer pool, size = 30.0G 2017-05-05 19:29:56 139970383452288 [Note] InnoDB: Completed initialization of buffer pool 2017-05-05 19:29:57 139970383452288 [Note] InnoDB: Highest supported file format is Barracuda. 2017-05-05 19:29:58 139970383452288 [Note] InnoDB: 128 rollback segment(s) are active. 2017-05-05 19:29:58 139970383452288 [Note] InnoDB: Waiting for purge to start 2017-05-05 19:29:58 139970383452288 [Note] InnoDB: Percona XtraDB (http://www.percona.com) 5.6.28-76.1 started; log sequence number 1616799 2017-05-05 19:29:58 139930411976448 [Note] InnoDB: Dumping buffer pool(s) not yet started 2017-05-05 19:30:03 140376982460544 [Note] WSREP: Read nil XID from storage engines, skipping position init 2017-05-05 19:30:03 140376982460544 [Note] WSREP: wsrep_load(): loading provider library 'none' 2017-05-05 19:30:03 7fac102c9880 InnoDB: Warning: Using innodb_locks_unsafe_for_binlog is DEPRECATED. This option may be removed in future releases. Please use READ COMMITTED transaction isolation level instead, see http://dev.mysql.com/doc/refman/5.6/en/set-transaction.html. 2017-05-05 19:30:03 140376982460544 [Note] InnoDB: Using mutexes to ref count buffer pool pages 2017-05-05 19:30:03 140376982460544 [Note] InnoDB: The InnoDB memory heap is disabled 2017-05-05 19:30:03 140376982460544 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2017-05-05 19:30:03 140376982460544 [Note] InnoDB: Memory barrier is not used 2017-05-05 19:30:03 140376982460544 [Note] InnoDB: Compressed tables use zlib 1.2.7 2017-05-05 19:30:03 140376982460544 [Note] InnoDB: Using Linux native AIO 2017-05-05 19:30:03 140376982460544 [Note] InnoDB: Using SSE crc32 instructions 2017-05-05 19:30:03 140376982460544 [Note] InnoDB: Initializing buffer pool, size = 30.0G 2017-05-05 19:30:05 140376982460544 [Note] InnoDB: Completed initialization of buffer pool 2017-05-05 19:30:07 140376982460544 [Note] InnoDB: Highest supported file format is Barracuda. 2017-05-05 19:30:07 140376982460544 [Note] InnoDB: 128 rollback segment(s) are active. 2017-05-05 19:30:07 140376982460544 [Note] InnoDB: Waiting for purge to start 2017-05-05 19:30:07 140376982460544 [Note] InnoDB: Percona XtraDB (http://www.percona.com) 5.6.28-76.1 started; log sequence number 1616809 2017-05-05 19:30:07 140337016190720 [Note] InnoDB: Dumping buffer pool(s) not yet started 2017-05-05 19:30:12 140202340792448 [Note] WSREP: Read nil XID from storage engines, skipping position init 2017-05-05 19:30:12 140202340792448 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib64/galera/libgalera_smm.so' 2017-05-05 19:30:12 140202340792448 [Note] WSREP: wsrep_load(): Galera 3.12(r9921e73) by Codership Oy info@codership.com loaded successfully. 2017-05-05 19:30:12 140202340792448 [Note] WSREP: CRC-32C: using hardware acceleration. 2017-05-05 19:30:12 140202340792448 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1 2017-05-05 19:30:12 140202340792448 [Note] WSREP: Passing config to GCS: base_dir = /var/lib/mysql/; base_host = 10.127.2.63; base_port = 4567; cert.log_conflicts = YES; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 16G; gcache.size = 16G; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 1024; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.checksum = false; pc.ignore_quorum = false; pc.ignore_sb = false; 2017-05-05 19:30:12 140202248271616 [Note] WSREP: Service thread queue flushed. 2017-05-05 19:30:12 140202340792448 [Note] WSREP: Assign initial position for certification: -1, protocol version: -1 2017-05-05 19:30:12 140202340792448 [Note] WSREP: wsrep_sst_grab() 2017-05-05 19:30:12 140202340792448 [Note] WSREP: Start replication 2017-05-05 19:30:12 140202340792448 [Note] WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:-1 2017-05-05 19:30:12 140202340792448 [Note] WSREP: protonet asio version 0 2017-05-05 19:30:12 140202340792448 [Note] WSREP: Using CRC-32C for message checksums. 2017-05-05 19:30:12 140202340792448 [Note] WSREP: backend: asio 2017-05-05 19:30:12 140202340792448 [Warning] WSREP: access file(/var/lib/mysql//gvwstate.dat) failed(No such file or directory) 2017-05-05 19:30:12 140202340792448 [Note] WSREP: restore pc from disk failed 2017-05-05 19:30:12 140202340792448 [Note] WSREP: GMCast version 0 2017-05-05 19:30:12 140202340792448 [Note] WSREP: (34002e52, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567 2017-05-05 19:30:12 140202340792448 [Note] WSREP: (34002e52, 'tcp://0.0.0.0:4567') multicast: , ttl: 1 2017-05-05 19:30:12 140202340792448 [Note] WSREP: EVS version 0 2017-05-05 19:30:12 140202340792448 [Note] WSREP: gcomm: connecting to group 'fh_wsrep_cluster', peer '10.127.2.60:,10.127.2.65:' 2017-05-05 19:30:12 140202340792448 [Note] WSREP: (34002e52, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers: 2017-05-05 19:30:13 140202340792448 [Note] WSREP: declaring 02d46a25 at tcp://10.127.2.65:4567 stable 2017-05-05 19:30:13 140202340792448 [Note] WSREP: declaring 1a7de35a at tcp://10.127.2.60:4567 stable 2017-05-05 19:30:13 140202340792448 [Note] WSREP: Node 02d46a25 state prim 2017-05-05 19:30:13 140202340792448 [Note] WSREP: view(view_id(PRIM,02d46a25,400) memb { 02d46a25,0 1a7de35a,0 34002e52,0 } joined { } left { } partitioned { }) 2017-05-05 19:30:13 140202340792448 [Note] WSREP: save pc into disk 2017-05-05 19:30:13 140202340792448 [Note] WSREP: gcomm: connected 2017-05-05 19:30:13 140202340792448 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636 2017-05-05 19:30:13 140202340792448 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0) 2017-05-05 19:30:13 140202340792448 [Note] WSREP: Opened channel 'fh_wsrep_cluster' 2017-05-05 19:30:13 140184955893504 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 2, memb_num = 3 2017-05-05 19:30:13 140184955893504 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID. 2017-05-05 19:30:13 140184955893504 [Note] WSREP: STATE EXCHANGE: sent state msg: 344caedd-3186-11e7-87e9-afb9a0cdb378 2017-05-05 19:30:13 140184955893504 [Note] WSREP: STATE EXCHANGE: got state msg: 344caedd-3186-11e7-87e9-afb9a0cdb378 from 0 (test-compute1) 2017-05-05 19:30:13 140184955893504 [Note] WSREP: STATE EXCHANGE: got state msg: 344caedd-3186-11e7-87e9-afb9a0cdb378 from 1 (test-controller1) 2017-05-05 19:30:13 140202340792448 [Note] WSREP: Waiting for SST to complete. 2017-05-05 19:30:13 140184955893504 [Note] WSREP: STATE EXCHANGE: got state msg: 344caedd-3186-11e7-87e9-afb9a0cdb378 from 2 (test-neutron1) 2017-05-05 19:30:13 140184955893504 [Note] WSREP: Quorum results: version = 3, component = PRIMARY, conf_id = 398, members = 2/3 (joined/total), act_id = 28352, last_appl. = -1, protocols = 0/7/3 (gcs/repl/appl), group UUID = f9b78904-3171-11e7-86f5-03aad28c2a6a 2017-05-05 19:30:13 140184955893504 [Note] WSREP: Flow-control interval: [1774, 1774] 2017-05-05 19:30:13 140184955893504 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 28352) 2017-05-05 19:30:13 140202339289856 [Note] WSREP: State transfer required: Group state: f9b78904-3171-11e7-86f5-03aad28c2a6a:28352 Local state: 00000000-0000-0000-0000-000000000000:-1 2017-05-05 19:30:13 140202339289856 [Note] WSREP: New cluster view: global state: f9b78904-3171-11e7-86f5-03aad28c2a6a:28352, view# 399: Primary, number of nodes: 3, my index: 2, protocol version 3 2017-05-05 19:30:13 140202339289856 [Warning] WSREP: Gap in state sequence. Need state transfer. 2017-05-05 19:30:13 140184947500800 [Note] WSREP: Running: 'wsrep_sst_xtrabackup-v2 --role 'joiner' --address '10.127.2.63' --datadir '/var/lib/mysql/' --parent '4747' '' ' WSREP_SST: [INFO] Streaming with xbstream (20170505 19:30:13.726) WSREP_SST: [INFO] Using socat as streamer (20170505 19:30:13.727) WSREP_SST: [INFO] Stale sst_in_progress file: /var/lib/mysql//sst_in_progress (20170505 19:30:13.731) WSREP_SST: [INFO] Evaluating timeout -k 110 100 socat -u TCP-LISTEN:4444,reuseaddr stdio | xbstream -x; RC=( ${PIPESTATUS[@]} ) (20170505 19:30:13.754) 2017-05-05 19:30:13 140202339289856 [Note] WSREP: Prepared SST request: xtrabackup-v2|10.127.2.63:4444/xtrabackup_sst//1 2017-05-05 19:30:13 140202339289856 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2017-05-05 19:30:13 140202339289856 [Note] WSREP: REPL Protocols: 7 (3, 2) 2017-05-05 19:30:13 140202248271616 [Note] WSREP: Service thread queue flushed. 2017-05-05 19:30:13 140202339289856 [Note] WSREP: Assign initial position for certification: 28352, protocol version: 3 2017-05-05 19:30:13 140202248271616 [Note] WSREP: Service thread queue flushed. 2017-05-05 19:30:13 140202339289856 [Warning] WSREP: Failed to prepare for incremental state transfer: Local state UUID (00000000-0000-0000-0000-000000000000) does not match group state UUID (f9b78904-3171-11e7-86f5-03aad28c2a6a): 1 (Operation not permitted) at galera/src/replicator_str.cpp:prepare_for_IST():482. IST will be unavailable. 2017-05-05 19:30:13 140184955893504 [Note] WSREP: Member 2.0 (test-neutron1) requested state transfer from 'any'. Selected 0.0 (test-compute1)(SYNCED) as donor. 2017-05-05 19:30:13 140184955893504 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 28355) 2017-05-05 19:30:13 140202339289856 [Note] WSREP: Requesting state transfer: success, donor: 0 WSREP_SST: [INFO] WARNING: Stale temporary SST directory: /var/lib/mysql//.sst from previous state transfer (20170505 19:30:14.160) WSREP_SST: [INFO] Proceeding with SST (20170505 19:30:14.163) WSREP_SST: [INFO] Evaluating socat -u TCP-LISTEN:4444,reuseaddr stdio | xbstream -x; RC=( ${PIPESTATUS[@]} ) (20170505 19:30:14.163) WSREP_SST: [INFO] Cleaning the existing datadir and innodb-data/log directories (20170505 19:30:14.165) removed ‘/var/lib/mysql/mysql_upgrade_info’ removed ‘/var/lib/mysql/ib_logfile0’ removed ‘/var/lib/mysql/aria_log.00000001’ removed ‘/var/lib/mysql/performance_schema/db.opt’ removed directory: ‘/var/lib/mysql/performance_schema’ removed ‘/var/lib/mysql/mysql/innodb_table_stats.frm’ removed ‘/var/lib/mysql/mysql/help_relation.MYD’ removed ‘/var/lib/mysql/mysql/proc.MYI’ removed ‘/var/lib/mysql/mysql/columns_priv.MYD’ removed ‘/var/lib/mysql/mysql/index_stats.frm’ removed ‘/var/lib/mysql/mysql/index_stats.MYI’ removed ‘/var/lib/mysql/mysql/procs_priv.MYI’ removed ‘/var/lib/mysql/mysql/column_stats.MYI’ removed ‘/var/lib/mysql/mysql/db.MYD’ removed ‘/var/lib/mysql/mysql/table_stats.frm’ removed ‘/var/lib/mysql/mysql/host.MYD’ removed ‘/var/lib/mysql/mysql/help_topic.MYD’ removed ‘/var/lib/mysql/mysql/proxies_priv.MYD’ removed ‘/var/lib/mysql/mysql/proxies_priv.frm’ removed ‘/var/lib/mysql/mysql/time_zone_transition.MYI’ removed ‘/var/lib/mysql/mysql/time_zone_leap_second.MYD’ removed ‘/var/lib/mysql/mysql/host.frm’ removed ‘/var/lib/mysql/mysql/columns_priv.MYI’ removed ‘/var/lib/mysql/mysql/servers.frm’ removed ‘/var/lib/mysql/mysql/func.frm’ removed ‘/var/lib/mysql/mysql/time_zone_transition_type.MYD’ removed ‘/var/lib/mysql/mysql/func.MYI’ removed ‘/var/lib/mysql/mysql/plugin.MYD’ removed ‘/var/lib/mysql/mysql/column_stats.MYD’ removed ‘/var/lib/mysql/mysql/time_zone_transition_type.frm’ removed ‘/var/lib/mysql/mysql/table_stats.MYD’ removed ‘/var/lib/mysql/mysql/time_zone.frm’ removed ‘/var/lib/mysql/mysql/general_log.CSV’ removed ‘/var/lib/mysql/mysql/help_topic.MYI’ removed ‘/var/lib/mysql/mysql/time_zone_name.MYI’ removed ‘/var/lib/mysql/mysql/general_log.CSM’ removed ‘/var/lib/mysql/mysql/time_zone_leap_second.frm’ removed ‘/var/lib/mysql/mysql/user.MYI’ removed ‘/var/lib/mysql/mysql/time_zone.MYD’ removed ‘/var/lib/mysql/mysql/gtid_slave_pos.frm’ removed ‘/var/lib/mysql/mysql/tables_priv.MYI’ removed ‘/var/lib/mysql/mysql/proc.frm’ removed ‘/var/lib/mysql/mysql/event.MYD’ removed ‘/var/lib/mysql/mysql/servers.MYI’ removed ‘/var/lib/mysql/mysql/slow_log.CSV’ removed ‘/var/lib/mysql/mysql/host.MYI’ removed ‘/var/lib/mysql/mysql/innodb_table_stats.ibd’ removed ‘/var/lib/mysql/mysql/help_category.MYI’ removed ‘/var/lib/mysql/mysql/column_stats.frm’ removed ‘/var/lib/mysql/mysql/slow_log.CSM’ removed ‘/var/lib/mysql/mysql/innodb_index_stats.frm’ removed ‘/var/lib/mysql/mysql/index_stats.MYD’ removed ‘/var/lib/mysql/mysql/tables_priv.frm’ removed ‘/var/lib/mysql/mysql/help_keyword.MYD’ removed ‘/var/lib/mysql/mysql/user.MYD’ removed ‘/var/lib/mysql/mysql/roles_mapping.frm’ removed ‘/var/lib/mysql/mysql/roles_mapping.MYD’ removed ‘/var/lib/mysql/mysql/time_zone_transition.MYD’ removed ‘/var/lib/mysql/mysql/help_relation.frm’ removed ‘/var/lib/mysql/mysql/tables_priv.MYD’ removed ‘/var/lib/mysql/mysql/general_log.frm’ removed ‘/var/lib/mysql/mysql/help_keyword.frm’ removed ‘/var/lib/mysql/mysql/procs_priv.frm’ removed ‘/var/lib/mysql/mysql/db.MYI’ removed ‘/var/lib/mysql/mysql/table_stats.MYI’ removed ‘/var/lib/mysql/mysql/servers.MYD’ removed ‘/var/lib/mysql/mysql/proc.MYD’ removed ‘/var/lib/mysql/mysql/time_zone_transition.frm’ removed ‘/var/lib/mysql/mysql/time_zone_transition_type.MYI’ removed ‘/var/lib/mysql/mysql/procs_priv.MYD’ removed ‘/var/lib/mysql/mysql/time_zone.MYI’ removed ‘/var/lib/mysql/mysql/help_topic.frm’ removed ‘/var/lib/mysql/mysql/help_category.MYD’ removed ‘/var/lib/mysql/mysql/db.frm’ removed ‘/var/lib/mysql/mysql/time_zone_name.MYD’ removed ‘/var/lib/mysql/mysql/time_zone_name.frm’ removed ‘/var/lib/mysql/mysql/help_category.frm’ removed ‘/var/lib/mysql/mysql/proxies_priv.MYI’ removed ‘/var/lib/mysql/mysql/user.frm’ removed ‘/var/lib/mysql/mysql/func.MYD’ removed ‘/var/lib/mysql/mysql/help_relation.MYI’ removed ‘/var/lib/mysql/mysql/roles_mapping.MYI’ removed ‘/var/lib/mysql/mysql/plugin.frm’ removed ‘/var/lib/mysql/mysql/gtid_slave_pos.ibd’ removed ‘/var/lib/mysql/mysql/event.frm’ removed ‘/var/lib/mysql/mysql/event.MYI’ removed ‘/var/lib/mysql/mysql/columns_priv.frm’ removed ‘/var/lib/mysql/mysql/help_keyword.MYI’ removed ‘/var/lib/mysql/mysql/innodb_index_stats.ibd’ removed ‘/var/lib/mysql/mysql/plugin.MYI’ removed ‘/var/lib/mysql/mysql/slow_log.frm’ removed ‘/var/lib/mysql/mysql/time_zone_leap_second.MYI’ removed directory: ‘/var/lib/mysql/mysql’ removed directory: ‘/var/lib/mysql/test’ removed ‘/var/lib/mysql/aria_log_control’ removed ‘/var/lib/mysql/ibdata1’ removed ‘/var/lib/mysql/ib_logfile1’ WSREP_SST: [INFO] Waiting for SST streaming to complete! (20170505 19:30:14.177) 2017-05-05 19:30:16 140184964286208 [Note] WSREP: (34002e52, 'tcp://0.0.0.0:4567') turning message relay requesting off xbstream: Can't create/write to file './ibdata1' (Errcode: 17 - File exists) xbstream: failed to create file. 2017/05/05 19:30:24 socat[5280] E write(1, 0x24d3340, 8192): Broken pipe WSREP_SST: [ERROR] Error while getting data from donor node: exit codes: 1 1 (20170505 19:30:24.554) WSREP_SST: [ERROR] Cleanup after exit with status:32 (20170505 19:30:24.556) 2017-05-05 19:30:24 140184947500800 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup-v2 --role 'joiner' --address '10.127.2.63' --datadir '/var/lib/mysql/' --parent '4747' '' : 32 (Broken pipe) 2017-05-05 19:30:24 140184947500800 [ERROR] WSREP: Failed to read uuid:seqno and wsrep_gtid_domain_id from joiner script. 2017-05-05 19:30:24 140202340792448 [ERROR] WSREP: SST failed: 32 (Broken pipe) 2017-05-05 19:30:24 140202340792448 [ERROR] Aborting

2017-05-05 19:30:24 140184955893504 [Warning] WSREP: 0.0 (test-compute1): State transfer to 2.0 (test-neutron1) failed: -22 (Invalid argument) 2017-05-05 19:30:24 140184955893504 [ERROR] WSREP: gcs/src/gcs_group.cpp:gcs_group_handle_join_msg():731: Will never receive state. Need to abort. 2017-05-05 19:30:30 140551864682624 [Note] WSREP: Read nil XID from storage engines, skipping position init 2017-05-05 19:30:30 140551864682624 [Note] WSREP: wsrep_load(): loading provider library 'none' 2017-05-05 19:30:30 7fd4c7f78880 InnoDB: Warning: Using innodb_locks_unsafe_for_binlog is DEPRECATED. This option may be removed in future releases. Please use READ COMMITTED transaction isolation level instead, see http://dev.mysql.com/doc/refman/5.6/en/set-transaction.html. 2017-05-05 19:30:30 140551864682624 [Note] InnoDB: Using mutexes to ref count buffer pool pages 2017-05-05 19:30:30 140551864682624 [Note] InnoDB: The InnoDB memory heap is disabled 2017-05-05 19:30:30 140551864682624 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2017-05-05 19:30:30 140551864682624 [Note] InnoDB: Memory barrier is not used 2017-05-05 19:30:30 140551864682624 [Note] InnoDB: Compressed tables use zlib 1.2.7 2017-05-05 19:30:30 140551864682624 [Note] InnoDB: Using Linux native AIO 2017-05-05 19:30:30 140551864682624 [Note] InnoDB: Using SSE crc32 instructions 2017-05-05 19:30:30 140551864682624 [Note] InnoDB: Initializing buffer pool, size = 30.0G 2017-05-05 19:30:31 140551864682624 [Note] InnoDB: Completed initialization of buffer pool 2017-05-05 19:30:33 140551864682624 [Note] InnoDB: The first specified data file ./ibdata1 did not exist: a new database to be created! 2017-05-05 19:30:33 140551864682624 [Note] InnoDB: Setting file ./ibdata1 size to 12 MB 2017-05-05 19:30:33 140551864682624 [Note] InnoDB: Database physically writes the file full: wait... 2017-05-05 19:30:33 140551864682624 [Note] InnoDB: Setting log file ./ib_logfile101 size to 1 MB 2017-05-05 19:30:33 140551864682624 [Note] InnoDB: Setting log file ./ib_logfile1 size to 1 MB 2017-05-05 19:30:33 140551864682624 [Note] InnoDB: Renaming log file ./ib_logfile101 to ./ib_logfile0 2017-05-05 19:30:33 140551864682624 [Warning] InnoDB: New log files created, LSN=45883 2017-05-05 19:30:33 140551864682624 [Note] InnoDB: Doublewrite buffer not found: creating new 2017-05-05 19:30:33 140551864682624 [Note] InnoDB: Doublewrite buffer created 2017-05-05 19:30:33 140551864682624 [Note] InnoDB: 128 rollback segment(s) are active. 2017-05-05 19:30:33 140551864682624 [Warning] InnoDB: Creating foreign key constraint system tables. 2017-05-05 19:30:33 140551864682624 [Note] InnoDB: Foreign key constraint system tables created 2017-05-05 19:30:33 140551864682624 [Note] InnoDB: Creating tablespace and datafile system tables. 2017-05-05 19:30:33 140551864682624 [Note] InnoDB: Tablespace and datafile system tables created. 2017-05-05 19:30:33 140551864682624 [Note] InnoDB: Waiting for purge to start 2017-05-05 19:30:34 140551864682624 [Note] InnoDB: Percona XtraDB (http://www.percona.com) 5.6.28-76.1 started; log sequence number 0 2017-05-05 19:30:34 140511255979776 [Note] InnoDB: Dumping buffer pool(s) not yet started 2017-05-05 19:30:38 139937196202112 [Note] WSREP: Read nil XID from storage engines, skipping position init 2017-05-05 19:30:38 139937196202112 [Note] WSREP: wsrep_load(): loading provider library 'none' 2017-05-05 19:30:38 7f45aaded880 InnoDB: Warning: Using innodb_locks_unsafe_for_binlog is DEPRECATED. This option may be removed in future releases. Please use READ COMMITTED transaction isolation level instead, see http://dev.mysql.com/doc/refman/5.6/en/set-transaction.html. 2017-05-05 19:30:39 139937196202112 [Note] InnoDB: Using mutexes to ref count buffer pool pages 2017-05-05 19:30:39 139937196202112 [Note] InnoDB: The InnoDB memory heap is disabled 2017-05-05 19:30:39 139937196202112 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2017-05-05 19:30:39 139937196202112 [Note] InnoDB: Memory barrier is not used 2017-05-05 19:30:39 139937196202112 [Note] InnoDB: Compressed tables use zlib 1.2.7 2017-05-05 19:30:39 139937196202112 [Note] InnoDB: Using Linux native AIO 2017-05-05 19:30:39 139937196202112 [Note] InnoDB: Using SSE crc32 instructions 2017-05-05 19:30:39 139937196202112 [Note] InnoDB: Initializing buffer pool, size = 30.0G 2017-05-05 19:30:40 139937196202112 [Note] InnoDB: Completed initialization of buffer pool 2017-05-05 19:30:42 139937196202112 [Note] InnoDB: Highest supported file format is Barracuda. 2017-05-05 19:30:43 139937196202112 [Note] InnoDB: 128 rollback segment(s) are active. 2017-05-05 19:30:43 139937196202112 [Note] InnoDB: Waiting for purge to start 2017-05-05 19:30:43 139937196202112 [Note] InnoDB: Percona XtraDB (http://www.percona.com) 5.6.28-76.1 started; log sequence number 1616799 2017-05-05 19:30:43 139897298867968 [Note] InnoDB: Dumping buffer pool(s) not yet started 2017-05-05 19:30:48 140654480840832 [Note] WSREP: Read nil XID from storage engines, skipping position init 2017-05-05 19:30:48 140654480840832 [Note] WSREP: wsrep_load(): loading provider library 'none' 2017-05-05 19:30:48 7fecac5dd880 InnoDB: Warning: Using innodb_locks_unsafe_for_binlog is DEPRECATED. This option may be removed in future releases. Please use READ COMMITTED transaction isolation level instead, see http://dev.mysql.com/doc/refman/5.6/en/set-transaction.html. 2017-05-05 19:30:48 140654480840832 [Note] InnoDB: Using mutexes to ref count buffer pool pages 2017-05-05 19:30:48 140654480840832 [Note] InnoDB: The InnoDB memory heap is disabled 2017-05-05 19:30:48 140654480840832 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2017-05-05 19:30:48 140654480840832 [Note] InnoDB: Memory barrier is not used 2017-05-05 19:30:48 140654480840832 [Note] InnoDB: Compressed tables use zlib 1.2.7 2017-05-05 19:30:48 140654480840832 [Note] InnoDB: Using Linux native AIO 2017-05-05 19:30:48 140654480840832 [Note] InnoDB: Using SSE crc32 instructions 2017-05-05 19:30:48 140654480840832 [Note] InnoDB: Initializing buffer pool, size = 30.0G 2017-05-05 19:30:50 140654480840832 [Note] InnoDB: Completed initialization of buffer pool 2017-05-05 19:30:52 140654480840832 [Note] InnoDB: Highest supported file format is Barracuda. 2017-05-05 19:30:52 140654480840832 [Note] InnoDB: 128 rollback segment(s) are active. 2017-05-05 19:30:52 140654480840832 [Note] InnoDB: Waiting for purge to start 2017-05-05 19:30:52 140654480840832 [Note] InnoDB: Percona XtraDB (http://www.percona.com) 5.6.28-76.1 started; log sequence number 1616809 2017-05-05 19:30:52 140614511343360 [Note] InnoDB: Dumping buffer pool(s) not yet started 2017-05-05 19:30:57 140214469929088 [Note] WSREP: Read nil XID from storage engines, skipping position init 2017-05-05 19:30:57 140214469929088 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib64/galera/libgalera_smm.so' 2017-05-05 19:30:57 140214469929088 [Note] WSREP: wsrep_load(): Galera 3.12(r9921e73) by Codership Oy info@codership.com loaded successfully. 2017-05-05 19:30:57 140214469929088 [Note] WSREP: CRC-32C: using hardware acceleration. 2017-05-05 19:30:57 140214469929088 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1 2017-05-05 19:30:57 140214469929088 [Note] WSREP: Passing config to GCS: base_dir = /var/lib/mysql/; base_host = 10.127.2.63; base_port = 4567; cert.log_conflicts = YES; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 16G; gcache.size = 16G; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 1024; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.checksum = false; pc.ignore_quorum = false; pc.ignore_sb = false; 2017-05-05 19:30:57 140197127776000 [Note] WSREP: Service thread queue flushed. 2017-05-05 19:30:57 140214469929088 [Note] WSREP: Assign initial position for certification: -1, protocol version: -1 2017-05-05 19:30:57 140214469929088 [Note] WSREP: wsrep_sst_grab() 2017-05-05 19:30:57 140214469929088 [Note] WSREP: Start replication 2017-05-05 19:30:57 140214469929088 [Note] WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:-1 2017-05-05 19:30:57 140214469929088 [Note] WSREP: protonet asio version 0 2017-05-05 19:30:57 140214469929088 [Note] WSREP: Using CRC-32C for message checksums. 2017-05-05 19:30:57 140214469929088 [Note] WSREP: backend: asio 2017-05-05 19:30:57 140214469929088 [Warning] WSREP: access file(/var/lib/mysql//gvwstate.dat) failed(No such file or directory) 2017-05-05 19:30:57 140214469929088 [Note] WSREP: restore pc from disk failed 2017-05-05 19:30:57 140214469929088 [Note] WSREP: GMCast version 0 2017-05-05 19:30:57 140214469929088 [Note] WSREP: (4ef29f5f, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567 2017-05-05 19:30:57 140214469929088 [Note] WSREP: (4ef29f5f, 'tcp://0.0.0.0:4567') multicast: , ttl: 1 2017-05-05 19:30:57 140214469929088 [Note] WSREP: EVS version 0 2017-05-05 19:30:57 140214469929088 [Note] WSREP: gcomm: connecting to group 'fh_wsrep_cluster', peer '10.127.2.60:,10.127.2.65:' 2017-05-05 19:30:57 140214469929088 [Note] WSREP: (4ef29f5f, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers: 2017-05-05 19:30:57 140214469929088 [Note] WSREP: declaring 02d46a25 at tcp://10.127.2.65:4567 stable 2017-05-05 19:30:57 140214469929088 [Note] WSREP: declaring 1a7de35a at tcp://10.127.2.60:4567 stable 2017-05-05 19:30:57 140214469929088 [Note] WSREP: Node 02d46a25 state prim 2017-05-05 19:30:57 140214469929088 [Note] WSREP: view(view_id(PRIM,02d46a25,402) memb { 02d46a25,0 1a7de35a,0 4ef29f5f,0 } joined { } left { } partitioned { }) 2017-05-05 19:30:57 140214469929088 [Note] WSREP: save pc into disk 2017-05-05 19:30:58 140214469929088 [Note] WSREP: gcomm: connected 2017-05-05 19:30:58 140214469929088 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636 2017-05-05 19:30:58 140214469929088 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0) 2017-05-05 19:30:58 140214469929088 [Note] WSREP: Opened channel 'fh_wsrep_cluster' 2017-05-05 19:30:58 140197085812480 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 2, memb_num = 3 2017-05-05 19:30:58 140197085812480 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID. 2017-05-05 19:30:58 140197085812480 [Note] WSREP: STATE EXCHANGE: sent state msg: 4efe4df1-3186-11e7-b177-56e4add5e3ce 2017-05-05 19:30:58 140214469929088 [Note] WSREP: Waiting for SST to complete. 2017-05-05 19:30:58 140197085812480 [Note] WSREP: STATE EXCHANGE: got state msg: 4efe4df1-3186-11e7-b177-56e4add5e3ce from 0 (test-compute1) 2017-05-05 19:30:58 140197085812480 [Note] WSREP: STATE EXCHANGE: got state msg: 4efe4df1-3186-11e7-b177-56e4add5e3ce from 1 (test-controller1) 2017-05-05 19:30:58 140197085812480 [Note] WSREP: STATE EXCHANGE: got state msg: 4efe4df1-3186-11e7-b177-56e4add5e3ce from 2 (test-neutron1) 2017-05-05 19:30:58 140197085812480 [Note] WSREP: Quorum results: version = 3, component = PRIMARY, conf_id = 400, members = 2/3 (joined/total), act_id = 28494, last_appl. = -1, protocols = 0/7/3 (gcs/repl/appl), group UUID = f9b78904-3171-11e7-86f5-03aad28c2a6a 2017-05-05 19:30:58 140197085812480 [Note] WSREP: Flow-control interval: [1774, 1774] 2017-05-05 19:30:58 140197085812480 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 28494) 2017-05-05 19:30:58 140214468426496 [Note] WSREP: State transfer required: Group state: f9b78904-3171-11e7-86f5-03aad28c2a6a:28494 Local state: 00000000-0000-0000-0000-000000000000:-1 2017-05-05 19:30:58 140214468426496 [Note] WSREP: New cluster view: global state: f9b78904-3171-11e7-86f5-03aad28c2a6a:28494, view# 401: Primary, number of nodes: 3, my index: 2, protocol version 3 2017-05-05 19:30:58 140214468426496 [Warning] WSREP: Gap in state sequence. Need state transfer. 2017-05-05 19:30:58 140197077419776 [Note] WSREP: Running: 'wsrep_sst_xtrabackup-v2 --role 'joiner' --address '10.127.2.63' --datadir '/var/lib/mysql/' --parent '9522' '' ' WSREP_SST: [INFO] Streaming with xbstream (20170505 19:30:58.438) WSREP_SST: [INFO] Using socat as streamer (20170505 19:30:58.439) WSREP_SST: [INFO] Stale sst_in_progress file: /var/lib/mysql//sst_in_progress (20170505 19:30:58.443) WSREP_SST: [INFO] Evaluating timeout -k 110 100 socat -u TCP-LISTEN:4444,reuseaddr stdio | xbstream -x; RC=( ${PIPESTATUS[@]} ) (20170505 19:30:58.466) 2017-05-05 19:30:58 140214468426496 [Note] WSREP: Prepared SST request: xtrabackup-v2|10.127.2.63:4444/xtrabackup_sst//1 2017-05-05 19:30:58 140214468426496 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2017-05-05 19:30:58 140214468426496 [Note] WSREP: REPL Protocols: 7 (3, 2) 2017-05-05 19:30:58 140197127776000 [Note] WSREP: Service thread queue flushed. 2017-05-05 19:30:58 140214468426496 [Note] WSREP: Assign initial position for certification: 28494, protocol version: 3 2017-05-05 19:30:58 140197127776000 [Note] WSREP: Service thread queue flushed. 2017-05-05 19:30:58 140214468426496 [Warning] WSREP: Failed to prepare for incremental state transfer: Local state UUID (00000000-0000-0000-0000-000000000000) does not match group state UUID (f9b78904-3171-11e7-86f5-03aad28c2a6a): 1 (Operation not permitted) at galera/src/replicator_str.cpp:prepare_for_IST():482. IST will be unavailable. 2017-05-05 19:30:58 140197085812480 [Note] WSREP: Member 2.0 (test-neutron1) requested state transfer from 'any'. Selected 0.0 (test-compute1)(SYNCED) as donor. 2017-05-05 19:30:58 140197085812480 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 28497) 2017-05-05 19:30:58 140214468426496 [Note] WSREP: Requesting state transfer: success, donor: 0 WSREP_SST: [INFO] WARNING: Stale temporary SST directory: /var/lib/mysql//.sst from previous state transfer (20170505 19:30:58.880) WSREP_SST: [INFO] Proceeding with SST (20170505 19:30:58.883) WSREP_SST: [INFO] Evaluating socat -u TCP-LISTEN:4444,reuseaddr stdio | xbstream -x; RC=( ${PIPESTATUS[@]} ) (20170505 19:30:58.883) WSREP_SST: [INFO] Cleaning the existing datadir and innodb-data/log directories (20170505 19:30:58.884) removed ‘/var/lib/mysql/mysql_upgrade_info’ removed ‘/var/lib/mysql/ib_logfile0’ removed ‘/var/lib/mysql/aria_log.00000001’ removed ‘/var/lib/mysql/performance_schema/db.opt’ removed directory: ‘/var/lib/mysql/performance_schema’ removed ‘/var/lib/mysql/mysql/innodb_table_stats.frm’ removed ‘/var/lib/mysql/mysql/help_relation.MYD’ removed ‘/var/lib/mysql/mysql/proc.MYI’ removed ‘/var/lib/mysql/mysql/columns_priv.MYD’ removed ‘/var/lib/mysql/mysql/index_stats.frm’ removed ‘/var/lib/mysql/mysql/index_stats.MYI’ removed ‘/var/lib/mysql/mysql/procs_priv.MYI’ removed ‘/var/lib/mysql/mysql/column_stats.MYI’ removed ‘/var/lib/mysql/mysql/db.MYD’ removed ‘/var/lib/mysql/mysql/table_stats.frm’ removed ‘/var/lib/mysql/mysql/host.MYD’ removed ‘/var/lib/mysql/mysql/help_topic.MYD’ removed ‘/var/lib/mysql/mysql/proxies_priv.MYD’ removed ‘/var/lib/mysql/mysql/proxies_priv.frm’ removed ‘/var/lib/mysql/mysql/time_zone_transition.MYI’ removed ‘/var/lib/mysql/mysql/time_zone_leap_second.MYD’ removed ‘/var/lib/mysql/mysql/host.frm’ removed ‘/var/lib/mysql/mysql/columns_priv.MYI’ removed ‘/var/lib/mysql/mysql/servers.frm’ removed ‘/var/lib/mysql/mysql/func.frm’ removed ‘/var/lib/mysql/mysql/time_zone_transition_type.MYD’ removed ‘/var/lib/mysql/mysql/func.MYI’ removed ‘/var/lib/mysql/mysql/plugin.MYD’ removed ‘/var/lib/mysql/mysql/column_stats.MYD’ removed ‘/var/lib/mysql/mysql/time_zone_transition_type.frm’ removed ‘/var/lib/mysql/mysql/table_stats.MYD’ removed ‘/var/lib/mysql/mysql/time_zone.frm’ removed ‘/var/lib/mysql/mysql/general_log.CSV’ removed ‘/var/lib/mysql/mysql/help_topic.MYI’ removed ‘/var/lib/mysql/mysql/time_zone_name.MYI’ removed ‘/var/lib/mysql/mysql/general_log.CSM’ removed ‘/var/lib/mysql/mysql/time_zone_leap_second.frm’ removed ‘/var/lib/mysql/mysql/user.MYI’ removed ‘/var/lib/mysql/mysql/time_zone.MYD’ removed ‘/var/lib/mysql/mysql/gtid_slave_pos.frm’ removed ‘/var/lib/mysql/mysql/tables_priv.MYI’ removed ‘/var/lib/mysql/mysql/proc.frm’ removed ‘/var/lib/mysql/mysql/event.MYD’ removed ‘/var/lib/mysql/mysql/servers.MYI’ removed ‘/var/lib/mysql/mysql/slow_log.CSV’ removed ‘/var/lib/mysql/mysql/host.MYI’ removed ‘/var/lib/mysql/mysql/innodb_table_stats.ibd’ removed ‘/var/lib/mysql/mysql/help_category.MYI’ removed ‘/var/lib/mysql/mysql/column_stats.frm’ removed ‘/var/lib/mysql/mysql/slow_log.CSM’ removed ‘/var/lib/mysql/mysql/innodb_index_stats.frm’ removed ‘/var/lib/mysql/mysql/index_stats.MYD’ removed ‘/var/lib/mysql/mysql/tables_priv.frm’ removed ‘/var/lib/mysql/mysql/help_keyword.MYD’ removed ‘/var/lib/mysql/mysql/user.MYD’ removed ‘/var/lib/mysql/mysql/roles_mapping.frm’ removed ‘/var/lib/mysql/mysql/roles_mapping.MYD’ removed ‘/var/lib/mysql/mysql/time_zone_transition.MYD’ removed ‘/var/lib/mysql/mysql/help_relation.frm’ removed ‘/var/lib/mysql/mysql/tables_priv.MYD’ removed ‘/var/lib/mysql/mysql/general_log.frm’ removed ‘/var/lib/mysql/mysql/help_keyword.frm’ removed ‘/var/lib/mysql/mysql/procs_priv.frm’ removed ‘/var/lib/mysql/mysql/db.MYI’ removed ‘/var/lib/mysql/mysql/table_stats.MYI’ removed ‘/var/lib/mysql/mysql/servers.MYD’ removed ‘/var/lib/mysql/mysql/proc.MYD’ removed ‘/var/lib/mysql/mysql/time_zone_transition.frm’ removed ‘/var/lib/mysql/mysql/time_zone_transition_type.MYI’ removed ‘/var/lib/mysql/mysql/procs_priv.MYD’ removed ‘/var/lib/mysql/mysql/time_zone.MYI’ removed ‘/var/lib/mysql/mysql/help_topic.frm’ removed ‘/var/lib/mysql/mysql/help_category.MYD’ removed ‘/var/lib/mysql/mysql/db.frm’ removed ‘/var/lib/mysql/mysql/time_zone_name.MYD’ removed ‘/var/lib/mysql/mysql/time_zone_name.frm’ removed ‘/var/lib/mysql/mysql/help_category.frm’ removed ‘/var/lib/mysql/mysql/proxies_priv.MYI’ removed ‘/var/lib/mysql/mysql/user.frm’ removed ‘/var/lib/mysql/mysql/func.MYD’ removed ‘/var/lib/mysql/mysql/help_relation.MYI’ removed ‘/var/lib/mysql/mysql/roles_mapping.MYI’ removed ‘/var/lib/mysql/mysql/plugin.frm’ removed ‘/var/lib/mysql/mysql/gtid_slave_pos.ibd’ removed ‘/var/lib/mysql/mysql/event.frm’ removed ‘/var/lib/mysql/mysql/event.MYI’ removed ‘/var/lib/mysql/mysql/columns_priv.frm’ removed ‘/var/lib/mysql/mysql/help_keyword.MYI’ removed ‘/var/lib/mysql/mysql/innodb_index_stats.ibd’ removed ‘/var/lib/mysql/mysql/plugin.MYI’ removed ‘/var/lib/mysql/mysql/slow_log.frm’ removed ‘/var/lib/mysql/mysql/time_zone_leap_second.MYI’ removed directory: ‘/var/lib/mysql/mysql’ removed directory: ‘/var/lib/mysql/test’ removed ‘/var/lib/mysql/aria_log_control’ removed ‘/var/lib/mysql/ibdata1’ removed ‘/var/lib/mysql/ib_logfile1’ WSREP_SST: [INFO] Waiting for SST streaming to complete! (20170505 19:30:58.897) 2017-05-05 19:31:01 140197094205184 [Note] WSREP: (4ef29f5f, 'tcp://0.0.0.0:4567') turning message relay requesting off xbstream: Can't create/write to file './ibdata1' (Errcode: 17 - File exists) xbstream: failed to create file. 2017/05/05 19:31:09 socat[10078] E write(1, 0x1e44340, 8192): Broken pipe WSREP_SST: [ERROR] Error while getting data from donor node: exit codes: 1 1 (20170505 19:31:09.273) WSREP_SST: [ERROR] Cleanup after exit with status:32 (20170505 19:31:09.275) 2017-05-05 19:31:09 140197077419776 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup-v2 --role 'joiner' --address '10.127.2.63' --datadir '/var/lib/mysql/' --parent '9522' '' : 32 (Broken pipe) 2017-05-05 19:31:09 140197077419776 [ERROR] WSREP: Failed to read uuid:seqno and wsrep_gtid_domain_id from joiner script. 2017-05-05 19:31:09 140214469929088 [ERROR] WSREP: SST failed: 32 (Broken pipe) 2017-05-05 19:31:09 140214469929088 [ERROR] Aborting

2017-05-05 19:31:09 140197085812480 [Warning] WSREP: 0.0 (test-compute1): State transfer to 2.0 (test-neutron1) failed: -22 (Invalid argument) 2017-05-05 19:31:09 140197085812480 [ERROR] WSREP: gcs/src/gcs_group.cpp:gcs_group_handle_join_msg():731: Will never receive state. Need to abort. 2017-05-05 19:31:14 140391883962496 [Note] WSREP: Read nil XID from storage engines, skipping position init 2017-05-05 19:31:14 140391883962496 [Note] WSREP: wsrep_load(): loading provider library 'none' 2017-05-05 19:31:14 7faf885f7880 InnoDB: Warning: Using innodb_locks_unsafe_for_binlog is DEPRECATED. This option may be removed in future releases. Please use READ COMMITTED transaction isolation level instead, see http://dev.mysql.com/doc/refman/5.6/en/set-transaction.html. 2017-05-05 19:31:14 140391883962496 [Note] InnoDB: Using mutexes to ref count buffer pool pages 2017-05-05 19:31:14 140391883962496 [Note] InnoDB: The InnoDB memory heap is disabled 2017-05-05 19:31:14 140391883962496 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2017-05-05 19:31:14 140391883962496 [Note] InnoDB: Memory barrier is not used 2017-05-05 19:31:14 140391883962496 [Note] InnoDB: Compressed tables use zlib 1.2.7 2017-05-05 19:31:14 140391883962496 [Note] InnoDB: Using Linux native AIO 2017-05-05 19:31:14 140391883962496 [Note] InnoDB: Using SSE crc32 instructions 2017-05-05 19:31:14 140391883962496 [Note] InnoDB: Initializing buffer pool, size = 30.0G 2017-05-05 19:31:16 140391883962496 [Note] InnoDB: Completed initialization of buffer pool 2017-05-05 19:31:18 140391883962496 [Note] InnoDB: The first specified data file ./ibdata1 did not exist: a new database to be created! 2017-05-05 19:31:18 140391883962496 [Note] InnoDB: Setting file ./ibdata1 size to 12 MB 2017-05-05 19:31:18 140391883962496 [Note] InnoDB: Database physically writes the file full: wait... 2017-05-05 19:31:18 140391883962496 [Note] InnoDB: Setting log file ./ib_logfile101 size to 1 MB 2017-05-05 19:31:18 140391883962496 [Note] InnoDB: Setting log file ./ib_logfile1 size to 1 MB 2017-05-05 19:31:18 140391883962496 [Note] InnoDB: Renaming log file ./ib_logfile101 to ./ib_logfile0 2017-05-05 19:31:18 140391883962496 [Warning] InnoDB: New log files created, LSN=45883 2017-05-05 19:31:18 140391883962496 [Note] InnoDB: Doublewrite buffer not found: creating new 2017-05-05 19:31:18 140391883962496 [Note] InnoDB: Doublewrite buffer created 2017-05-05 19:31:18 140391883962496 [Note] InnoDB: 128 rollback segment(s) are active. 2017-05-05 19:31:18 140391883962496 [Warning] InnoDB: Creating foreign key constraint system tables. 2017-05-05 19:31:18 140391883962496 [Note] InnoDB: Foreign key constraint system tables created 2017-05-05 19:31:18 140391883962496 [Note] InnoDB: Creating tablespace and datafile system tables. 2017-05-05 19:31:18 140391883962496 [Note] InnoDB: Tablespace and datafile system tables created. 2017-05-05 19:31:18 140391883962496 [Note] InnoDB: Waiting for purge to start 2017-05-05 19:31:18 140391883962496 [Note] InnoDB: Percona XtraDB (http://www.percona.com) 5.6.28-76.1 started; log sequence number 0 2017-05-05 19:31:18 140351276840704 [Note] InnoDB: Dumping buffer pool(s) not yet started 2017-05-05 19:31:23 140196351211648 [Note] WSREP: Read nil XID from storage engines, skipping position init 2017-05-05 19:31:23 140196351211648 [Note] WSREP: wsrep_load(): loading provider library 'none' 2017-05-05 19:31:23 7f8201b67880 InnoDB: Warning: Using innodb_locks_unsafe_for_binlog is DEPRECATED. This option may be removed in future releases. Please use READ COMMITTED transaction isolation level instead, see http://dev.mysql.com/doc/refman/5.6/en/set-transaction.html. 2017-05-05 19:31:23 140196351211648 [Note] InnoDB: Using mutexes to ref count buffer pool pages 2017-05-05 19:31:23 140196351211648 [Note] InnoDB: The InnoDB memory heap is disabled 2017-05-05 19:31:23 140196351211648 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2017-05-05 19:31:23 140196351211648 [Note] InnoDB: Memory barrier is not used 2017-05-05 19:31:23 140196351211648 [Note] InnoDB: Compressed tables use zlib 1.2.7 2017-05-05 19:31:23 140196351211648 [Note] InnoDB: Using Linux native AIO 2017-05-05 19:31:23 140196351211648 [Note] InnoDB: Using SSE crc32 instructions 2017-05-05 19:31:23 140196351211648 [Note] InnoDB: Initializing buffer pool, size = 30.0G 2017-05-05 19:31:25 140196351211648 [Note] InnoDB: Completed initialization of buffer pool 2017-05-05 19:31:27 140196351211648 [Note] InnoDB: Highest supported file format is Barracuda. 2017-05-05 19:31:27 140196351211648 [Note] InnoDB: 128 rollback segment(s) are active. 2017-05-05 19:31:27 140196351211648 [Note] InnoDB: Waiting for purge to start 2017-05-05 19:31:27 140196351211648 [Note] InnoDB: Percona XtraDB (http://www.percona.com) 5.6.28-76.1 started; log sequence number 1616799 2017-05-05 19:31:27 140156384306944 [Note] InnoDB: Dumping buffer pool(s) not yet started 2017-05-05 19:31:32 140094620481664 [Note] WSREP: Read nil XID from storage engines, skipping position init

[indicolite]

update mariadb-10.1.12 script as below:

#!/bin/sh

# This script creates the mysql data directory during first service start.
# In subsequent starts, it does nothing much.
set -x

source "`dirname ${BASH_SOURCE[0]}`/mysql-scripts-common"

# If two args given first is user, second is group
# otherwise the arg is the systemd service file
if [ "$#" -eq 2 ]
then
    myuser="$1"
    mygroup="$2"
else
    # Absorb configuration settings from the specified systemd service file,
    # or the default service if not specified
    SERVICE_NAME="$1"
    if [ x"$SERVICE_NAME" = x ]
    then
        SERVICE_NAME=mariadb.service
    fi

    myuser=`systemctl show -p User "${SERVICE_NAME}" |
      sed 's/^User=//'`
    if [ x"$myuser" = x ]
    then
        myuser=mysql
    fi

    mygroup=`systemctl show -p Group "${SERVICE_NAME}" |
      sed 's/^Group=//'`
    if [ x"$mygroup" = x ]
    then
        mygroup=mysql
    fi
fi

# Set up the errlogfile with appropriate permissions
touch "$errlogfile"
ret=$?
# Provide some advice if the log file cannot be touched
if [ $ret -ne 0 ] ; then
    errlogdir=$(dirname $errlogfile)
    if ! [ -d "$errlogdir" ] ; then
        echo "The directory $errlogdir does not exist."
    elif [ -f "$errlogfile" ] ; then
        echo "The log file $errlogfile cannot be touched, please, fix its permissions."
    else
        echo "The log file $errlogfile could not be created."
    fi
    echo "The daemon will be run under $myuser:$mygroup"
    exit 1
fi
chown "$myuser:$mygroup" "$errlogfile"
chmod 0640 "$errlogfile"
[ -x /sbin/restorecon ] && /sbin/restorecon "$errlogfile"

# Make the data directory
if [ ! -d "$datadir/mysql" ] ; then
    # First, make sure $datadir is there with correct permissions
    # (note: if it's not, and we're not root, this'll fail ...)
    if [ ! -e "$datadir" -a ! -h "$datadir" ]
    then
        mkdir -p "$datadir" || exit 1
    fi
    chown "$myuser:$mygroup" "$datadir"
    chmod 0755 "$datadir"
    [ -x /sbin/restorecon ] && /sbin/restorecon "$datadir"

    # Now create the database
    echo "Initializing MariaDB database"
    INITDB_TIMESTAMP=`LANG=C date -u`
    sleep 1
    /usr/bin/mysql_install_db --rpm --datadir="$datadir" --user="$myuser"
    ret=$?
    if [ $ret -ne 0 ] ; then
        echo "Initialization of MariaDB database failed." >&2
        echo "Perhaps /etc/my.cnf is misconfigured." >&2
        # Clean up any partially-created database files
        #if [ ! -e "$datadir/mysql/user.frm" ] ; then
        #    rm -rf "$datadir"/*
        #fi
        if [ ! -e "$datadir/mysql/user.frm" ] && [ -d "$datadir" ] ; then
            echo "Initialization of MariaDB database was not finished successfully." >&2
            echo "Files created so far will be removed." >&2
            find "$datadir" -mindepth 1 -maxdepth 1 -newermt "$INITDB_TIMESTAMP" \
                 -not -name "lost+found" -exec rm -rf {} +
            if [ $? -ne 0 ] ; then
                echo "Removing of created files was not successfull." >&2
                echo "Please, clean directory $datadir manually." >&2
            fi
        else
            echo "However, part of data has been initialized and those will not be removed." >&2
            echo "Please, clean directory $datadir manually." >&2
        fi
        exit $ret
    fi
    # upgrade does not need to be run on a fresh datadir
    echo "10.1.12-MariaDB" >"$datadir/mysql_upgrade_info"
    # In case we're running as root, make sure files are owned properly
    chown -R "$myuser:$mygroup" "$datadir"
fi

exit 0

[indicolite]

Close this bug now.

[indicolite]

bugzilla：https://bugzilla.redhat.com/show_bug.cgi?id=1456084