search

yoshinorim / mha4mysql-manager

Development tree of Master High Availability Manager and tools for MySQL (MHA), Manager part
http://code.google.com/p/mysql-master-ha/
GNU General Public License v2.0
1.46k stars 501 forks source link

The mysqlmha online switch is not successful（masterha_master_switch --master_state=alive），ERROR:master_ip_online_change: error: no such option: --orig_master_ssh_user。MySQL-8.0.12，mha4mysql-manager-0.58 #126

Open lixiaofeng1026 opened 4 years ago

lixiaofeng1026 commented 4 years ago

Hi, The mysqlmha online switch is not successful（masterha_master_switch --master_state=alive）。 [root@mysql-manager-23 conf]# masterha_master_switch --master_state=alive --global_conf=/usr/local/mha-helper-master/conf/global.conf --conf=/usr/local/mha-helper-master/conf/manager.conf Tue Oct 29 19:40:02 2019 - [info] MHA::MasterRotate version 0.58. Tue Oct 29 19:40:02 2019 - [info] Starting online master switch.. Tue Oct 29 19:40:02 2019 - [info] Tue Oct 29 19:40:02 2019 - [info] * Phase 1: Configuration Check Phase.. Tue Oct 29 19:40:02 2019 - [info] Tue Oct 29 19:40:02 2019 - [info] Reading default configuration from /usr/local/mha-helper-master/conf/global.conf.. Tue Oct 29 19:40:02 2019 - [info] Reading application default configuration from /usr/local/mha-helper-master/conf/manager.conf.. Tue Oct 29 19:40:02 2019 - [info] Reading server configuration from /usr/local/mha-helper-master/conf/manager.conf.. Tue Oct 29 19:40:02 2019 - [debug] Connecting to servers.. Tue Oct 29 19:40:03 2019 - [debug] Connected to: 172.19.14.21(172.19.14.21:3306), user=root Tue Oct 29 19:40:03 2019 - [debug] Number of slave worker threads on host 172.19.14.21(172.19.14.21:3306): 0 Tue Oct 29 19:40:03 2019 - [debug] Connected to: 172.19.14.23(172.19.14.23:3306), user=root Tue Oct 29 19:40:03 2019 - [debug] Number of slave worker threads on host 172.19.14.23(172.19.14.23:3306): 0 Tue Oct 29 19:40:03 2019 - [debug] Connected to: 172.19.14.22(172.19.14.22:3306), user=root Tue Oct 29 19:40:03 2019 - [debug] Number of slave worker threads on host 172.19.14.22(172.19.14.22:3306): 0 Tue Oct 29 19:40:03 2019 - [debug] Comparing MySQL versions.. Tue Oct 29 19:40:03 2019 - [debug] Comparing MySQL versions done. Tue Oct 29 19:40:03 2019 - [debug] Connecting to servers done. Tue Oct 29 19:40:03 2019 - [info] GTID failover mode = 0 Tue Oct 29 19:40:03 2019 - [info] Current Alive Master: 172.19.14.21(172.19.14.21:3306) Tue Oct 29 19:40:03 2019 - [info] Alive Slaves: Tue Oct 29 19:40:03 2019 - [info] 172.19.14.23(172.19.14.23:3306) Version=8.0.12 (oldest major version between slaves) log-bin:enabled Tue Oct 29 19:40:03 2019 - [debug] Relay log info repository: TABLE Tue Oct 29 19:40:03 2019 - [info] Replicating from 172.19.14.21(172.19.14.21:3306) Tue Oct 29 19:40:03 2019 - [info] Primary candidate for the new Master (candidate_master is set) Tue Oct 29 19:40:03 2019 - [info] 172.19.14.22(172.19.14.22:3306) Version=8.0.12 (oldest major version between slaves) log-bin:enabled Tue Oct 29 19:40:03 2019 - [debug] Relay log info repository: TABLE Tue Oct 29 19:40:03 2019 - [info] Replicating from 172.19.14.21(172.19.14.21:3306) Tue Oct 29 19:40:03 2019 - [info] Not candidate for the new Master (no_master is set)

It is better to execute FLUSH NO_WRITE_TO_BINLOG TABLES on the master before switching. Is it ok to execute on 172.19.14.21(172.19.14.21:3306)? (YES/no): yes Tue Oct 29 19:40:05 2019 - [info] Executing FLUSH NO_WRITE_TO_BINLOG TABLES. This may take long time.. Tue Oct 29 19:40:05 2019 - [info] ok. Tue Oct 29 19:40:05 2019 - [info] Checking MHA is not monitoring or doing failover.. Tue Oct 29 19:40:05 2019 - [info] Checking replication health on 172.19.14.23.. Tue Oct 29 19:40:05 2019 - [info] ok. Tue Oct 29 19:40:05 2019 - [info] Checking replication health on 172.19.14.22.. Tue Oct 29 19:40:05 2019 - [info] ok. Tue Oct 29 19:40:05 2019 - [info] Searching new master from slaves.. Tue Oct 29 19:40:05 2019 - [info] Candidate masters from the configuration file: Tue Oct 29 19:40:05 2019 - [info] 172.19.14.21(172.19.14.21:3306) Version=8.0.12 log-bin:enabled Tue Oct 29 19:40:05 2019 - [info] 172.19.14.23(172.19.14.23:3306) Version=8.0.12 (oldest major version between slaves) log-bin:enabled Tue Oct 29 19:40:05 2019 - [debug] Relay log info repository: TABLE Tue Oct 29 19:40:05 2019 - [info] Replicating from 172.19.14.21(172.19.14.21:3306) Tue Oct 29 19:40:05 2019 - [info] Primary candidate for the new Master (candidate_master is set) Tue Oct 29 19:40:05 2019 - [info] Non-candidate masters: Tue Oct 29 19:40:05 2019 - [info] 172.19.14.22(172.19.14.22:3306) Version=8.0.12 (oldest major version between slaves) log-bin:enabled Tue Oct 29 19:40:05 2019 - [debug] Relay log info repository: TABLE Tue Oct 29 19:40:05 2019 - [info] Replicating from 172.19.14.21(172.19.14.21:3306) Tue Oct 29 19:40:05 2019 - [info] Not candidate for the new Master (no_master is set) Tue Oct 29 19:40:05 2019 - [info] Searching from candidate_master slaves which have received the latest relay log events.. Tue Oct 29 19:40:05 2019 - [info] From: 172.19.14.21(172.19.14.21:3306) (current master) +--172.19.14.23(172.19.14.23:3306) +--172.19.14.22(172.19.14.22:3306)

To: 172.19.14.23(172.19.14.23:3306) (new master) +--172.19.14.22(172.19.14.22:3306)

Starting master switch from 172.19.14.21(172.19.14.21:3306) to 172.19.14.23(172.19.14.23:3306)? (yes/NO): yes Tue Oct 29 19:40:07 2019 - [info] Checking whether 172.19.14.23(172.19.14.23:3306) is ok for the new master.. Tue Oct 29 19:40:07 2019 - [info] ok. Tue Oct 29 19:40:07 2019 - [info] * Phase 1: Configuration Check Phase completed. Tue Oct 29 19:40:07 2019 - [info] Tue Oct 29 19:40:07 2019 - [debug] Disconnected from 172.19.14.21(172.19.14.21:3306) Tue Oct 29 19:40:07 2019 - [info] Phase 2: Rejecting updates Phase.. Tue Oct 29 19:40:07 2019 - [info] Tue Oct 29 19:40:07 2019 - [info] Executing master ip online change script to disable write on the current master: Tue Oct 29 19:40:07 2019 - [info] /usr/local/mha-helper-master/scripts/master_ip_online_change --command=stop --orig_master_host=172.19.14.21 --orig_master_ip=172.19.14.21 --orig_master_port=3306 --orig_master_user='root' --new_master_host=172.19.14.23 --new_master_ip=172.19.14.23 --new_master_port=3306 --new_master_user='root' --orig_master_ssh_user=root --new_master_ssh_user=root --orig_master_ssh_port=11017 --ssh_options='-o ServerAliveInterval=60 -o ServerAliveCountMax=20 -o StrictHostKeyChecking=no -o ConnectionAttempts=5 -o PasswordAuthentication=no -o BatchMode=yes -i /root/.ssh/id_rsa' --new_master_ssh_port=11017 --orig_master_password=xxx --new_master_password=xxx Usage: master_ip_online_change [options]

master_ip_online_change: error: no such option: --orig_master_ssh_user Tue Oct 29 19:40:07 2019 - [error][/usr/share/perl5/vendor_perl/MHA/ManagerUtil.pm, ln177] Got ERROR: at /usr/bin/masterha_master_switch line 53. Tue Oct 29 19:40:07 2019 - [debug] Already disconnected from 172.19.14.21(172.19.14.21:3306) Tue Oct 29 19:40:07 2019 - [debug] Disconnected from 172.19.14.23(172.19.14.23:3306) Tue Oct 29 19:40:07 2019 - [debug] Disconnected from 172.19.14.22(172.19.14.22:3306)

lixiaofeng1026 commented 4 years ago

The mysqlmha manual failover is a success。 [root@mysql-manager-23 conf]# masterha_master_switch --conf=/usr/local/mha-helper-master/conf/manager.conf --master_state=dead --dead_master_host=172.19.14.23

[root@mysql-manager-23 support-files]# masterha_check_repl --conf=/usr/local/mha-helper-master/conf/manager.conf Tue Oct 29 19:52:10 2019 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping. Tue Oct 29 19:52:10 2019 - [info] Reading application default configuration from /usr/local/mha-helper-master/conf/manager.conf.. Tue Oct 29 19:52:10 2019 - [info] Reading server configuration from /usr/local/mha-helper-master/conf/manager.conf.. Tue Oct 29 19:52:10 2019 - [info] MHA::MasterMonitor version 0.58. Tue Oct 29 19:52:10 2019 - [debug] Connecting to servers.. Tue Oct 29 19:52:11 2019 - [debug] Connected to: 172.19.14.21(172.19.14.21:3306), user=root Tue Oct 29 19:52:11 2019 - [debug] Number of slave worker threads on host 172.19.14.21(172.19.14.21:3306): 0 Tue Oct 29 19:52:11 2019 - [debug] Connected to: 172.19.14.23(172.19.14.23:3306), user=root Tue Oct 29 19:52:11 2019 - [debug] Number of slave worker threads on host 172.19.14.23(172.19.14.23:3306): 0 Tue Oct 29 19:52:11 2019 - [debug] Connected to: 172.19.14.22(172.19.14.22:3306), user=root Tue Oct 29 19:52:11 2019 - [debug] Number of slave worker threads on host 172.19.14.22(172.19.14.22:3306): 0 Tue Oct 29 19:52:11 2019 - [debug] Comparing MySQL versions.. Tue Oct 29 19:52:11 2019 - [debug] Comparing MySQL versions done. Tue Oct 29 19:52:11 2019 - [debug] Connecting to servers done. Tue Oct 29 19:52:11 2019 - [info] GTID failover mode = 0 Tue Oct 29 19:52:11 2019 - [info] Dead Servers: Tue Oct 29 19:52:11 2019 - [info] Alive Servers: Tue Oct 29 19:52:11 2019 - [info] 172.19.14.21(172.19.14.21:3306) Tue Oct 29 19:52:11 2019 - [info] 172.19.14.23(172.19.14.23:3306) Tue Oct 29 19:52:11 2019 - [info] 172.19.14.22(172.19.14.22:3306) Tue Oct 29 19:52:11 2019 - [info] Alive Slaves: Tue Oct 29 19:52:11 2019 - [info] 172.19.14.23(172.19.14.23:3306) Version=8.0.12 (oldest major version between slaves) log-bin:enabled Tue Oct 29 19:52:11 2019 - [debug] Relay log info repository: TABLE Tue Oct 29 19:52:11 2019 - [info] Replicating from 172.19.14.21(172.19.14.21:3306) Tue Oct 29 19:52:11 2019 - [info] Primary candidate for the new Master (candidate_master is set) Tue Oct 29 19:52:11 2019 - [info] 172.19.14.22(172.19.14.22:3306) Version=8.0.12 (oldest major version between slaves) log-bin:enabled Tue Oct 29 19:52:11 2019 - [debug] Relay log info repository: TABLE Tue Oct 29 19:52:11 2019 - [info] Replicating from 172.19.14.21(172.19.14.21:3306) Tue Oct 29 19:52:11 2019 - [info] Not candidate for the new Master (no_master is set) Tue Oct 29 19:52:11 2019 - [info] Current Alive Master: 172.19.14.21(172.19.14.21:3306) Tue Oct 29 19:52:11 2019 - [info] Checking slave configurations.. Tue Oct 29 19:52:11 2019 - [info] Checking replication filtering settings.. Tue Oct 29 19:52:11 2019 - [info] binlog_do_db= apolloconfigdb,apolloportaldb, binlog_ignore_db= information_schema,mysql,performance_schema,sys Tue Oct 29 19:52:11 2019 - [info] Replication filtering check ok. Tue Oct 29 19:52:11 2019 - [info] GTID (with auto-pos) is not supported Tue Oct 29 19:52:11 2019 - [info] Starting SSH connection tests.. Tue Oct 29 19:52:12 2019 - [debug] Tue Oct 29 19:52:11 2019 - [debug] Connecting via SSH from root@172.19.14.21(172.19.14.21:11017) to root@172.19.14.23(172.19.14.23:11017).. Tue Oct 29 19:52:11 2019 - [debug] ok. Tue Oct 29 19:52:11 2019 - [debug] Connecting via SSH from root@172.19.14.21(172.19.14.21:11017) to root@172.19.14.22(172.19.14.22:11017).. Tue Oct 29 19:52:12 2019 - [debug] ok. Tue Oct 29 19:52:12 2019 - [debug] Tue Oct 29 19:52:11 2019 - [debug] Connecting via SSH from root@172.19.14.23(172.19.14.23:11017) to root@172.19.14.21(172.19.14.21:11017).. Tue Oct 29 19:52:12 2019 - [debug] ok. Tue Oct 29 19:52:12 2019 - [debug] Connecting via SSH from root@172.19.14.23(172.19.14.23:11017) to root@172.19.14.22(172.19.14.22:11017).. Tue Oct 29 19:52:12 2019 - [debug] ok. Tue Oct 29 19:52:13 2019 - [debug] Tue Oct 29 19:52:12 2019 - [debug] Connecting via SSH from root@172.19.14.22(172.19.14.22:11017) to root@172.19.14.21(172.19.14.21:11017).. Tue Oct 29 19:52:12 2019 - [debug] ok. Tue Oct 29 19:52:12 2019 - [debug] Connecting via SSH from root@172.19.14.22(172.19.14.22:11017) to root@172.19.14.23(172.19.14.23:11017).. Tue Oct 29 19:52:13 2019 - [debug] ok. Tue Oct 29 19:52:13 2019 - [info] All SSH connection tests passed successfully. Tue Oct 29 19:52:13 2019 - [info] Checking MHA Node version.. Tue Oct 29 19:52:14 2019 - [info] Version check ok. Tue Oct 29 19:52:14 2019 - [info] Checking SSH publickey authentication settings on the current master.. Tue Oct 29 19:52:14 2019 - [debug] SSH connection test to 172.19.14.21, option -o StrictHostKeyChecking=no -o PasswordAuthentication=no -o BatchMode=yes -o ConnectTimeout=5 -i /root/.ssh/id_rsa, timeout 5 Tue Oct 29 19:52:14 2019 - [info] HealthCheck: SSH to 172.19.14.21 is reachable. Tue Oct 29 19:52:14 2019 - [info] Master MHA Node version is 0.58. Tue Oct 29 19:52:14 2019 - [info] Checking recovery script configurations on 172.19.14.21(172.19.14.21:3306).. Tue Oct 29 19:52:14 2019 - [info] Executing command: save_binary_logs --command=test --start_pos=4 --binlog_dir=/var/mysql/data --output_file=/var/mysqlha/save_binary_logs_test --manager_version=0.58 --start_file=mysql-bin-9501.000008 --debug
Tue Oct 29 19:52:14 2019 - [info] Connecting to root@172.19.14.21(172.19.14.21:11017).. Creating /var/mysqlha if not exists.. ok. Checking output directory is accessible or not.. ok. Binlog found at /var/mysql/data, up to mysql-bin-9501.000008 Tue Oct 29 19:52:15 2019 - [info] Binlog setting check done. Tue Oct 29 19:52:15 2019 - [info] Checking SSH publickey authentication and checking recovery script configurations on all alive slave servers.. Tue Oct 29 19:52:15 2019 - [info] Executing command : apply_diff_relay_logs --command=test --slave_user='root' --slave_host=172.19.14.23 --slave_ip=172.19.14.23 --slave_port=3306 --workdir=/var/mysqlha --target_version=8.0.12 --manager_version=0.58 --relay_dir=/var/mysql/log --current_relay_log=mysql-relay-bin.000002 --debug --ssh_options='-o ServerAliveInterval=60 -o ServerAliveCountMax=20 -o StrictHostKeyChecking=no -o ConnectionAttempts=5 -o PasswordAuthentication=no -o BatchMode=yes -i /root/.ssh/id_rsa' --slave_pass=xxx Tue Oct 29 19:52:15 2019 - [info] Connecting to root@172.19.14.23(172.19.14.23:11017).. Checking slave recovery environment settings.. Relay log found at /var/mysql/log, up to mysql-relay-bin.000002 Temporary relay log file is /var/mysql/log/mysql-relay-bin.000002 Checking if super_read_only is defined and turned on.. not present or turned off, ignoring. Testing mysql connection and privileges.. mysql: [Warning] Using a password on the command line interface can be insecure. done. Testing mysqlbinlog output.. done. Cleaning up test file(s).. done. Tue Oct 29 19:52:15 2019 - [info] Executing command : apply_diff_relay_logs --command=test --slave_user='root' --slave_host=172.19.14.22 --slave_ip=172.19.14.22 --slave_port=3306 --workdir=/var/mysqlha --target_version=8.0.12 --manager_version=0.58 --relay_dir=/var/mysql/log --current_relay_log=mysql-relay-bin.000002 --debug --ssh_options='-o ServerAliveInterval=60 -o ServerAliveCountMax=20 -o StrictHostKeyChecking=no -o ConnectionAttempts=5 -o PasswordAuthentication=no -o BatchMode=yes -i /root/.ssh/id_rsa' --slave_pass=xxx Tue Oct 29 19:52:15 2019 - [info] Connecting to root@172.19.14.22(172.19.14.22:11017).. Checking slave recovery environment settings.. Relay log found at /var/mysql/log, up to mysql-relay-bin.000002 Temporary relay log file is /var/mysql/log/mysql-relay-bin.000002 Checking if super_read_only is defined and turned on.. not present or turned off, ignoring. Testing mysql connection and privileges.. mysql: [Warning] Using a password on the command line interface can be insecure. done. Testing mysqlbinlog output.. done. Cleaning up test file(s).. done. Tue Oct 29 19:52:16 2019 - [info] Slaves settings check done. Tue Oct 29 19:52:16 2019 - [info] 172.19.14.21(172.19.14.21:3306) (current master) +--172.19.14.23(172.19.14.23:3306) +--172.19.14.22(172.19.14.22:3306)

Tue Oct 29 19:52:16 2019 - [info] Checking replication health on 172.19.14.23.. Tue Oct 29 19:52:16 2019 - [info] ok. Tue Oct 29 19:52:16 2019 - [info] Checking replication health on 172.19.14.22.. Tue Oct 29 19:52:16 2019 - [info] ok. Tue Oct 29 19:52:16 2019 - [info] Checking master_ip_failover_script status: Tue Oct 29 19:52:16 2019 - [info] /usr/local/mha-helper-master/scripts/master_ip_failover --command=status --ssh_user=root --orig_master_host=172.19.14.21 --orig_master_ip=172.19.14.21 --orig_master_port=3306 --orig_master_ssh_port=11017 --ssh_options='-o ServerAliveInterval=60 -o ServerAliveCountMax=20 -o StrictHostKeyChecking=no -o ConnectionAttempts=5 -o PasswordAuthentication=no -o BatchMode=yes -i /root/.ssh/id_rsa' [2019-10-29 19:52:16] Doing sanity checks Tue Oct 29 19:52:16 2019 - [info] OK. Tue Oct 29 19:52:16 2019 - [warning] shutdown_script is not defined. Tue Oct 29 19:52:16 2019 - [debug] Disconnected from 172.19.14.21(172.19.14.21:3306) Tue Oct 29 19:52:16 2019 - [debug] Disconnected from 172.19.14.23(172.19.14.23:3306) Tue Oct 29 19:52:16 2019 - [debug] Disconnected from 172.19.14.22(172.19.14.22:3306) Tue Oct 29 19:52:16 2019 - [info] Got exit code 0 (Not master dead).

MySQL Replication Health is OK.

lixiaofeng1026 commented 4 years ago

--orig_master_ssh_user=(from 0.56, current master's ssh user) --orig_master_is_new_slave=(from 0.56, notifying whether the orig master will be new slave or not) --new_master_ssh_user=(from 0.56, new master's ssh user)