masterha_check_repl crashing

[GoogleCodeExporter]

I am trying to run this script as a non-root user (ie mhadmin). i am using 
simple password like for mysql user without any special chars.

[mhadmin@test4 bin]# rpm -qa |grep -i mha
mha4mysql-node-0.56-0.el6.noarch
mha4mysql-manager-0.56-0.el6.noarch

[mhadmin@test4 bin]# masterha_check_repl --conf=/etc/mha.conf 
Mon Jul 20 15:45:14 2015 - [warning] Global configuration file 
/etc/masterha_default.cnf not found. Skipping.
Mon Jul 20 15:45:14 2015 - [info] Reading application default configuration 
from /etc/mha.conf..
Mon Jul 20 15:45:14 2015 - [info] Reading server configuration from 
/etc/mha.conf..
Mon Jul 20 15:45:14 2015 - [info] MHA::MasterMonitor version 0.56.
Mon Jul 20 15:45:15 2015 - [info] GTID failover mode = 0
Mon Jul 20 15:45:15 2015 - [info] Dead Servers:
Mon Jul 20 15:45:15 2015 - [info] Alive Servers:
Mon Jul 20 15:45:15 2015 - [info] test1(192.168.1.1:3306)
Mon Jul 20 15:45:15 2015 - [info] test2(192.168.1.2:3306)
Mon Jul 20 15:45:15 2015 - [info] test3(192.168.1.3:3306)
Mon Jul 20 15:45:15 2015 - [info] Alive Slaves:
Mon Jul 20 15:45:15 2015 - [info] test2(192.168.1.2:3306) 
Version=5.6.19-67.0-log (oldest major version between slaves) log-bin:enabled
Mon Jul 20 15:45:15 2015 - [info] Replicating from 192.168.1.1(192.168.1.1:3306)
Mon Jul 20 15:45:15 2015 - [info] Primary candidate for the new Master 
(candidate_master is set)
Mon Jul 20 15:45:15 2015 - [info] test3(192.168.1.3:3306) 
Version=5.6.19-67.0-log (oldest major version between slaves) log-bin:enabled
Mon Jul 20 15:45:15 2015 - [info] Replicating from 192.168.1.1(192.168.1.1:3306)
Mon Jul 20 15:45:15 2015 - [info] Primary candidate for the new Master 
(candidate_master is set)
Mon Jul 20 15:45:15 2015 - [info] Current Alive Master: test1(192.168.1.1:3306)
Mon Jul 20 15:45:15 2015 - [info] Checking slave configurations..
Mon Jul 20 15:45:15 2015 - [info] read_only=1 is not set on slave 
test2(192.168.1.2:3306).
Mon Jul 20 15:45:15 2015 - [warning] relay_log_purge=0 is not set on slave 
test2(192.168.1.2:3306).
Mon Jul 20 15:45:15 2015 - [info] read_only=1 is not set on slave 
test3(192.168.1.3:3306).
Mon Jul 20 15:45:15 2015 - [warning] relay_log_purge=0 is not set on slave 
test3(192.168.1.3:3306).
Mon Jul 20 15:45:15 2015 - [info] Checking replication filtering settings..
Mon Jul 20 15:45:15 2015 - [info] binlog_do_db= , binlog_ignore_db= 
Mon Jul 20 15:45:15 2015 - [info] Replication filtering check ok.
Mon Jul 20 15:45:15 2015 - [info] GTID (with auto-pos) is not supported
Mon Jul 20 15:45:15 2015 - [info] Starting SSH connection tests..
Mon Jul 20 15:45:16 2015 - [info] All SSH connection tests passed successfully.
Mon Jul 20 15:45:16 2015 - [info] Checking MHA Node version..
Mon Jul 20 15:45:16 2015 - [info] Version check ok.
Mon Jul 20 15:45:16 2015 - [info] Checking SSH publickey authentication 
settings on the current master..
Warning:
This a private computing system that is restricted to authorized users.
Any actual or attempted unauthorized use of this computer system is
prohibited and may result in criminal and/or civil prosecution. By
continuing, you acknowledge your activity on this system may be viewed,
monitored or recorded without notice or permission.
Mon Jul 20 15:45:17 2015 - [info] HealthCheck: SSH to test1 is reachable.
Mon Jul 20 15:45:17 2015 - [info] Master MHA Node version is 0.56.
Mon Jul 20 15:45:17 2015 - [info] Checking recovery script configurations on 
test1(192.168.1.1:3306)..
Mon Jul 20 15:45:17 2015 - [info] Executing command: save_binary_logs 
--command=test --start_pos=4 --binlog_dir=/var/log/mysql/logs/binlogs 
--output_file=/tmp/mha/save_binary_logs_test --manager_version=0.56 
--start_file=test1-mysql-bin.000001 
Mon Jul 20 15:45:17 2015 - [info] Connecting to mhadmin@192.168.1.1(test1:22)..
Warning:
This a private computing system that is restricted to authorized users.
Any actual or attempted unauthorized use of this computer system is
prohibited and may result in criminal and/or civil prosecution. By
continuing, you acknowledge your activity on this system may be viewed,
monitored or recorded without notice or permission.
Creating /tmp/mha if not exists.. ok.
Checking output directory is accessible or not..
ok.
Binlog found at /var/log/mysql/logs/binlogs, up to test1-mysql-bin.000001
Mon Jul 20 15:45:17 2015 - [info] Binlog setting check done.
Mon Jul 20 15:45:17 2015 - [info] Checking SSH publickey authentication and 
checking recovery script configurations on all alive slave servers..
Mon Jul 20 15:45:17 2015 - [info] Executing command : apply_diff_relay_logs 
--command=test --slave_user='root' --slave_host=test2 --slave_ip=192.168.1.2 
--slave_port=3306 --workdir=/tmp/mha --target_version=5.6.19-67.0-log 
--manager_version=0.56 --relay_log_info=/var/lib/mysql/relay-log.info 
--relay_dir=/var/lib/mysql/ --slave_pass=xxx
Mon Jul 20 15:45:17 2015 - [info] Connecting to mhadmin@192.168.1.2(test2:22)..
Warning:
This a private computing system that is restricted to authorized users.
Any actual or attempted unauthorized use of this computer system is
prohibited and may result in criminal and/or civil prosecution. By
continuing, you acknowledge your activity on this system may be viewed,
monitored or recorded without notice or permission.
Checking slave recovery environment settings..
Opening /var/lib/mysql/relay-log.info ... ok.
Relay log found at /var/log/mysql/logs/relaylogs, up to test2-relay-log.000002
Temporary relay log file is /var/log/mysql/logs/relaylogs/test2-relay-log.000002
Testing mysql connection and privileges..Warning: Using a password on the 
command line interface can be insecure.
ERROR 1045 (28000): Access denied for user 'root'@'test2' (using password: YES)
mysql command failed with rc 1:0!
at /usr/bin/apply_diff_relay_logs line 375
main::check() called at /usr/bin/apply_diff_relay_logs line 497
eval {...} called at /usr/bin/apply_diff_relay_logs line 475
main::main() called at /usr/bin/apply_diff_relay_logs line 120
Mon Jul 20 15:45:17 2015 - 
[error][/usr/share/perl5/vendor_perl/MHA/MasterMonitor.pm, ln205] Slaves 
settings check failed!
Mon Jul 20 15:45:17 2015 - 
[error][/usr/share/perl5/vendor_perl/MHA/MasterMonitor.pm, ln413] Slave 
configuration failed.
Mon Jul 20 15:45:17 2015 - 
[error][/usr/share/perl5/vendor_perl/MHA/MasterMonitor.pm, ln424] Error 
happened on checking configurations. at /usr/bin/masterha_check_repl line 48
Mon Jul 20 15:45:17 2015 - 
[error][/usr/share/perl5/vendor_perl/MHA/MasterMonitor.pm, ln523] Error 
happened on monitoring servers.
Mon Jul 20 15:45:17 2015 - [info] Got exit code 1 (Not master dead).
MySQL Replication Health is NOT OK!

Original issue reported on code.google.com by ashok.ji...@gmail.com on 20 Jul 2015 at 11:00

[GoogleCodeExporter]

more info on this

1. mha.conf file 

[mhadmin@test4 ~] cat /etc/mha.conf

[server default]
user=root
password=xxxx
ssh_user=mhadmin
manager_workdir = /tmp/mha
remote_workdir = /tmp/mha
manager_log = /tmp/mha/mha.log
master_binlog_dir = /var/log/mysql/logs/binlogs
[server1]
hostname=test1
candidate_master=1
[server2]
hostname=test2
candidate_master=1
[server3]
hostname=test3
candidate_master=1

2. 'root' user exists on host test2 
root@test2 [(none)]> select user,host from mysql.user where user = 'root';
+---------+-----------+
| user | host |
+---------+-----------+
| root | % |
| root | localhost |
+---------+-----------+

3. able to connect to mysql locally on test2

[mhadmin@test2 ~]# mysql -uroot -p
Enter password: 
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 99
Server version: 5.6.19-67.0-log Percona Server (GPL), Release 67.0, Revision 618
Copyright (c) 2009-2014 Percona LLC and/or its affiliates
Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
root@test2 [(none)]>

Original comment by ashok.ji...@gmail.com on 21 Jul 2015 at 11:13