Failed: Starting master failover

[GoogleCodeExporter]

What steps will reproduce the problem?
1. masterha_check_ssh:   OK(No Error)
2. masterha_check_repl:  OK(No Error)
3. masterha_manager:     OK(No Error. End of filed is "Ping Succeeded, ...)
4. Master node is down(shutdown -r now)
5. Failed: Starting master failover.

Output with bellow:

---------------------
Wed Jun 20 12:08:44 2012 - [info] Starting ping health check on 
10.1.10.80(10.1.10.80:3306)..
Wed Jun 20 12:08:44 2012 - [info] Ping succeeded, sleeping until it doesn't 
respond..
         : 
         :  Master node is down(shutdown -r now)
         :
Wed Jun 20 12:10:02 2012 - [warning] Got error on MySQL ping: 2006 (MySQL 
server has gone away)
ssh: connect to host 10.1.10.80 port 22: Connection refused
Wed Jun 20 12:10:02 2012 - [warning] HealthCheck: SSH to 10.1.10.80 is NOT 
reachable.
Wed Jun 20 12:10:08 2012 - [warning] Got error on MySQL connect: 2003 (Can't 
connect to MySQL server on '10.1.10.80' (4))
Wed Jun 20 12:10:08 2012 - [warning] Connection failed 1 time(s)..
Wed Jun 20 12:10:11 2012 - [warning] Got error on MySQL connect: 2003 (Can't 
connect to MySQL server on '10.1.10.80' (4))
Wed Jun 20 12:10:11 2012 - [warning] Connection failed 2 time(s)..
Wed Jun 20 12:10:14 2012 - [warning] Got error on MySQL connect: 2003 (Can't 
connect to MySQL server on '10.1.10.80' (4))
Wed Jun 20 12:10:14 2012 - [warning] Connection failed 3 time(s)..
Wed Jun 20 12:10:14 2012 - [warning] Master is not reachable from health 
checker!
Wed Jun 20 12:10:14 2012 - [warning] Master 10.1.10.80(10.1.10.80:3306) is not 
reachable!
Wed Jun 20 12:10:14 2012 - [warning] SSH is NOT reachable.
Wed Jun 20 12:10:14 2012 - [info] Connecting to a master server failed. Reading 
configuration file /etc/masterha_default.cnf and /etc/mha_manager/app1.cnf 
again, and trying to connect to all servers to check server status..
Wed Jun 20 12:10:14 2012 - [warning] Global configuration file 
/etc/masterha_default.cnf not found. Skipping.
Wed Jun 20 12:10:14 2012 - [info] Reading application default configurations 
from /etc/mha_manager/app1.cnf..
Wed Jun 20 12:10:14 2012 - [info] Reading server configurations from 
/etc/mha_manager/app1.cnf..
Wed Jun 20 12:10:14 2012 - [info] Dead Servers:
Wed Jun 20 12:10:14 2012 - [info]   10.1.10.80(10.1.10.80:3306)
Wed Jun 20 12:10:14 2012 - [info] Alive Servers:
Wed Jun 20 12:10:14 2012 - [info]   10.1.10.81(10.1.10.81:3306)
Wed Jun 20 12:10:14 2012 - [info]   10.1.20.80(10.1.20.80:3306)
Wed Jun 20 12:10:14 2012 - [info] Alive Slaves:
Wed Jun 20 12:10:14 2012 - [info]   10.1.10.81(10.1.10.81:3306)  
Version=5.5.25-log (oldest major version between slaves) log-bin:enabled
Wed Jun 20 12:10:14 2012 - [info]     Replicating from 
10.1.10.80(10.1.10.80:3306)
Wed Jun 20 12:10:14 2012 - [info]   10.1.20.80(10.1.20.80:3306)  
Version=5.5.25-log (oldest major version between slaves) log-bin:enabled
Wed Jun 20 12:10:14 2012 - [info]     Replicating from 
10.1.10.80(10.1.10.80:3306)
Wed Jun 20 12:10:14 2012 - [info] Checking slave configurations..
Wed Jun 20 12:10:14 2012 - [warning]  read_only=1 is not set on slave 
10.1.10.81(10.1.10.81:3306).
Wed Jun 20 12:10:14 2012 - [warning]  relay_log_purge=0 is not set on slave 
10.1.10.81(10.1.10.81:3306).
Wed Jun 20 12:10:14 2012 - [warning]  read_only=1 is not set on slave 
10.1.20.80(10.1.20.80:3306).
Wed Jun 20 12:10:14 2012 - [warning]  relay_log_purge=0 is not set on slave 
10.1.20.80(10.1.20.80:3306).
Wed Jun 20 12:10:14 2012 - [info] Checking replication filtering settings..
Wed Jun 20 12:10:14 2012 - [info]  Replication filtering check ok.
Wed Jun 20 12:10:14 2012 - [info] Master is down!
Wed Jun 20 12:10:14 2012 - [info] Terminating monitoring script.
Wed Jun 20 12:10:14 2012 - [info] Got exit code 20 (Master dead).
Wed Jun 20 12:10:14 2012 - [warning] Global configuration file 
/etc/masterha_default.cnf not found. Skipping.
Wed Jun 20 12:10:14 2012 - [info] Reading application default configurations 
from /etc/mha_manager/app1.cnf..
Wed Jun 20 12:10:14 2012 - [info] Reading server configurations from 
/etc/mha_manager/app1.cnf..
Wed Jun 20 12:10:14 2012 - [info] MHA::MasterFailover version 0.52.
Wed Jun 20 12:10:14 2012 - [info] Starting master failover.
Wed Jun 20 12:10:14 2012 - 
[error][/usr/lib/perl5/site_perl/5.8.8/MHA/ManagerUtil.pm, ln158] Got ERROR: 
Use of uninitialized value in scalar chomp at 
/usr/lib/perl5/site_perl/5.8.8/MHA/ManagerConst.pm line 84.
---------------------

What is the expected output? What do you see instead?
1. Why this problem means.
2. Please tell me the workaround.

What version of the product are you using? On what operating system?
Manager: 
  - OS:  RHEL5.7 (2.6.18-274.el5)
  - MHA Manager: 0.52   ※This issue happend 0.53
  - MHA Node: 0.52
Node:(10.1.10.8[01], 10.1.20.80)  
  - OS:  RHEL5.7 (2.6.18-274.el5)
  - MHA Node: 0.52
  - MySQL 5.5.25

Original issue reported on code.google.com by gmt.t...@gmail.com on 20 Jun 2012 at 10:30

[GoogleCodeExporter]

I had the same problem and I cleared up the following things...

Comment out "use warnings FATAL => 'all';"  in ManagerConst.pm and 
MasterFailover.pm

I guess that warning level of Perl changed ???

Original comment by yuka0...@gmail.com on 29 Jun 2012 at 9:45

[GoogleCodeExporter]

Hi!

Would you please test the following patch to ManagerConst.pm then check whether 
the error disappears?

 our $log_fmt = sub {
   my %args = @_;
   my $msg  = $args{message};
+   $msg = "" unless($msg);
   if ( $args{level} eq "error" ) {
....

Original comment by Yoshinor...@gmail.com on 29 Jun 2012 at 5:42

[GoogleCodeExporter]

Original comment by Yoshinor...@gmail.com on 29 Jun 2012 at 5:42

• Changed state: Started
• Added labels: Priority-High
• Removed labels: Priority-Medium

[GoogleCodeExporter]

I changed the following things, and the error disappears.

 our $log_fmt = sub {
   my %args = @_;
   my $msg  = $args{message};
+   $msg = "" unless($msg);
+   #chomp $msg;
-    chomp $msg;
 if ( $args{level} eq "error" ) {
....

Original comment by yuka0...@gmail.com on 2 Jul 2012 at 6:28

[GoogleCodeExporter]

I committed the patch to development tree.

https://github.com/yoshinorim/mha4mysql-manager

Thanks for the bug report!

Original comment by Yoshinor...@gmail.com on 2 Jul 2012 at 7:01

• Changed state: Done