Feature request: avoid hard links in purge_relay_logs

[GoogleCodeExporter]

Right now purge_relay_logs creates hard links to relay log files before 
enabling log purge in slave mysqlds to delay physical file deletion and so to 
speed up the mysqld side of the purge.

This fails if the directory the links are created in is not on the same file 
system / mount point as the relay log files. A different --workdir needs to be 
given in this case. purge_relay_log also needs to take care of removing the 
hard links, both after mysqld purge completed, and before creating links as 
there may be stale ones from an incomplete previous run.

An alternative approach to delay physical file removal could be to open all 
relay log files for reading form within purge_relay_logs instead of using hard 
links. This would also delay physical file removal, even with the last links to 
the files inodes gone after the mysqld purge, until the last open handles to 
the files are closed.

Advantages: 

* no need to worry about file system boundaries
* no need to clean up after incomplete runs as file handles are closed and 
files are removed on process termination anyway

Original issue reported on code.google.com by hartmut....@gmail.com on 11 Sep 2013 at 7:17

[GoogleCodeExporter]

The feature request makes sense though there are many ways to make purging 
relay logs faster (i.e. using xfs/ext4 for storing binlogs).

Here is a preliminary patch. --use_hardlink=0 (default is 1: same as current 
behavior) makes purge_relay_logs open relay log files until the script ends.

If you have a test environment, please try the patch and give me your feedback.

-----------------------------------------
diff --git a/bin/purge_relay_logs b/bin/purge_relay_logs
index e07bee7..04e6647 100755
--- a/bin/purge_relay_logs
+++ b/bin/purge_relay_logs
@@ -43,6 +43,7 @@ GetOptions(
     host=s
     port=i
     disable_relay_log_purge
+    use_hardlink=i
     /,
 ) or pod2usage(1);
 $opt{workdir}  ||= "/var/tmp";
@@ -50,11 +51,13 @@ $opt{user}     ||= "root";
 $opt{password} ||= "";
 $opt{host}     ||= "127.0.0.1";
 $opt{port}     ||= 3306;
+$opt{use_hardlink} ||= 1;

 $| = 1;

 my $_binlog_manager;
 my $_relay_log_info_path = $opt{relay_log_info};
+my @relay_log_files_handle= ();

 if ( $opt{help} ) {
   pod2usage(0);
@@ -67,6 +70,20 @@ if ( $opt{version} ) {

 exit &main();

+sub open_relay_logs() {
+  my $relay_dir = $_binlog_manager->{dir};
+  my @files =
+    MHA::BinlogManager::get_all_binlogs_from_prefix( 
$_binlog_manager->{prefix},
+    $relay_dir );
+  foreach my $relay_log_basename (@files) {
+    print(
+" Opening $relay_dir/$relay_log_basename ..\n"
+    );
+    open(my $fh, "<", "$relay_dir/$relay_log_basename") or croak $!;
+    push @relay_log_files_handle, $fh;
+  }
+}
+
 sub hardlink_relay_logs($) {
   my $current_relay_log = shift;

@@ -204,9 +221,12 @@ sub main {
       $current_relay_log = $_binlog_manager->{cur_log};
     }

-    remove_hardlinked_relay_logs();
-
-    croak if ( hardlink_relay_logs($current_relay_log) );
+    if ($opt{use_hard_link}) {
+      remove_hardlinked_relay_logs();
+      croak if ( hardlink_relay_logs($current_relay_log) );
+    } else {
+      open_relay_logs();
+    }

     if ( MHA::SlaveUtil::get_failover_advisory_lock( $dbh, 200 ) ) {
       croak
@@ -219,7 +239,9 @@ sub main {
     print " ok.\n";

     MHA::SlaveUtil::release_failover_advisory_lock($dbh);
-    remove_hardlinked_relay_logs();
+    if ($opt{use_hard_link}) {
+      remove_hardlinked_relay_logs();
+    }
     my $end = MHA::NodeUtil::current_time();
     printf("$end: All relay log purging operations succeeded.\n");
     $exit_code = 0;

Original comment by Yoshinor...@gmail.com on 11 Sep 2013 at 7:58

• Changed state: Accepted
• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Committed to the github branch (https://github.com/yoshinorim/mha4mysql-node).

Original comment by Yoshinor...@gmail.com on 16 Sep 2013 at 6:41

• Changed state: Done
• Added labels: ****
• Removed labels: ****