Support direct extract of MySQL binlog across network

[GoogleCodeExporter]

1. To which tool/application/daemon will this feature apply?

Tungsten Replicator. 

2. Describe the feature in general

Tungsten currently downloads binlog contents to on-disk files (aka Tungten 
relay logs) when operating in relay extraction mode.   This behavior is 
analogous to the MySQL relay log, except that the downloaded files are 
extracted to the THL.  Using extra files induces complexity at restart, creates 
possible error conditions like running out of space for downloaded files, and 
degrades throughput in systems that are already I/O bound.  

Tungsten will be upgraded as follows to eliminate intermediate on-disk relay 
logs.  

a. Binlogs will be downloaded as a stream that is fed directly to the 
extractor.  The stream will be structured to preserve the binlog file name and 
log position as with a real file. 

b. In the event that the connection to the master is lost, Tungsten will go 
into the ONLINE-SYNCHRONIZING state until it can reconnect to the MySQL master. 

c. Downloading to local relay log files will continue to be supported.   There 
will be a switch to choose the relay log method.  Either approach generates 
identical THL records.   

3. Describe the feature interface

This behavior will be controlled by a property setting on the MySQLExtractor in 
the replication service property file.  It will require a corresponding 
tungsten-installer option.  

4. Give an idea (if applicable) of a possible implementation

This is an extension to the MySQL extractor, specifically a new interface to 
the RelayLogClient class. 

5. Describe pros and cons of this feature.

5a. Why the world will be a better place with this feature.

Described above. 

5b. What hardship will the human race have to endure if this feature is
implemented.

If there is any latency writing to the THL it will cause Tungsten to become 
slower at downloading transactions from the master.  This in turn could 
increase data loss on failover. 

6. Notes

Original issue reported on code.google.com by berkeley...@gmail.com on 3 Jul 2011 at 11:34

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Original comment by g.maxia on 5 Jul 2011 at 3:45

[GoogleCodeExporter]

The '--native-slave-takeover' property has been added for the direct 
replication tungsten-installer command.

Original comment by jeffm...@gmail.com on 5 Jul 2011 at 8:25

• Changed state: Fixed

[GoogleCodeExporter]

Reopened.  There was confusion between this issue and 151, which involves MySQL 
slave takeover. This issue will be implemented later. 

Original comment by berkeley...@gmail.com on 5 Jul 2011 at 9:11

• Changed state: Accepted

[GoogleCodeExporter]

Original comment by berkeley...@gmail.com on 8 Sep 2011 at 5:17

• Added labels: FixedIn-2.0.5

[GoogleCodeExporter]

Original comment by berkeley...@gmail.com on 8 Sep 2011 at 5:18

• Removed labels: FixedIn-2.0.6

[GoogleCodeExporter]

Original comment by jeffm...@gmail.com on 14 Oct 2011 at 7:54

• Changed state: Fixed