Add support for MySQL 5.6 with binlog-checksum=NONE

[dentarg]

I'm not 100% sure about all details here, so I will write the complete story of our MySQL 5.5 -> 5.6 upgrade (5.5.46 -> 5.6.30 to be exact). Not sure if Ecco should handle all the cases.

We have one application called Bruce using Ecco the read the binlog from our MySQL server. We stopped Bruce before stopping writes to the MySQL sever (in hindsight, this was not so smart). So some events were written to the binlog by MySQL 5.5. We then upgraded to the server to MySQL 5.6, and started taking writes to the MySQL server again. Events were now written to the binlog by MySQL 5.6 (with binlog-checksum=CRC32).

We then started Bruce, but it stopped directly with the error: Ecco::Error::CommunicationError: Failed to deserialize data of EventHeaderV4{timestamp=1469174183000, eventType=TABLE_MAP, serverId=1100, headerLength=19, dataLength=43, nextPosition=538610156, flags=0}

Another not so smart thing we did then was to change binlog-checksum=CRC32 -> binlog-checksum=NONE for the MySQL 5.6 server. This didn't help us, so we had to skip the events until we reached events written by MySQL 5.6 (with binlog-checksum=CRC32).

Then Bruce stopped with error: Ecco::Error::CommunicationError: Slave can not handle replication events with the checksum that master is configured to log; the first event 'mysql-bin.003992' at 538610005, the last event read from './mysql-bin.003993' at 120, the last byte read from './mysql-bin.003993' at 120.

Events causing this error was written by MySQL 5.6 (with binlog-checksum=NONE). I think Ecco could handle the last error here once a version of mysql-binlog-connector-java with the following commits is released:

• https://github.com/shyiko/mysql-binlog-connector-java/commit/1f4fe856b4a0274a8946734f130ef146462fb493
• https://github.com/shyiko/mysql-binlog-connector-java/commit/671c7e8cebeec61487dabb0c12f728b51916753a

We got things going by changing back to binlog-checksum=CRC32 and skipping the problematic events.

[dentarg]

I think Ecco could handle the last error here once a version of mysql-binlog-connector-java with the following commits is released:

• shyiko/mysql-binlog-connector-java@1f4fe85
• shyiko/mysql-binlog-connector-java@671c7e8

These probably wont be released as the develop branch at shyiko/mysql-binlog-connector-java seems forgotten. :)

But just 6 days ago https://github.com/shyiko/mysql-binlog-connector-java/commit/68c0ec754c0e620b395e71c49bcdecd8c8a2b415 happened, which is released in 0.18.0, which we now use (https://github.com/twingly/ecco/pull/34). I asked in https://github.com/shyiko/mysql-binlog-connector-java/issues/256 if it is the same thing.

[dentarg]

I asked in shyiko/mysql-binlog-connector-java#256 if it is the same thing.

It is!

[walro]

But just 6 days ago shyiko/mysql-binlog-connector-java@68c0ec7 happened, which is released in 0.18.0, which we now use (#34). I asked in shyiko/mysql-binlog-connector-java#256 if it is the same thing.

In master we use a version later than 0.18.0, so we could check if we can now handle this.

[Chrizpy]

Not sure in which version this was solved, but we're now running version 0.23.2, which is a fork made on the original here: https://github.com/osheroff/mysql-binlog-connector-java

I tested Bruce with ecco version 0.7.0, and I got the same error described above when having sequences of CRC32 and NONE in the binlog. By just updating ecco to the latest version 0.8.0, this error no longer showed.

[walro]

Not sure in which version this was solved,

It seems like it was solved in 0.18.0, as mentioned in https://github.com/twingly/ecco/issues/22#issuecomment-458079229

... and that was included in the release of Ecco 0.8.0