Shopify / ghostferry

The swiss army knife of live data migrations
https://shopify.github.io/ghostferry
MIT License
748 stars 70 forks source link

MariaDB binlog events differ from MySQL #296

Open HemeraOne opened 3 years ago

HemeraOne commented 3 years ago

Running the binlog_streamer from MariaDB seems to be a little bit different.

an error seems to occur:

PANI[0001] tried to advance to a zero log position: mysql-bin.047756 0 string  tag=source_binlog_streamer

I tracked it down to the following events: replication.TableMapEvent, replication.GenericEvent, *replication.QueryEvent

Which is documented to happen when a replica is unable to understand MariaDB's protocol: "Dummy event replacing event type X that slave cannot handle."

To me it looks like it comes from the following events: ANNOTATE_ROWS_EVENT (https://mariadb.com/kb/en/annotate_rows_event/) As this is what I sniffed from the traffic:

# Dummy event replacing event type 160 that slave cannot handle

and the plain text "BEGIN" which both gets shoved in the replication.QueryEvent

Further more the also state that the TABLE_MAP_EVENT is used for row-based binlog: https://mariadb.com/kb/en/table_map_event/

Currently debugging for more data, will come with a PR that works with my current MariaDB: 10.3.28-MariaDB-1:10.3.28+maria~focal-log