linkedin / databus

Source-agnostic distributed change data capture system
Apache License 2.0
3.64k stars 735 forks source link

process transaction produced by mysql trigger #66

Closed andyqzb closed 9 years ago

andyqzb commented 9 years ago

I found the binlog produced by mysql trigger don't like the style the relay assumed. The relay will have problems when processing it. So I make this patch to process the mysql trigger style binlog.

The open replicator event producer assume the binlog style in transaction likes below(a and b are two tables in mysql): begin table map event for a; update or insert event on a; table map event for b; update or insert event on b; commit; While, the binlog event produced by mysql trigger is not in this style. For example, a trigger likes below: When update some rows in table a, to update some rows in table b. The binlog produced by this trigger likes below: begin; table map event for a; table map event for b; update event in a; update event in b; commit;

So when the relay received the update event in a, It will assume its the update event for b, then the relay will find there is a mismatch between the schema of b and the event of a. In the patch, I will store all the table map event in a map(cleared when binlog rotated). When the relay received the update or insert event, it will find its table map event from the map, then process the binlog event with its really table schema.

Another, the relay will also have problems when a transaction has two sqls in sequence modify the same table. For example, if a user write a transaction likes below: begin; update some rows in a; insert some rows in a; commit; The binlog produced by this tracsaction likes below: begin; table map event for a; update event in a; table map event for a; insert event in a; commit;

While, the relay assume that two table map events in sequence must be for different tables. Then it will throw a exception. So I removed the table changed check in the patch when received a new table map event.

chavdar commented 9 years ago

LGTM