search

osheroff / mysql-binlog-connector-java

MySQL Binary Log connector
680 stars 167 forks source link

Could not parse the table map event when mysql version > 8.0.18 #104

Open ruanhang1993 opened 1 year ago

ruanhang1993 commented 1 year ago

We got NullPointerExceptiom when parsing the binlog file.

Caused by: java.lang.NullPointerException
    at com.github.shyiko.mysql.binlog.event.deserialization.TableMapEventDataDeserializer.readMetadata(TableMapEventDataDeserializer.java:81)
    at com.github.shyiko.mysql.binlog.event.deserialization.TableMapEventDataDeserializer.deserialize(TableMapEventDataDeserializer.java:42)
    at com.github.shyiko.mysql.binlog.event.deserialization.TableMapEventDataDeserializer.deserialize(TableMapEventDataDeserializer.java:27)
    at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeEventData(EventDeserializer.java:303)
    at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeTableMapEventData(EventDeserializer.java:281)
    at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.nextEvent(EventDeserializer.java:228)
    at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$1.nextEvent(MySqlStreamingChangeEventSource.java:259)
    at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:952)
    ... 3 more

After searching, I find it is a new type MYSQL_TYPE_TYPED_ARRAY(https://dev.mysql.com/doc/dev/mysql-server/latest/classbinary__log_1_1Table__map__event.html), and it has been changed in https://github.com/mysql/mysql-server/commit/9082b6a820f3948fd563cc32a050f5e8775f2855.

I want to know how to store this metadata in an int array when parsing in TableMapEventDataDeserializer. The MYSQL_TYPE_TYPED_ARRAY will have two parts metadata: the type of the array and the metadata of the type.