TableMapEventMetadata Signedness Parsed Incorrectly

[jmlw]

Parsing binlog events for MySQL 8.0.29+ (may occur in older versions as well) result in unexpected signedness bit when parsing INT column following a YEAR column.

CREATE TABLE `demo1` (`primary_key` INT NOT NULL, `year_col` YEAR, `int_col` INT);
INSERT INTO `demo1` (`primary_key`, `year_col`, `int_col`) VALUES (123, 2041, 8489734);

CREATE TABLE `demo2` (`year_col` YEAR, `int_col` INT, `primary_key` INT NOT NULL);
INSERT INTO `demo2` (`year_col`, `int_col`, `primary_key`) VALUES (2041, 8489734, 123);

CREATE TABLE `demo3` (`primary_key` INT NOT NULL, `int_col` INT, `year_col` YEAR);
INSERT INTO `demo3` (`primary_key`, `int_col`, `year_col`) VALUES (123, 8489734, 2041);

The sql above results in table map event metadata which indicates column int_col is unsigned in demo1 and int_col is unsigned in demo2 table, but demo3 table does not indicate any column is unsigned.

The binlog that shows this is here as a hexdump: https://gist.github.com/jmlw/2b318ae6090ab28f8c3fb6b86bd57f62

[hongwei366]

the same question in mysql v5.7.1

[jpechane]

Hi, I believe this is fixed via https://github.com/osheroff/mysql-binlog-connector-java/pull/114

I come from other side of the problem DBZ-6413. As the parsing is done incorrectly then with certain combination and count of columns the parsing gets out of sync and fails completely

See

CREATE TABLE `table1` (`c1` int(20) NOT NULL AUTO_INCREMENT, `c2` year, `c3` int, `c4` int, `c5` int, `c6` int, `c7` int, `c8` int, `c9` int, c10 char(10), PRIMARY KEY (`c1`));
INSERT INTO `table1` (c1, c2, c3, c4, c5, c6, c7, c8, c9, c10) VALUES (1, 2, 3, 4, 5, 6, 7, 8, 9, '');

The result is

    ... 3 common frames omitted
Caused by: java.io.IOException: Unsupported table metadata field type 0
    at com.github.shyiko.mysql.binlog.event.deserialization.TableMapEventMetadataDeserializer.deserialize(TableMapEventMetadataDeserializer.java:52)
    at com.github.shyiko.mysql.binlog.event.deserialization.TableMapEventDataDeserializer.deserialize(TableMapEventDataDeserializer.java:47)
    at com.github.shyiko.mysql.binlog.event.deserialization.TableMapEventDataDeserializer.deserialize(TableMapEventDataDeserializer.java:27)
    at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeEventData(EventDeserializer.java:335)
    ... 7 common frames omitted

[keweishang]

Thanks for the investigation and fix @jpechane 👍

[jmlw]

Closing as https://github.com/osheroff/mysql-binlog-connector-java/pull/114 appears to correct the issue