Discrepancy in parsing results for the same event before and after adding a new field

[zj5220924]

Version

Please specify the versions you are using. Exact version numbers are preferred.

• Pymyrepl (0.45.1):
• Pymysql (1.1.0)
• OS (Centos 7):
• Database and version (Remove unnecessary options):
  • MySQL: 8.0.32

System Variables

List relevant system variables using the query SHOW VARIABLES LIKE '<variable_name_here>';

• log_bin: YES
• binlog_format: ROW
• binlog_row_image: FULL
• enforce_gtid_consistency: True
• gtid_mode: YES

If you are using MySQL 8.0.14 or later, please also provide the following variables:

• binlog_row_metadata: FULL
• binlog_row_value_options: Null

Symptoms

Insert a record into the test01 table. Parse this insertion statement using python-mysql-replication. Then, add a new field age after the is_del field in the test01 table. Once again, parse the initial insertion statement using python-mysql-replication. You will observe that the two parsed statements are different.

Steps to Reproduce

table structure: CREATE TABLE test01 ( ID bigint unsigned NOT NULL AUTO_INCREMENT, is_del longtext NOT NULL, name char(10) DEFAULT NULL, message varchar(100) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci DEFAULT NULL, PRIMARY KEY (ID) ) ENGINE=InnoDB AUTO_INCREMENT=27 DEFAULT CHARSET=utf8mb3;

step 1：

start 276 end 743 time 2024-01-15 04:57:31 gtid 54c86606-1b13-11ee-acab-005056960eee:16

{'values': {'UIDPK': 22, 'is_del': '0', 'name': None, 'message': 'xxxxxxx'}} step 2： alter table test01 add column age int after is_del; step 3：

start 276 end 743 time 2024-01-15 04:57:31 gtid 54c86606-1b13-11ee-acab-005056960eee:16

{'values': {'UIDPK': 22, 'is_del': '0', 'age': None, 'name': 'xxxxxxx'}}

I noticed that the parsing results for the same event in steps one and two are inconsistent. This event occurs before adding the age field, and it should not be parsed, leading to a misalignment between field names and values.

[sean-k1]

@zj5220924 Mysql 8.0xx users use latest python-mysql-replication Vesrion (1.0.x). 0.45.1 can not support column sync when column changed .

if you want more detail Information, check this Pr(https://github.com/julien-duponchelle/python-mysql-replication/pull/446)

[zj5220924]

@zj5220924 Mysql 8.0xx users use latest python-mysql-replication Vesrion (1.0.x). 0.45.1 can not support column sync when column changed .

if you want more detail Information, check this Pr(#446)

thanks for your reply, if i upgrade python-mnysql-replication to 1.0.6, the parsing results don't display normally, like below: {'values': {'UNKNOWN_COL0': 22, 'UNKNOWN_COL1': '0', 'UNKNOWN_COL2': None, 'UNKNOWN_COL3': 'xxxxxxx'}, 'none_sources': {'UNKNOWN_COL4': 'null'}}

[sean-k1]

@zj5220924 set varaible binlog_row_metadata= FULL and binlog_row_image=FULL If your data is prior to setting binlog_row_metadata: FULL, it will come out as UNKONWN_COLUMN.

[zj5220924]

@sean-k1 Thank you very much. As you mentioned, the parsing results now appear to be correct.