mysqldba-nandy
commented
1 month ago Canal的DDL解析使用的是正则表达式而不是标准的AST语法树，遇到单条SQL里面包括多表、多字段修改的DDL，很容易正则匹配失败，这是根本原因：https://github.com/alibaba/canal/blob/master/parse/src/main/java/com/alibaba/otter/canal/parse/inbound/mysql/ddl/SimpleDdlParser.java
ap0405140
运行环境: Windows10 MYSQL 8.0.33 canal 1.1.7 canal c#: CanalSharp 1.2.1
问题重现: 在mysql执行以下代码(逐句执行) create table tab0919 ( x1 binary(10), x2 datetime, x3 bool, y varchar(100), constraint pk_tab0919 primary key(x1,x2,x3) );
insert into tab0919 select 0x123F,now(),101,'ddd';
insert into tab0919 select 0x123F,now(),102,'ddd';
update tab0919 set x2=now();
alter table tab0919 modify column x1 varbinary(34) not null comment 'xx not null';
insert into tab0919 select 0x123F,now(),103,'ddd';
alter table tab0919 rename column x1 to x1B,rename column x2 to x2B;
alter table tab0919 rename column x1B to x111B;
==> 以上SQL都能正确解析,都已Ack确认过. 但是当执行以下SQL:
insert into tab0919 select 0x123F,now(),104,'1122';
==> 得到的Header.EventType=EventType.Alter, 包含2个SQL: alter table tab0919 rename column x1 to x1B,rename column x2 to x2B; alter table tab0919 rename column x1B to x111B 即是前面执行过的SQL, 重复执行该SQL导致报错了! 请问为何重复发送, 前面已接收过一次了, 且已Ack确认过, 还重复接收一次.
附上server日志文件报错内容: 2024-09-20 15:55:43.043 [destination = example , address = /127.0.0.1:3306 , EventParser] ERROR c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - dump address 127.0.0.1/127.0.0.1:3306 has an error, retrying. caused by com.alibaba.otter.canal.parse.exception.CanalParseException: com.alibaba.otter.canal.parse.exception.CanalParseException: java.lang.RuntimeException: unknow column : x1 Caused by: com.alibaba.otter.canal.parse.exception.CanalParseException: java.lang.RuntimeException: unknow column : x1 Caused by: java.lang.RuntimeException: unknow column : x1 at com.alibaba.otter.canal.parse.inbound.TableMeta.getFieldMetaByName(TableMeta.java:74) at com.alibaba.otter.canal.parse.inbound.mysql.tsdb.MemoryTableMeta.processTableElement(MemoryTableMeta.java:256) at com.alibaba.otter.canal.parse.inbound.mysql.tsdb.MemoryTableMeta.parse(MemoryTableMeta.java:174) at com.alibaba.otter.canal.parse.inbound.mysql.tsdb.MemoryTableMeta.find(MemoryTableMeta.java:122) at com.alibaba.otter.canal.parse.inbound.mysql.tsdb.DatabaseTableMeta.find(DatabaseTableMeta.java:137) at com.alibaba.otter.canal.parse.inbound.mysql.dbsync.TableMetaCache.getTableMeta(TableMetaCache.java:174) at com.alibaba.otter.canal.parse.inbound.mysql.dbsync.LogEventConvert.getTableMeta(LogEventConvert.java:990) at com.alibaba.otter.canal.parse.inbound.mysql.dbsync.LogEventConvert.parseRowsEventForTableMeta(LogEventConvert.java:494) at com.alibaba.otter.canal.parse.inbound.mysql.MysqlMultiStageCoprocessor$SimpleParserStage.processEvent(MysqlMultiStageCoprocessor.java:293) at com.alibaba.otter.canal.parse.inbound.mysql.MysqlMultiStageCoprocessor$SimpleParserStage.onEvent(MysqlMultiStageCoprocessor.java:276) at com.alibaba.otter.canal.parse.inbound.mysql.MysqlMultiStageCoprocessor$SimpleParserStage.onEvent(MysqlMultiStageCoprocessor.java:239) at com.lmax.disruptor.BatchEventProcessor.processEvents(BatchEventProcessor.java:168) at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:125) at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source)