alibaba / canal

阿里巴巴 MySQL binlog 增量订阅&消费组件
Apache License 2.0
28.57k stars 7.62k forks source link

canal 1.1.5在canal instance里指定gtid后,报错:com.alibaba.otter.canal.parse.exception.CanalParseException: java.lang.ArrayIndexOutOfBoundsException: 1 #3493

Open peony123456 opened 3 years ago

peony123456 commented 3 years ago

canal源端指向mariadb数据库,gtid这样配置的: canal.instance.master.gtid=0-2-55 报错: 2021-04-28 18:02:07.587 [destination = aa-test , address = /192.168.144.244:3306 , EventParser] WARN c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - ---> begin to find start position, it will be long time for reset or first position 2021-04-28 18:02:07.798 [destination = aa-test , address = /192.168.144.244:3306 , EventParser] WARN c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - ---> find start position successfully, EntryPosition[included=false,journalName=master-bin.000001,position=6780,serverId=,gtid=0-1-33,timestamp=] cost : 28ms , the next step is binlog dump 2021-04-28 18:02:23.067 [destination = aa-test , address = /192.168.144.244:3306 , EventParser] ERROR c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - dump address /192.168.144.244:3306 has an error, retrying. caused by com.alibaba.otter.canal.parse.exception.CanalParseException: java.lang.ArrayIndexOutOfBoundsException: 1 Caused by: java.lang.ArrayIndexOutOfBoundsException: 1 at com.alibaba.otter.canal.parse.driver.mysql.packets.MariaGtid.(MariaGtid.java:27) ~[canal.parse.driver-1.1.5.jar:na] at com.alibaba.otter.canal.parse.driver.mysql.packets.MariaGtid.parse(MariaGtid.java:32) ~[canal.parse.driver-1.1.5.jar:na] at com.alibaba.otter.canal.parse.driver.mysql.packets.MariaGTIDSet.update(MariaGTIDSet.java:26) ~[canal.parse.driver-1.1.5.jar:na] at com.taobao.tddl.dbsync.binlog.LogDecoder.decode(LogDecoder.java:432) ~[canal.parse.dbsync-1.1.5.jar:na] at com.taobao.tddl.dbsync.binlog.LogDecoder.decode(LogDecoder.java:111) ~[canal.parse.dbsync-1.1.5.jar:na] at com.alibaba.otter.canal.parse.inbound.mysql.MysqlMultiStageCoprocessor$SimpleParserStage.onEvent(MysqlMultiStageCoprocessor.java:269) ~[canal.parse-1.1.5.jar:na] at com.alibaba.otter.canal.parse.inbound.mysql.MysqlMultiStageCoprocessor$SimpleParserStage.onEvent(MysqlMultiStageCoprocessor.java:251) ~[canal.parse-1.1.5.jar:na] at com.lmax.disruptor.BatchEventProcessor.processEvents(BatchEventProcessor.java:168) ~[disruptor-3.4.2.jar:na] at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:125) ~[disruptor-3.4.2.jar:na] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_231] at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_231] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_231] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_231] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_231] 有时不报错,但是发生mariadb主从切换的时候,也有这个错误。 指定成binlog和postion后不报这个错误,但是主从切换的时候,提示在新主库上找不到binlog: 2021-04-20 22:00:54.172 [destination = aa-test , address = /192.168.144.247:3306 , EventParser] WARN c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - the binlogfile:slave-bin.000001 doesn't exist, to continue to search the next binlogfile , caused by java.lang.ArrayIndexOutOfBoundsException: 1 at com.alibaba.otter.canal.parse.driver.mysql.packets.MariaGtid.(MariaGtid.java:27) ~[canal.parse.driver-1.1.5.jar:na] at com.alibaba.otter.canal.parse.driver.mysql.packets.MariaGtid.parse(MariaGtid.java:32) ~[canal.parse.driver-1.1.5.jar:na] at com.alibaba.otter.canal.parse.driver.mysql.packets.MariaGTIDSet.update(MariaGTIDSet.java:26) ~[canal.parse.driver-1.1.5.jar:na] at com.taobao.tddl.dbsync.binlog.LogDecoder.decode(LogDecoder.java:432) ~[canal.parse.dbsync-1.1.5.jar:na] at com.taobao.tddl.dbsync.binlog.LogDecoder.decode(LogDecoder.java:111) ~[canal.parse.dbsync-1.1.5.jar:na] at com.alibaba.otter.canal.parse.inbound.mysql.MysqlConnection.seek(MysqlConnection.java:154) ~[canal.parse-1.1.5.jar:na] at com.alibaba.otter.canal.parse.inbound.mysql.MysqlEventParser.findAsPerTimestampInSpecificLogFile(MysqlEventParser.java:761) [canal.parse-1.1.5.jar:na] at com.alibaba.otter.canal.parse.inbound.mysql.MysqlEventParser.findByStartTimeStamp(MysqlEventParser.java:591) [canal.parse-1.1.5.jar:na] at com.alibaba.otter.canal.parse.inbound.mysql.MysqlEventParser.findStartPositionInternal(MysqlEventParser.java:513) [canal.parse-1.1.5.jar:na] at com.alibaba.otter.canal.parse.inbound.mysql.MysqlEventParser.findStartPosition(MysqlEventParser.java:357) [canal.parse-1.1.5.jar:na] at com.alibaba.otter.canal.parse.inbound.AbstractEventParser$1.run(AbstractEventParser.java:191) [canal.parse-1.1.5.jar:na] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_231] 请问该如何处理呢?是我哪里配置得不正确吗?

peony123456 commented 3 years ago

解决办法: 在配置canal同步前执行flush logs;刷新binlog,确保Gtid_list非空。 详细请参考https://blog.csdn.net/yabingshi_tech/article/details/117228451 建议完善canal代码,增加Gtid_list为空时的处理

ajiang02 commented 1 year ago

我执行完 flush logs; 后,删除 zk 节点,重启 canal 还是不行。