search

alibaba / canal

阿里巴巴 MySQL binlog 增量订阅&消费组件
Apache License 2.0
28.46k stars 7.61k forks source link

canal读取PolarDB #3566

Closed yqzdy1207 closed 3 years ago

yqzdy1207 commented 3 years ago

ERROR c.a.o.canal.parse.inbound.mysql.dbsync.DirectLogFetcher

出现canal读取不到polarDB的位点的情况，重启该instance就可以从保存的位点继续读取。这种情况是因为什么引起的呀？不重启会一直延迟

yqzdy1207 commented 3 years ago

1.1.4版本

limingxinleo commented 3 years ago

我这边是 PolarDB 同步的很慢。。。本来与线上差两个小时，同步几个小时之后，就与线上差四个小时了。。。

agapple commented 3 years ago

could not find next log; the first event 'mysql-bin.000648' at 524288058, the last event read from '../log/mysql-bin.000649' at 524404562, the last byte read from '../log/mysql-bin.000649' at 524404562.

这个报错一般是binlog被重置了，要么是主备切换

chenrlbj commented 2 years ago

2022-10-21 17:20:00.076 [MultiStageCoprocessor-Parser-product-3] ERROR com.alibaba.otter.canal.common.utils.NamedThreadFactory - from MultiStageCoprocessor-Parser-product-3 com.alibaba.otter.canal.parse.exception.CanalParseException: com.alibaba.otter.canal.parse.exception.CanalParseException: com.alibaba.otter.canal.parse.exception.CanalParseException: parse row data failed. Caused by: com.alibaba.otter.canal.parse.exception.CanalParseException: com.alibaba.otter.canal.parse.exception.CanalParseException: parse row data failed. Caused by: com.alibaba.otter.canal.parse.exception.CanalParseException: parse row data failed. Caused by: java.lang.IllegalArgumentException: limit excceed: 80506 at com.taobao.tddl.dbsync.binlog.LogBuffer.fillBytes(LogBuffer.java:1557) ~[canal.parse.dbsync-1.1.6.jar:na] at com.taobao.tddl.dbsync.binlog.event.RowsLogBuffer.fetchValue(RowsLogBuffer.java:966) ~[canal.parse.dbsync-1.1.6.jar:na] at com.taobao.tddl.dbsync.binlog.event.RowsLogBuffer.nextValue(RowsLogBuffer.java:125) ~[canal.parse.dbsync-1.1.6.jar:na] at com.alibaba.otter.canal.parse.inbound.mysql.dbsync.LogEventConvert.parseOneRow(LogEventConvert.java:757) ~[canal.parse-1.1.6.jar:na] at com.alibaba.otter.canal.parse.inbound.mysql.dbsync.LogEventConvert.parseRowsEvent(LogEventConvert.java:568) ~[canal.parse-1.1.6.jar:na] at com.alibaba.otter.canal.parse.inbound.mysql.MysqlMultiStageCoprocessor$DmlParserStage.onEvent(MysqlMultiStageCoprocessor.java:341) ~[canal.parse-1.1.6.jar:na] at com.alibaba.otter.canal.parse.inbound.mysql.MysqlMultiStageCoprocessor$DmlParserStage.onEvent(MysqlMultiStageCoprocessor.java:327) ~[canal.parse-1.1.6.jar:na] at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143) ~[disruptor-3.4.2.jar:na] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_181] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_181] at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_181]