alibaba / canal

阿里巴巴 MySQL binlog 增量订阅&消费组件
Apache License 2.0
28.44k stars 7.6k forks source link

解决MySQL8.0和canal1.1.6出现的控制台和Java程序无报错出现的数据库改变但canal未抓取问题 #4709

Open Bigmanwimi opened 1 year ago

Bigmanwimi commented 1 year ago

打开logs\example\example.log文件发现有以下错误 2023-05-14 22:33:43.989 [destination = example , address = /127.0.0.1:3306 , EventParser] WARN c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - ---> begin to find start position, it will be long time for reset or first position 2023-05-14 22:33:43.989 [destination = example , address = /127.0.0.1:3306 , EventParser] WARN c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - prepare to find start position just show master status 2023-05-14 22:33:44.108 [destination = example , address = /127.0.0.1:3306 , EventParser] ERROR c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - dump address /127.0.0.1:3306 has an error, retrying. caused by java.lang.NullPointerException: null at com.alibaba.druid.sql.visitor.SQLASTOutputVisitor.visit(SQLASTOutputVisitor.java:5397) ~[druid-1.2.11.jar:1.2.11] at com.alibaba.druid.sql.ast.statement.SQLCheck.accept0(SQLCheck.java:47) ~[druid-1.2.11.jar:1.2.11] at com.alibaba.druid.sql.ast.SQLObjectImpl.accept(SQLObjectImpl.java:48) ~[druid-1.2.11.jar:1.2.11] at com.alibaba.druid.sql.visitor.SQLASTOutputVisitor.printTableElements(SQLASTOutputVisitor.java:3530) ~[druid-1.2.11.jar:1.2.11] at com.alibaba.druid.sql.visitor.SQLASTOutputVisitor.visit(SQLASTOutputVisitor.java:9568) ~[druid-1.2.11.jar:1.2.11] at com.alibaba.druid.sql.dialect.mysql.ast.statement.MySqlCreateTableStatement.accept0(MySqlCreateTableStatement.java:133) ~[druid-1.2.11.jar:1.2.11] at com.alibaba.druid.sql.dialect.mysql.ast.statement.MySqlCreateTableStatement.accept0(MySqlCreateTableStatement.java:93) ~[druid-1.2.11.jar:1.2.11] at com.alibaba.druid.sql.ast.SQLObjectImpl.accept(SQLObjectImpl.java:48) ~[druid-1.2.11.jar:1.2.11] at com.alibaba.druid.sql.ast.SQLObjectImpl.output(SQLObjectImpl.java:89) ~[druid-1.2.11.jar:1.2.11] at com.alibaba.druid.sql.ast.SQLObjectImpl.output(SQLObjectImpl.java:74) ~[druid-1.2.11.jar:1.2.11] at com.alibaba.otter.canal.parse.inbound.mysql.tsdb.MemoryTableMeta.snapshot(MemoryTableMeta.java:151) ~[canal.parse-1.1.6.jar:na] at com.alibaba.otter.canal.parse.inbound.mysql.tsdb.DatabaseTableMeta.applySnapshotToDB(DatabaseTableMeta.java:303) ~[canal.parse-1.1.6.jar:na] at com.alibaba.otter.canal.parse.inbound.mysql.tsdb.DatabaseTableMeta.rollback(DatabaseTableMeta.java:176) ~[canal.parse-1.1.6.jar:na] at com.alibaba.otter.canal.parse.inbound.mysql.AbstractMysqlEventParser.processTableMeta(AbstractMysqlEventParser.java:142) ~[canal.parse-1.1.6.jar:na] at com.alibaba.otter.canal.parse.inbound.AbstractEventParser$1.run(AbstractEventParser.java:197) ~[canal.parse-1.1.6.jar:na] at java.lang.Thread.run(Unknown Source) [na:1.8.0_341]

解决方案:到conf\example\instance.properties 里修改 canal.instance.tsdb.enable = false

Xiaofeng05 commented 1 year ago

打开logs\example\example.log文件发现有以下错误 2023-06-04 11:42:42.678 [destination = example , address = /127.0.0.1:3306 , EventParser] WARN c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - ---> begin to find start position, it will be long time for reset or first position 2023-06-04 11:42:42.679 [destination = example , address = /127.0.0.1:3306 , EventParser] WARN c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - prepare to find start position just show master status 2023-06-04 11:42:42.680 [destination = example , address = /127.0.0.1:3306 , EventParser] ERROR c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - dump address /127.0.0.1:3306 has an error, retrying. caused by com.alibaba.otter.canal.parse.exception.CanalParseException: command : 'show master status' has an error! pls check. you need (at least one of) the SUPER,REPLICATION CLIENT privilege(s) for this operation 2023-06-04 11:42:42.683 [destination = example , address = /127.0.0.1:3306 , EventParser] ERROR com.alibaba.otter.canal.common.alarm.LogAlarmHandler - destination:example[com.alibaba.otter.canal.parse.exception.CanalParseException: command : 'show master status' has an error! pls check. you need (at least one of) the SUPER,REPLICATION CLIENT privilege(s) for this operation ] 2023-06-04 11:42:43.107 [New I/O server worker #1-3] INFO c.a.otter.canal.instance.core.AbstractCanalInstance - subscribe filter change to ... 2023-06-04 11:42:43.117 [New I/O server worker #1-3] WARN c.a.o.canal.parse.inbound.mysql.dbsync.LogEventConvert - --> init table filter : ^...$ 这个问题时一样的嘛?

agapple commented 1 year ago

command : 'show master status' has an error! pls check. you need (at least one of) the SUPER,REPLICATION CLIENT privilege(s) for this operation

command : 'show master status' has an error! pls check. you need (at least one of) the SUPER,REPLICATION CLIENT privilege(s) for this operation,你这个是没权限,不是一回事

agapple commented 1 year ago

MySqlCreateTableStatement 你的源端是mysql啥版本?这个报错是show create table的结果,canal无法识别

可以找到对应的show create table语句反馈给我