表结构变更导致canal解析异常，消息重复投递

CarlaX commented 2 months ago

[ ] I have searched the issues of this repository and believe that this is not a duplicate.
[ ] I have checked the FAQ of this repository and believe that this is not a duplicate.

environment

canal version 1.1.7
mysql version 8.0.38

Issue Description

canal监听mysql，并原生投递消息到kafka 当把一张表的一个字段从小写改为大写，如：job->JOB，表其他不变，kafka能收到字段变更的消息之后修改该表的一行记录，更新任意字段值，kafka无法收到变更数据的消息，canal日志提示如下异常： com.alibaba.otter.canal.parse.exception.CanalParseException: com.alibaba.otter.canal.parse.exception.CanalParseException: parse row data failed. Caused by: com.alibaba.otter.canal.parse.exception.CanalParseException: parse row data failed. Caused by: com.alibaba.otter.canal.parse.exception.CanalParseException: column size is not match for table:test_db.t_user,7 vs 1 并且kafka不断收到记录更新前的数条消息的重复投递

Steps to reproduce

Expected behaviour

Actual behaviour

If there is an exception, please attach the exception trace:

Just put your stack trace here!

ap0405140 commented 4 weeks ago

相同问题, 请问怎么解决?

ChainFans commented 1 week ago

canal version 1.1.7 mysql version 8.0.36 2024-10-08 17:39:06.782 [MultiStageCoprocessor-Parser-instance01-0] ERROR com.alibaba.otter.canal.common.utils.NamedThreadFactory - from MultiStageCoprocessor-Parser-instance01-0 com.alibaba.otter.canal.parse.exception.CanalParseException: com.alibaba.otter.canal.parse.exception.CanalParseException: com.alibaba.otter.canal.parse.exception.CanalParseException: parse row data failed. Caused by: com.alibaba.otter.canal.parse.exception.CanalParseException: com.alibaba.otter.canal.parse.exception.CanalParseException: parse row data failed. Caused by: com.alibaba.otter.canal.parse.exception.CanalParseException: parse row data failed. Caused by: com.alibaba.otter.canal.parse.exception.CanalParseException: column size is not match for table:xxx,13 vs 12

alibaba / canal