alibaba / canal

阿里巴巴 MySQL binlog 增量订阅&消费组件
Apache License 2.0
28.44k stars 7.6k forks source link

使用canal-adapter同步mysql数据到ES,增量更新时报空指针异常 #4296

Open Qxx-799 opened 2 years ago

Qxx-799 commented 2 years ago

Issue Description

1.1.6版本中,canal-adapter同步mysql的数据到ES,增量更新的时候报空指针;

经过验证,配置中的sql语句必须要有别名,即使单表查询也需要有,加上别名后不再报错。

canal-adapter 错误日志:

2022-07-14 14:49:53.324 [pool-2-thread-1] ERROR c.a.otter.canal.adapter.launcher.loader.AdapterProcessor - java.lang.NullPointerException
java.lang.RuntimeException: java.lang.NullPointerException
    at com.alibaba.otter.canal.client.adapter.es.core.service.ESSyncService.sync(ESSyncService.java:112) ~[na:na]
    at com.alibaba.otter.canal.client.adapter.es.core.service.ESSyncService.sync(ESSyncService.java:60) ~[na:na]
    at com.alibaba.otter.canal.client.adapter.es.core.ESAdapter.sync(ESAdapter.java:104) ~[na:na]
    at com.alibaba.otter.canal.client.adapter.es.core.ESAdapter.sync(ESAdapter.java:83) ~[na:na]
    at com.alibaba.otter.canal.adapter.launcher.loader.AdapterProcessor.batchSync(AdapterProcessor.java:140) ~[client-adapter.launcher-1.1.6.jar:na]
    at com.alibaba.otter.canal.adapter.launcher.loader.AdapterProcessor.lambda$null$1(AdapterProcessor.java:98) ~[client-adapter.launcher-1.1.6.jar:na]
    at java.base/java.util.concurrent.CopyOnWriteArrayList.forEach(CopyOnWriteArrayList.java:807) ~[na:na]
    at com.alibaba.otter.canal.adapter.launcher.loader.AdapterProcessor.lambda$null$2(AdapterProcessor.java:95) ~[client-adapter.launcher-1.1.6.jar:na]
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[na:na]
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na]
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na]
    at java.base/java.lang.Thread.run(Thread.java:829) ~[na:na]
Caused by: java.lang.NullPointerException: null
    at com.alibaba.otter.canal.client.adapter.es7x.support.ES7xTemplate.getESDataFromDmlData(ES7xTemplate.java:321) ~[na:na]
    at com.alibaba.otter.canal.client.adapter.es.core.service.ESSyncService.singleTableSimpleFiledUpdate(ESSyncService.java:814) ~[na:na]
    at com.alibaba.otter.canal.client.adapter.es.core.service.ESSyncService.update(ESSyncService.java:208) ~[na:na]
    at com.alibaba.otter.canal.client.adapter.es.core.service.ESSyncService.sync(ESSyncService.java:97) ~[na:na]
    ... 11 common frames omitted
JasonPengZiTing commented 2 years ago

解决了吗

Qxx-799 commented 2 years ago

解决了吗

解决了 配置的sql加上别名就不报错了 能正常同步

JasonPengZiTing commented 2 years ago

表是加了别名,有些字段没加别名,执行能打印同步日志,但是报这个空指针

yujiaxing88 commented 2 years ago

表是加了别名,有些字段没加别名,执行能打印同步日志,但是报这个空指针

同样的问题,兄弟你咋解决的

yujiaxing88 commented 1 year ago

这是来自QQ邮箱的假期自动回复邮件。   您好,我最近正在休假中,无法亲自回复您的邮件。我将在假期结束后,尽快给您回复。

afewnotes commented 1 year ago

https://github.com/alibaba/canal/blob/ee51f338e0bced4d28784ba1e5a72858c5dca3d5/client-adapter/es7x/src/main/java/com/alibaba/otter/canal/client/adapter/es7x/support/ES7xTemplate.java#L321

这个代码已经强制使用表别名了,没有别名就会报 NPE