canal发送kafka支持的kafka的版本以及scala的版本

BigData-YC commented 4 years ago

2019-09-25 15:59:49.046 [pool-6-thread-1] ERROR com.alibaba.otter.canal.kafka.CanalKafkaProducer - java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms. java.lang.RuntimeException: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms. at com.alibaba.otter.canal.kafka.CanalKafkaProducer.produce(CanalKafkaProducer.java:215) ~[canal.server-1.1.4.jar:na] at com.alibaba.otter.canal.kafka.CanalKafkaProducer.send(CanalKafkaProducer.java:179) ~[canal.server-1.1.4.jar:na] at com.alibaba.otter.canal.kafka.CanalKafkaProducer.send(CanalKafkaProducer.java:117) ~[canal.server-1.1.4.jar:na] at com.alibaba.otter.canal.server.CanalMQStarter.worker(CanalMQStarter.java:183) [canal.server-1.1.4.jar:na] at com.alibaba.otter.canal.server.CanalMQStarter.access$500(CanalMQStarter.java:23) [canal.server-1.1.4.jar:na] at com.alibaba.otter.canal.server.CanalMQStarter$CanalMQRunnable.run(CanalMQStarter.java:225) [canal.server-1.1.4.jar:na] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_181] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_181] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_181] Caused by: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms. at org.apache.kafka.clients.producer.KafkaProducer$FutureFailure.(KafkaProducer.java:1150) ~[kafka-clients-1.1.1.jar:na] at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:846) ~[kafka-clients-1.1.1.jar:na] at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:784) ~[kafka-clients-1.1.1.jar:na] at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:671) ~[kafka-clients-1.1.1.jar:na] at com.alibaba.otter.canal.kafka.CanalKafkaProducer.produce(CanalKafkaProducer.java:199) ~[canal.server-1.1.4.jar:na] ... 8 common frames omitted

D-YuLin commented 4 years ago

版本问题么？

tj-eric commented 4 years ago

遇到了相同的问题，版本也是不一致，目前没有办法解决

shuhong123 commented 4 years ago

这个问题有解决吗，我前几天还好好，今天突然就报这个错

zxjin1989 commented 4 years ago

不是版本的问题代码中加上properties.put("sasl.mechanism", "PLAIN"); 就可以了

tj-eric commented 4 years ago

不是版本的问题代码中加上properties.put("sasl.mechanism", "PLAIN"); 就可以了

通过修改源码的方式解决吗？

zxjin1989 commented 4 years ago

不是版本的问题代码中加上properties.put("sasl.mechanism", "PLAIN"); 就可以了

通过修改源码的方式解决吗？

嗯 server那个模块中CanalKafkaProducer 加进去重新打包替换了就可以了

Jtiag commented 4 years ago

加上properties.put("sasl.mechanism", "PLAIN");这个后包替换了还是报错

kirin-13 commented 4 years ago

加上properties.put("sasl.mechanism", "PLAIN");这个后包替换了还是报错

启用canal server 配置里canal.mq.topic=example 这个默认的topic解决了这个问题。

zongsforce commented 4 years ago

canal.mq.topic和canal.mq.dynamicTopic两个参数是都要配置的，后者存放你正则匹配到的数据，前者存放一些没有匹配到的schema和table的日志感谢@chenxyz707大佬~，详见https://github.com/alibaba/canal/issues/1716

yangle9567 commented 4 years ago

我们这边是调整了kafka里面默认的消息大小好了

twodog-tt commented 4 years ago

mark

doyoungmoon commented 3 years ago

同样的问题，连接kafka需要sasl认证，一直没连成功

alibaba / canal

canal发送kafka支持的kafka的版本以及scala的版本 #2244