Open azydbly opened 2 years ago
我使用的源码打包操作的,使用 github上下载的包 canal.deployer 一直报错。 #4223 可以监控到 数据库的操作但是没有往es插入数据
2022-06-14 11:29:15.109 [pool-2-thread-1] INFO c.a.o.canal.client.adapter.logger.LoggerAdapterExample - DML: {"data":null,"database":"","destination":"example","es":1655177354000,"groupId":"g1","isDdl":false,"old":null,"pkNames":[],"sql":"SET INSERT_ID = 3","table":"","ts":1655177354988,"type":"QUERY"} 2022-06-14 11:29:15.109 [pool-2-thread-1] INFO c.a.o.canal.client.adapter.logger.LoggerAdapterExample - DML: {"data":null,"database":"es","destination":"example","es":1655177354000,"groupId":"g1","isDdl":false,"old":null,"pkNames":[],"sql":"INSERT INTOes.hotel(title,city) VALUES ('1212', '1212121')","table":"hotel","ts":1655177354988,"type":"INSERT"} 2022-06-14 11:29:15.110 [pool-2-thread-1] DEBUG c.a.o.canal.client.adapter.es.core.service.ESSyncService - DML: {"data":null,"database":"es","destination":"example","es":1655177354000,"groupId":"g1","isDdl":false,"old":null,"pkNames":[],"sql":"INSERT INTOes.hotel(title,city) VALUES ('1212', '1212121')","table":"hotel","ts":1655177354988,"type":"INSERT"} Affected indexes: hotel
2022-06-14 11:29:15.109 [pool-2-thread-1] INFO c.a.o.canal.client.adapter.logger.LoggerAdapterExample - DML: {"data":null,"database":"","destination":"example","es":1655177354000,"groupId":"g1","isDdl":false,"old":null,"pkNames":[],"sql":"SET INSERT_ID = 3","table":"","ts":1655177354988,"type":"QUERY"} 2022-06-14 11:29:15.109 [pool-2-thread-1] INFO c.a.o.canal.client.adapter.logger.LoggerAdapterExample - DML: {"data":null,"database":"es","destination":"example","es":1655177354000,"groupId":"g1","isDdl":false,"old":null,"pkNames":[],"sql":"INSERT INTO
.
(
,
) VALUES ('1212', '1212121')","table":"hotel","ts":1655177354988,"type":"INSERT"} 2022-06-14 11:29:15.110 [pool-2-thread-1] DEBUG c.a.o.canal.client.adapter.es.core.service.ESSyncService - DML: {"data":null,"database":"es","destination":"example","es":1655177354000,"groupId":"g1","isDdl":false,"old":null,"pkNames":[],"sql":"INSERT INTO
) VALUES ('1212', '1212121')","table":"hotel","ts":1655177354988,"type":"INSERT"} Affected indexes: hotel
`server: port: 8081 spring: jackson: date-format: yyyy-MM-dd HH:mm:ss time-zone: GMT+8 default-property-inclusion: non_null
canal.conf: mode: tcp #tcp kafka rocketMQ rabbitMQ flatMessage: true zookeeperHosts: syncBatchSize: 1000 retries: -1 timeout: accessKey: secretKey: consumerProperties: canal.tcp.server.host: 127.0.0.1:11111 canal.tcp.zookeeper.hosts: canal.tcp.batch.size: 500 canal.tcp.username: canal.tcp.password: srcDataSources: defaultDS: url: jdbc:mysql://127.0.0.1:3306/es?useUnicode=true username: root password: 123456 canalAdapters:
dataSourceKey: defaultDS destination: example groupId: g1 esMapping: _index: hotel _id: _id _type: _doc upsert: true sql: " SELECT a.id as _id, a.title, a.city, a.price, a.score, a.create_date as createDate from hotel a " commitBatch: 3000
同样,但去ES查看文档数量,实际已经+1了,但就是无法搜索。刷新索引、持久化都没有用。用http发送了一下就能查到了,文档数量也没再次+1. 莫名其妙,令人头疼!
canal 1.1.6
mysql5.7
ES7.17.4
环境 mac 本地环境,所有服务都在本地,没有使用 docker
问题描述:
2022-06-14 11:29:15.109 [pool-2-thread-1] INFO c.a.o.canal.client.adapter.logger.LoggerAdapterExample - DML: {"data":null,"database":"","destination":"example","es":1655177354000,"groupId":"g1","isDdl":false,"old":null,"pkNames":[],"sql":"SET INSERT_ID = 3","table":"","ts":1655177354988,"type":"QUERY"} 2022-06-14 11:29:15.109 [pool-2-thread-1] INFO c.a.o.canal.client.adapter.logger.LoggerAdapterExample - DML: {"data":null,"database":"es","destination":"example","es":1655177354000,"groupId":"g1","isDdl":false,"old":null,"pkNames":[],"sql":"INSERT INTO
es.
hotel(
title,
city) VALUES ('1212', '1212121')","table":"hotel","ts":1655177354988,"type":"INSERT"} 2022-06-14 11:29:15.110 [pool-2-thread-1] DEBUG c.a.o.canal.client.adapter.es.core.service.ESSyncService - DML: {"data":null,"database":"es","destination":"example","es":1655177354000,"groupId":"g1","isDdl":false,"old":null,"pkNames":[],"sql":"INSERT INTO
es.
hotel(
title,
city) VALUES ('1212', '1212121')","table":"hotel","ts":1655177354988,"type":"INSERT"} Affected indexes: hotel
配置文件中的数据库账号为 root 用户
canal.adapter 配置:
`server: port: 8081 spring: jackson: date-format: yyyy-MM-dd HH:mm:ss time-zone: GMT+8 default-property-inclusion: non_null
canal.conf: mode: tcp #tcp kafka rocketMQ rabbitMQ flatMessage: true zookeeperHosts: syncBatchSize: 1000 retries: -1 timeout: accessKey: secretKey: consumerProperties: canal.tcp.server.host: 127.0.0.1:11111 canal.tcp.zookeeper.hosts: canal.tcp.batch.size: 500 canal.tcp.username: canal.tcp.password: srcDataSources: defaultDS: url: jdbc:mysql://127.0.0.1:3306/es?useUnicode=true username: root password: 123456 canalAdapters:
es7 中的配置
dataSourceKey: defaultDS destination: example groupId: g1 esMapping: _index: hotel _id: _id _type: _doc upsert: true sql: " SELECT a.id as _id, a.title, a.city, a.price, a.score, a.create_date as createDate from hotel a " commitBatch: 3000