xnxz123
commented
1 month ago tdengine 3.2.2 kafka_2.13-3.7.0 git clone --branch 3.0 https://github.com/taosdata/kafka-connect-tdengine.git
Open xnxz123 opened 1 month ago
xnxz123
commented
1 month ago tdengine 3.2.2 kafka_2.13-3.7.0 git clone --branch 3.0 https://github.com/taosdata/kafka-connect-tdengine.git
yu285
commented
1 month ago kafka / TDengine 的日志中有报错信息吗
xnxz123
commented
1 month ago at org.apache.kafka.connect.runtime.WorkerConnector.doTransitionTo(WorkerConnector.java:357)
at org.apache.kafka.connect.runtime.WorkerConnector.doRun(WorkerConnector.java:144)
at org.apache.kafka.connect.runtime.WorkerConnector.run(WorkerConnector.java:122)
at org.apache.kafka.connect.runtime.isolation.Plugins.lambda$withClassLoader$1(Plugins.java:237)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.UnsatisfiedLinkError: no taos in java.library.path: /usr/java/packages/lib:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr>
at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2434)
at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:818)
at java.base/java.lang.System.loadLibrary(System.java:1993)
at com.taosdata.jdbc.TSDBJNIConnector.
xnxz123
commented
1 month ago tdengine 和kafka在不同的机器上
yu285
commented
1 month ago kafka 那边没装TDengine客户端吧。
2779382063
commented
1 month ago kafka装了客户端还是报了以下错误，我看了kafka-connect-tdengine-3.0的lib库下taos-jdbc驱动是3.3.3，所以在上面也装了3.3.3的TDEngine客户端（装3.0.7.1的客户端也是一样报以下错误），但我的TDEngine服务端是3.0.7.1。
[2024-10-22 16:19:11,277] ERROR [tdengine-source-3|worker] [Worker clientId=connect-1, groupId=connect-cluster] Failed to start connector 'tdengine-source-3' (org.apache.kafka.connect.runtime.distributed.DistributedHerder:1829)
org.apache.kafka.connect.errors.ConnectException: Failed to start connector: tdengine-source-3
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.lambda$startConnector$34(DistributedHerder.java:1800)
at org.apache.kafka.connect.runtime.WorkerConnector.doTransitionTo(WorkerConnector.java:346)
at org.apache.kafka.connect.runtime.WorkerConnector.doRun(WorkerConnector.java:143)
at org.apache.kafka.connect.runtime.WorkerConnector.run(WorkerConnector.java:121)
at org.apache.kafka.connect.runtime.isolation.Plugins.lambda$withClassLoader$1(Plugins.java:177)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: org.apache.kafka.connect.errors.ConnectException: Failed to transition connector tdengine-source-3 to state STARTED
... 9 more
Caused by: java.lang.NoClassDefFoundError: Could not initialize class com.taosdata.jdbc.TSDBJNIConnector
at com.taosdata.jdbc.TSDBDriver.connect(TSDBDriver.java:175)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:208)
at com.taosdata.kafka.connect.db.TSDBConnectionProvider.getConnection(TSDBConnectionProvider.java:35)
at com.taosdata.kafka.connect.source.MonitorThread.init(MonitorThread.java:51)
at com.taosdata.kafka.connect.source.MonitorThread.
按照官方文档的配置 https://docs.taosdata.com/third-party/collection/kafka/ root@pcserver006:~/kafka_2.13-3.7.0/bin# curl http://localhost:8083/connectors ["TDengineSourceConnector"]root@pcserver006:~/kafka_2.13-3.7.0/bin# source-demo.json内如如下： { "name":"TDengineSourceConnector", "config":{ "connector.class": "com.taosdata.kafka.connect.source.TDengineSourceConnector", "tasks.max": 1, "subscription.group.id": "source-demo", "connection.url": "jdbc:TAOS://192.168.201.152:6041", "connection.user": "root", "connection.password": "***", "connection.database": "test", "connection.attempts": 3, "connection.backoff.ms": 5000, "topic.prefix": "tdengine", "topic.delimiter": "-", "poll.interval.ms": 1000, "fetch.max.rows": 100, "topic.per.stable": true, "topic.ignore.db": false, "out.format": "line", "data.precision": "ms", "key.converter": "org.apache.kafka.connect.storage.StringConverter", "value.converter": "org.apache.kafka.connect.storage.StringConverter" } }
tdengine中也写入按文档建了test库与超级表meters，初始化了数据， 执行 curl -X POST -d @source-demo.json http://localhost:8083/connectors -H "Content-Type: application/json" {"name":"TDengineSourceConnector","config":{"connector.class":"com.taosdata.kafka.connect.source.TDengineSourceConnector","tasks.max":"1","subscription.group.id":"source-demo","connection.url":"jdbc:TAOS://192.168.201.152:6041","connection.user":"root","connection.password":"taosdata","connection.database":"test","connection.attempts":"3","connection.backoff.ms":"5000","topic.prefix":"tdengine","topic.delimiter":"-","poll.interval.ms":"1000","fetch.max.rows":"100","topic.per.stable":"true","topic.ignore.db":"false","out.format":"line","data.precision":"ms","key.converter":"org.apache.kafka.connect.storage.StringConverter","value.converter":"org.apache.kafka.connect.storage.StringConverter","name":"TDengineSourceConnector"},"tasks":[],"type":"source"}root@pcserver006:~/kafka_2.13-3.7.0#
kafka中没有数据 root@pcserver006:~/kafka_2.13-3.7.0/bin# ./kafka-console-consumer.sh --bootstrap-server localhost:9092 --from-beginning --topic tdengine-test-meters ^CProcessed a total of 0 messages