confluentinc / confluent-kubernetes-examples

Example scenario workflows for Confluent for Kubernetes
Apache License 2.0
43 stars 175 forks source link

[CFK-3021] Added `confluentinc-kafka-connect-jdbc-10.7.4` with mysql `8.0.30` #268

Open abhijeet2096-confluent opened 1 year ago

abhijeet2096-confluent commented 1 year ago

What

Current confluentinc-kafka-connect-jdbc 10.2.5has oldermysql-connector.jar(5.1.4.0) which causes following error against MySql server (8.0.32`)

Caused by: org.apache.kafka.connect.errors.ConnectException: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Could not create connection to database server.
    at io.confluent.connect.jdbc.util.CachedConnectionProvider.getConnection(CachedConnectionProvider.java:59)
    at io.confluent.connect.jdbc.JdbcSourceConnector.start(JdbcSourceConnector.java:92)
    at org.apache.kafka.connect.runtime.WorkerConnector.doStart(WorkerConnector.java:193)
    at org.apache.kafka.connect.runtime.WorkerConnector.start(WorkerConnector.java:218)
    at org.apache.kafka.connect.runtime.WorkerConnector.doTransitionTo(WorkerConnector.java:377)
    at org.apache.kafka.connect.runtime.WorkerConnector.doTransitionTo(WorkerConnector.java:358)
    ... 8 more
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Could not create connection to database server.
    at jdk.internal.reflect.GeneratedConstructorAccessor53.newInstance(Unknown Source)
    at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
    at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
    at com.mysql.jdbc.Util.getInstance(Util.java:408)
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:918)
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:897)
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:886)
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:860)
    at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2330)
    at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2083)
    at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:806)
    at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47)
    at jdk.internal.reflect.GeneratedConstructorAccessor50.newInstance(Unknown Source)
    at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
    at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
    at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:410)
    at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:328)
    at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:677)
    at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:189)
    at io.confluent.connect.jdbc.dialect.GenericDatabaseDialect.getConnection(GenericDatabaseDialect.java:239)
    at io.confluent.connect.jdbc.util.CachedConnectionProvider.newConnection(CachedConnectionProvider.java:80)
    at io.confluent.connect.jdbc.util.CachedConnectionProvider.getConnection(CachedConnectionProvider.java:52)
    ... 13 more
Caused by: java.lang.NullPointerException
    at com.mysql.jdbc.ConnectionImpl.getServerCharset(ConnectionImpl.java:2997)
    at com.mysql.jdbc.MysqlIO.sendConnectionAttributes(MysqlIO.java:1934)
    at com.mysql.jdbc.MysqlIO.proceedHandshakeWithPluggableAuthentication(MysqlIO.java:1863)
    at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1226)
    at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2253)
    at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2284)
    ... 27 more

This P.R adds another updated confluentinc-kafka-connect-jdbc-10.7.4.zip.

Usage

        url:
          - name: kafka-connect-jdbc-10-7-4
            archivePath: https://raw.githubusercontent.com/confluentinc/confluent-kubernetes-examples/abhijeet2096.NewJDBCJarWithMySqlConnector/hybrid/ccloud-JDBC-mysql/confluentinc-kafka-connect-jdbc-10.7.4.zip
            checksum: d526490393ee586a6f0ab7eddf4aa5bcaef947fbe93bcec1a334846b3ed45b6ca2bbc73b0caa76acec184ede8da69c51ed1402575832fd9014176bc1073ac3bb

The above path would get updated once merged.

Why do we require this zip, cannot use kafka-connect-jdbc directly from confluent-hub?

This is because kafka-connect-jdbc from confluent-hub doesn't contain mysql connector by default, which is required by jdbc to connect to MySql based SQL servers.

ToDo

[ ] Replace the older jdbc zip (10.2.5) with newer jdbc zip (10.7.4)

Reference

cla-assistant[bot] commented 1 year ago

CLA assistant check
All committers have signed the CLA.

cla-assistant[bot] commented 1 year ago

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.