search

StarRocks / starrocks-connector-for-kafka

Apache License 2.0
4 stars 8 forks source link

we can't use debezium jdbc sink is because hibernate doesn't work with StarRocks. debezium jdbc insert error #20

Open alberttwong opened 2 months ago

alberttwong commented 2 months ago

The reason why we can't use debezium jdbc sink is because hibernate doesn't work with StarRocks. If we fix hibernate, then we don't need our own sink (unless you want to use stream load). 

2024-04-23 15:29:34 org.hibernate.service.spi.ServiceException: Unable to create requested service [org.hibernate.engine.jdbc.env.spi.JdbcEnvironment] due to: Unable to determine Dialect without JDBC metadata (please set 'javax.persistence.jdbc.url', 'hibernate.connection.url', or 'hibernate.dialect')
2024-04-23 15:29:34     at org.hibernate.service.internal.AbstractServiceRegistryImpl.createService(AbstractServiceRegistryImpl.java:277)
2024-04-23 15:29:34     at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:239)
2024-04-23 15:29:34     at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:216)
2024-04-23 15:29:34     at org.hibernate.boot.model.relational.Database.<init>(Database.java:45)
2024-04-23 15:29:34     at org.hibernate.boot.internal.InFlightMetadataCollectorImpl.getDatabase(InFlightMetadataCollectorImpl.java:231)
2024-04-23 15:29:34     at org.hibernate.boot.internal.InFlightMetadataCollectorImpl.<init>(InFlightMetadataCollectorImpl.java:199)
2024-04-23 15:29:34     at org.hibernate.boot.model.process.spi.MetadataBuildingProcess.complete(MetadataBuildingProcess.java:169)
2024-04-23 15:29:34     at org.hibernate.boot.model.process.spi.MetadataBuildingProcess.build(MetadataBuildingProcess.java:128)
2024-04-23 15:29:34     at org.hibernate.boot.internal.MetadataBuilderImpl.build(MetadataBuilderImpl.java:451)
2024-04-23 15:29:34     at org.hibernate.boot.internal.MetadataBuilderImpl.build(MetadataBuilderImpl.java:102)
2024-04-23 15:29:34     at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:910)
2024-04-23 15:29:34     at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:960)
2024-04-23 15:29:34     at io.debezium.connector.jdbc.JdbcSinkConnectorTask.start(JdbcSinkConnectorTask.java:75)
2024-04-23 15:29:34     at org.apache.kafka.connect.runtime.WorkerSinkTask.initializeAndStart(WorkerSinkTask.java:329)
2024-04-23 15:29:34     at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:202)
2024-04-23 15:29:34     at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:259)
2024-04-23 15:29:34     at org.apache.kafka.connect.runtime.isolation.Plugins.lambda$withClassLoader$1(Plugins.java:236)
2024-04-23 15:29:34     at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
2024-04-23 15:29:34     at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
2024-04-23 15:29:34     at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
2024-04-23 15:29:34     at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
2024-04-23 15:29:34     at java.base/java.lang.Thread.run(Thread.java:829)
2024-04-23 15:29:34 Caused by: org.hibernate.HibernateException: Unable to determine Dialect without JDBC metadata (please set 'javax.persistence.jdbc.url', 'hibernate.connection.url', or 'hibernate.dialect')
2024-04-23 15:29:34     at org.hibernate.engine.jdbc.dialect.internal.DialectFactoryImpl.determineDialect(DialectFactoryImpl.java:190)
2024-04-23 15:29:34     at org.hibernate.engine.jdbc.dialect.internal.DialectFactoryImpl.buildDialect(DialectFactoryImpl.java:86)
2024-04-23 15:29:34     at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.getJdbcEnvironmentWithDefaults(JdbcEnvironmentInitiator.java:229)
2024-04-23 15:29:34     at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.getJdbcEnvironmentUsingJdbcMetadata(JdbcEnvironmentInitiator.java:381)
2024-04-23 15:29:34     at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:193)
2024-04-23 15:29:34     at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:69)
2024-04-23 15:29:34     at org.hibernate.boot.registry.internal.StandardServiceRegistryImpl.initiateService(StandardServiceRegistryImpl.java:119)
2024-04-23 15:29:34     at org.hibernate.service.internal.AbstractServiceRegistryImpl.createService(AbstractServiceRegistryImpl.java:264)
2024-04-23 15:29:34     ... 21 more
alberttwong commented 2 months ago 
2024-04-23 15:13:16 2024-04-23 22:13:16,087 WARN   ||  HHH000342: Could not obtain connection to query metadata   [org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator]
2024-04-23 15:13:16 java.lang.NullPointerException
2024-04-23 15:13:16     at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcIsolationDelegate.delegateWork(JdbcIsolationDelegate.java:116)
2024-04-23 15:13:16     at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.getJdbcEnvironmentUsingJdbcMetadata(JdbcEnvironmentInitiator.java:279)
2024-04-23 15:13:16     at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:193)
2024-04-23 15:13:16     at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:69)
2024-04-23 15:13:16     at org.hibernate.boot.registry.internal.StandardServiceRegistryImpl.initiateService(StandardServiceRegistryImpl.java:119)
2024-04-23 15:13:16     at org.hibernate.service.internal.AbstractServiceRegistryImpl.createService(AbstractServiceRegistryImpl.java:264)
2024-04-23 15:13:16     at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:239)
2024-04-23 15:13:16     at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:216)
2024-04-23 15:13:16     at org.hibernate.boot.model.relational.Database.<init>(Database.java:45)
2024-04-23 15:13:16     at org.hibernate.boot.internal.InFlightMetadataCollectorImpl.getDatabase(InFlightMetadataCollectorImpl.java:231)
2024-04-23 15:13:16     at org.hibernate.boot.internal.InFlightMetadataCollectorImpl.<init>(InFlightMetadataCollectorImpl.java:199)
2024-04-23 15:13:16     at org.hibernate.boot.model.process.spi.MetadataBuildingProcess.complete(MetadataBuildingProcess.java:169)
2024-04-23 15:13:16     at org.hibernate.boot.model.process.spi.MetadataBuildingProcess.build(MetadataBuildingProcess.java:128)
2024-04-23 15:13:16     at org.hibernate.boot.internal.MetadataBuilderImpl.build(MetadataBuilderImpl.java:451)
2024-04-23 15:13:16     at org.hibernate.boot.internal.MetadataBuilderImpl.build(MetadataBuilderImpl.java:102)
2024-04-23 15:13:16     at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:910)
2024-04-23 15:13:16     at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:960)
2024-04-23 15:13:16     at io.debezium.connector.jdbc.JdbcSinkConnectorTask.start(JdbcSinkConnectorTask.java:75)
2024-04-23 15:13:16     at org.apache.kafka.connect.runtime.WorkerSinkTask.initializeAndStart(WorkerSinkTask.java:329)
2024-04-23 15:13:16     at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:202)
2024-04-23 15:13:16     at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:259)
2024-04-23 15:13:16     at org.apache.kafka.connect.runtime.isolation.Plugins.lambda$withClassLoader$1(Plugins.java:236)
2024-04-23 15:13:16     at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
2024-04-23 15:13:16     at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
2024-04-23 15:13:16     at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
2024-04-23 15:13:16     at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
2024-04-23 15:13:16     at java.base/java.lang.Thread.run(Thread.java:829)
alberttwong commented 2 months ago

https://debezium.zulipchat.com/#narrow/stream/348104-community-mysql/topic/Unable.20to.20determine.20Dialect.20without.20JDBC.20metadata/near/435077946

alberttwong commented 2 months ago

This is very helpful https://rmoff.net/2020/01/16/changing-the-logging-level-for-kafka-connect-dynamically/