I pull data from oracle11g, ~ source oracle; below my flume-ng config:
agentTest.channels = channelTest
agentTest.sources = sourceTest
agentTest.sinks = sinkTest
###########sql source#################
# For each Test of the sources, the type is defined
agentTest.sources.sourceTest.type = org.keedio.flume.source.SQLSource
#agentTest.sources.sourceTest.type = org.keedio.flume.source.SQLServerCustomDialect
agentTest.sources.sourceTest.hibernate.connection.url = jdbc:oracle:thin:@:1521:orcl
# Hibernate Database connection properties
agentTest.sources.sourceTest.hibernate.connection.user = ...
agentTest.sources.sourceTest.hibernate.connection.password = ...
agentTest.sources.sourceTest.hibernate.connection.autocommit = true
agentTest.sources.sourceTest.hibernate.dialect = org.hibernate.dialect.Oracle10gDialect
agentTest.sources.sourceTest.hibernate.connection.driver_class = oracle.jdbc.driver.OracleDriver
agentTest.sources.sourceTest.run.query.delay=10000
agentTest.sources.sourceTest.enclose.by.quotes = false
agentTest.sources.sourceTest.status.file.path = /usr/local/flume
agentTest.sources.sourceTest.status.file.name = agentTest.sqlSource.status
# Custom query
agentTest.sources.sourceTest.start.from = 0
agentTest.sources.sourceTest.custom.query = my sql select
agentTest.sources.sourceTest.batch.size = 1000
agentTest.sources.sourceTest.max.rows = 1000
agentTest.sources.sourceTest.hibernate.connection.provider_class = org.hibernate.connection.C3P0ConnectionProvider
agentTest.sources.sourceTest.hibernate.c3p0.min_size=1
agentTest.sources.sourceTest.hibernate.c3p0.max_size=10
...
Running, errors belowed:
2019-08-21 16:52:58,153 (SinkRunner-PollingRunner-DefaultSinkProcessor) [DEBUG - org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:141)] Polling sink runner starting
2019-08-21 16:52:58,183 (PollableSourceRunner-SQLSource-sourceTest) [ERROR - org.keedio.flume.source.HibernateHelper.executeQuery(HibernateHelper.java:128)] Exception thrown, resetting connection.
org.hibernate.MappingException: No Dialect mapping for JDBC type: -9
at org.hibernate.dialect.TypeNames.get(TypeNames.java:87)
at org.hibernate.dialect.TypeNames.get(TypeNames.java:118)
at org.hibernate.dialect.Dialect.getHibernateTypeName(Dialect.java:653)
at org.hibernate.loader.custom.JdbcResultMetadata.getHibernateType(JdbcResultMetadata.java:93)
at org.hibernate.loader.custom.ScalarResultColumnProcessor.performDiscovery(ScalarResultColumnProcessor.java:62)
at org.hibernate.loader.custom.CustomLoader.autoDiscoverTypes(CustomLoader.java:498)
at org.hibernate.loader.Loader.getResultSet(Loader.java:2074)
at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1863)
at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1839)
at org.hibernate.loader.Loader.doQuery(Loader.java:910)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:355)
at org.hibernate.loader.Loader.doList(Loader.java:2554)
at org.hibernate.loader.Loader.doList(Loader.java:2540)
at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2370)
at org.hibernate.loader.Loader.list(Loader.java:2365)
at org.hibernate.loader.custom.CustomLoader.list(CustomLoader.java:353)
at org.hibernate.internal.SessionImpl.listCustomQuery(SessionImpl.java:1909)
at org.hibernate.internal.AbstractSessionImpl.list(AbstractSessionImpl.java:311)
at org.hibernate.internal.SQLQueryImpl.list(SQLQueryImpl.java:141)
at org.keedio.flume.source.HibernateHelper.executeQuery(HibernateHelper.java:126)
at org.keedio.flume.source.SQLSource.process(SQLSource.java:102)
at org.apache.flume.source.PollableSourceRunner$PollingRunner.run(PollableSourceRunner.java:133)
at java.lang.Thread.run(Thread.java:748)
I pull data from oracle11g, ~ source oracle; below my flume-ng config:
Running, errors belowed:
2019-08-21 16:52:58,153 (SinkRunner-PollingRunner-DefaultSinkProcessor) [DEBUG - org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:141)] Polling sink runner starting 2019-08-21 16:52:58,183 (PollableSourceRunner-SQLSource-sourceTest) [ERROR - org.keedio.flume.source.HibernateHelper.executeQuery(HibernateHelper.java:128)] Exception thrown, resetting connection. org.hibernate.MappingException: No Dialect mapping for JDBC type: -9 at org.hibernate.dialect.TypeNames.get(TypeNames.java:87) at org.hibernate.dialect.TypeNames.get(TypeNames.java:118) at org.hibernate.dialect.Dialect.getHibernateTypeName(Dialect.java:653) at org.hibernate.loader.custom.JdbcResultMetadata.getHibernateType(JdbcResultMetadata.java:93) at org.hibernate.loader.custom.ScalarResultColumnProcessor.performDiscovery(ScalarResultColumnProcessor.java:62) at org.hibernate.loader.custom.CustomLoader.autoDiscoverTypes(CustomLoader.java:498) at org.hibernate.loader.Loader.getResultSet(Loader.java:2074) at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1863) at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1839) at org.hibernate.loader.Loader.doQuery(Loader.java:910) at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:355) at org.hibernate.loader.Loader.doList(Loader.java:2554) at org.hibernate.loader.Loader.doList(Loader.java:2540) at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2370) at org.hibernate.loader.Loader.list(Loader.java:2365) at org.hibernate.loader.custom.CustomLoader.list(CustomLoader.java:353) at org.hibernate.internal.SessionImpl.listCustomQuery(SessionImpl.java:1909) at org.hibernate.internal.AbstractSessionImpl.list(AbstractSessionImpl.java:311) at org.hibernate.internal.SQLQueryImpl.list(SQLQueryImpl.java:141) at org.keedio.flume.source.HibernateHelper.executeQuery(HibernateHelper.java:126) at org.keedio.flume.source.SQLSource.process(SQLSource.java:102) at org.apache.flume.source.PollableSourceRunner$PollingRunner.run(PollableSourceRunner.java:133) at java.lang.Thread.run(Thread.java:748)