search

streamsets / tutorials

StreamSets Tutorials
Apache License 2.0
348 stars 192 forks source link

How can I use JDBC Query Consumer to query data from hive? #98

Closed dijiekstra closed 5 years ago

dijiekstra commented 6 years ago

Hi,i want to query data from hive by JDBC Query Consumer .But it didn't work . Error is "JJDBC_00 - Cannot connect to specified database: com.streamsets.pipeline.api.StageException: JDBC_06 - Failed to initialize connection pool: com.zaxxer.hikari.pool.PoolInitializationException: Exception during pool initialization: Method not supported" How can i solve it?

qinzl1 commented 5 years ago

JDBC_00 - Cannot connect to specified database: com.streamsets.pipeline.api.StageException: JDBC_06 - Failed to initialize connection pool: com.zaxxer.hikari.pool.PoolInitializationException: Exception during pool initialization: Connection.isValid() is not supported, configure connection test query.

metadaddy commented 5 years ago

Hi @qinzl1 - as the error says, configure a connection test query - see Legacy Drivers tab - https://streamsets.com/documentation/datacollector/latest/help/datacollector/UserGuide/Origins/JDBCConsumer.html#task_ryz_tkr_bs

metadaddy commented 5 years ago

@lonelyGhostisdog The open source Hive driver does not properly implement the JDBC spec. If you can, download the vendor Hive driver for your distribution - Cloudera/Hortonworks

qinzl1 commented 5 years ago

@metadaddy Specific which version is valid，i copy the driver from streamsets-datacollector-cdh_5_15-lib dir

metadaddy commented 5 years ago

Which Hadoop distribution are you using?

Aditya-dixit commented 5 years ago

Worked for me once I switched to cloudera hive jdbc driver jar. As @metadaddy rightly mentioned, open source hive driver doesn't implement all the methods needed by streamsets for making JDBC connection.

metadaddy commented 5 years ago

@lonelyGhostisdog / @qinzl1 Did you get this working? Can I close the issue?

qinzl1 commented 5 years ago

@metadaddy it seem hive jdbc not only need configuration jdbc url ，but need hive-site.xml and hadoop config file. and JDBC origins stage didn't have Configuration items

metadaddy commented 5 years ago

@qinzl1 Use the Cloudera driver from https://www.cloudera.com/documentation/other/connectors/hive-jdbc/latest.html

metadaddy commented 5 years ago

Closing this since it's not actually an issue in the tutorials. Feel free to engage further via the StreamSets community https://streamsets.com/community/