search

dstreev / cloudera_upgrade_utils

Various tools to help plan HDP and CDH upgrades to CDP
14 stars 11 forks source link

Unable to connect hive metastore #21

Closed shyamraj242 closed 2 years ago

shyamraj242 commented 3 years ago
1
shyamraj242 commented 3 years ago

HI @dstreev

When we are trying to get the hive tables report, we are getting the below error.

Error: Application JAVA_OPTS=-Xms1024m -Xmx2048m -XX:+UseG1GC PRG_ARGS= "sre" "-db" "hivesmallfiles" "-cfg" "/home/yarn/hivesmallfiles/hive-sre/default.yaml" "-o" "./sre-out" openjdk version "1.8.0_252" OpenJDK Runtime Environment (build 1.8.0_252-b09) OpenJDK 64-Bit Server VM (build 25.252-b09, mixed mode) Launching: sre Using Config: /home/yarn/hivesmallfiles/hive-sre/default.yaml

Table and Partition Scan - Small Files (v.2.3.6.0-SNAPSHOT)

java.sql.SQLException: No suitable driver found for jdbc:oracle:thin:@//XXXXXXXXXXXXX:2063/infadev at java.sql.DriverManager.getConnection(DriverManager.java:689) at java.sql.DriverManager.getConnection(DriverManager.java:208) at org.apache.commons.dbcp2.DriverManagerConnectionFactory.createConnection(DriverManagerConnectionFactory.java:123) at org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:355)

Steps We Performed:

  1. extracted the hive-sre-dist.tar.gz at "/home/yarn/hivesmallfiles"
  2. Modified the default.yml file added the below properties.

yarn@XXXXXXXX-/home/yarn/hivesmallfiles/hive-sre $ cat default.yaml metastore_direct: uri: "jdbc:oracle:thin:@//XXXXXXX:2063/infadev" type: ORACLE

Required for Oracle Connections to Select proper schema

initSql: "ALTER SESSION SET CURRENT_SCHEMA=infadev" connectionProperties: user: omgdev_hive password: XXXXXX connectionPool: min: 3 max: 5 hs2: uri: "jdbc:hive2://XXXXXXX:2181,XXXXXXXXXXX:2181,XXXXXXXX:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2" connectionPool: min: 3 max: 5 parallelism: 4 queries: db_tbl_count: parameters: dbs: override: "%"

  1. We added ojdbc8.jar also in the path as well.
  2. ./hive-sre sre -db hivesmallfiles -cfg /home/yarn/hivesmallfiles/hive-sre/default.yaml -o ./sre-out We ran the above command and we are getting the above mentioned error.
  3. I also attached the hivemetastore db connection details as well.

Please let me know if you need any other information on this.

shyamraj242 commented 3 years ago

Hi @dstreev

any inputs you needed from my side.

dstreev commented 3 years ago

@shyamraj242 was the jdbc jar file added to $HOME/.hive-sre/aux_libs ?

dstreev commented 2 years ago

Closing with no response to previous question.