hortonworks-spark / spark-llap

Apache License 2.0
102 stars 68 forks source link

Passed in wrong arguments for new getConnector helper method. #226

Closed EricWohlstadter closed 6 years ago

EricWohlstadter commented 6 years ago

Order should be (userProvidedDriverClass, url, userName, dbcp2Configs)

What changes were proposed in this pull request?

Fix getConnector helper method which derives arguments from HWConf class.

How was this patch tested?

Uncommitted integration test:

val hive = com.hortonworks.spark.sql.hive.llap.HiveWarehouseBuilder.session(spark).build()
hive.setDatabase("tpcds_bin_partitioned_orc_1000")
val df = hive.executeQuery("select * from web_sales")
hive.setDatabase("spark_llap")
val tempTable = "t_" + System.currentTimeMillis()
hive.createTable(tempTable).ifNotExists().column("ws_sold_time_sk", "bigint").column("ws_ship_date_sk", "bigint").create()
df.select("ws_sold_time_sk", "ws_ship_date_sk").filter("ws_sold_time_sk > 80000").write.format("com.hortonworks.spark.sql.hive.llap.HiveWarehouseConnector").option("table", tempTable).save()
val df2 = hive.executeQuery("select * from " + tempTable)
df2.show(20)
hive.dropTable(tempTable, true, false)
EricWohlstadter commented 6 years ago

+1 from @jdere offline