search

hortonworks-spark / shc

The Apache Spark - Apache HBase Connector is a library to support Spark accessing HBase table as external data source or sink.
Apache License 2.0
552 stars 281 forks source link

Set hbase host & port with code #321

Open mohitgargk opened 4 years ago

mohitgargk commented 4 years ago

Is it possible to provide spark.hbase.host and spark.hbase.port in code? I tried following

`val sparkConf = new SparkConf().setAppName("Spark-HBase").setMaster("local[4]")

sparkConf.set("spark.hbase.host", "hbase_dev.myorg.net");

val spark: SparkSession = SparkSession.builder()
  .config(sparkConf)
  .getOrCreate()`

but getting error as shc tries to connect to localhost:

19/09/25 21:16:34 INFO ClientCnxn: Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error) 19/09/25 21:16:34 WARN ReadOnlyZKClient: 0x1a785fd5 to localhost:2181 failed for get of /hbase/hbaseid, code = CONNECTIONLOSS, retries = 3 19/09/25 21:16:34 WARN ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1125)

injulkarnilesh commented 3 years ago

I know it is a long shot, but did you ever solve it, and how?