search

hortonworks-spark / shc

The Apache Spark - Apache HBase Connector is a library to support Spark accessing HBase table as external data source or sink.
Apache License 2.0
552 stars 281 forks source link

Hang connection issue when write on hbase #320

Open attolini opened 5 years ago

attolini commented 5 years ago

Hello everyone, I'm experiencing hang connection issue when I try to write on Hbase table with a Spark Shell. I launch the Spark Shell as indicates in README: spark-shell --packages com.hortonworks:shc-core:1.1.1-2.1-s_2.11 --repositories https://repo.hortonworks.com/content/groups/public/ --files /etc/hadoop/conf/hbase-site.xml --jars /usr/hdp/current/phoenix-client/phoenix-server.jar

and I'm stucked with this: scala> sc.parallelize(newData).toDF.write.options(Map(HBaseTableCatalog.tableCatalog -> cat, HBaseTableCatalog.newTable -> "5")).format("org.apache.spark.sql.execution.datasources.hbase").save() [Stage 0:===================================================> (7 + 1) / 8]

Actually, if I interrupt the task and I run df.show, I see the new value in the table. So this is only an annoying hanging issue. If I use /usr/hdp/current/phoenix-client/phoenix-client.jar I solve the issue. Any tips to solve this issue? Best regards, Alessio