search

hortonworks-spark / shc

The Apache Spark - Apache HBase Connector is a library to support Spark accessing HBase table as external data source or sink.
Apache License 2.0
552 stars 281 forks source link

Update Readme for Kerberized HBase Cluster #98

Closed ludochane closed 7 years ago

ludochane commented 7 years ago

Hi, I tried to run a Spark job to read from/write to HBase in a Horton Works cluster securized by Kerberos and passing the hbase-site.xml with --files never worked for me. As described in https://community.hortonworks.com/content/supportkb/48988/how-to-run-spark-job-to-interact-with-secured-hbas.html (point 2), the only solution which worked was to copy the hbase-site.xml directly in the Spark conf directory of our Edge node (/etc/spark/conf). Maybe I'm wrong and it is cluster dependant, but might be good to suggest this solution in the Readme. I could do a PR if needed.

Regards,

weiqingy commented 7 years ago

Did you try to add hbase-site.xml to the classpath instead of copying it in the Spark conf directory? If this works, we should recommend users to do this since copying configs leads to them going out of sync after updates to the main config and future source of issues.

Yes, we'd better mention this in the Readme. Could you make a PR for this? Thanks.

ludochane commented 7 years ago

Yep I tried to put it in the classpath but it did not work. Ok I'll make a PR soon. Thanks.

weiqingy commented 7 years ago

This has been resolved by PR#100, so close this.