search

nerdammer / spark-hbase-connector

Connect Spark to HBase for reading and writing data with ease
Apache License 2.0
297 stars 107 forks source link

how do i apply a customized partitioner to hBaseRDD? #69

Closed gypsysunny closed 6 years ago

gypsysunny commented 6 years ago

val hBaseRDD = sc.hbaseTable(String, Int, String) .select("column1", "column2") .inColumnFamily("mycf")

thank you.

nicolaferraro commented 6 years ago

Hi @gypsysunny, what you mean by custom partitioner? The connector allows you to use salting to randomize the prefix on the keys in order to spread them. Setting regions is something that should be at table creation time outside the scope of the connector.

gypsysunny commented 6 years ago

i mean a custom partitioner analagous to HashPartitioner. and i have got a way to do that. thank you.