search

hortonworks-spark / shc

The Apache Spark - Apache HBase Connector is a library to support Spark accessing HBase table as external data source or sink.
Apache License 2.0
552 stars 281 forks source link

Push in existing table failure: InvalidRegionNumberException then IllegalArgumentException. #323

Closed soapchr closed 3 years ago

soapchr commented 4 years ago

I want to push data into an already existing table, single column family, no records. I am using shc-core:1.1.1-2.1-s_2.11 on a windows machine. I have hbase 1.2.6 installed and use scala 2.11.8.

When I try to push data I got first the following error: org.apache.spark.sql.execution.datasources.hbase.InvalidRegionNumberException: Number of regions specified for new table must be greater than 3.

After following the advice of this link https://github.com/hortonworks-spark/shc/issues/249#issue-318285217, I added: HBaseTableCatalog.newTable -> "5" to my options. It still failed but with: java.lang.IllegalArgumentException: Can not create a Path from a null string.

Following this link: https://github.com/hortonworks-spark/shc/issues/151#issuecomment-313800739, I added to my catalog: , "tableCoder":"PrimitiveType". Still facing the same error.

I saw people are expecting some clarification about that issue (https://github.com/hortonworks-spark/shc/issues/249#issuecomment-463528032).

It is known issue and apparently it seems fixed (https://github.com/hortonworks-spark/shc/issues/155#issuecomment-315236736). I do not know what to do next. Is there a solution about this?