davidmontgom
commented
7 years ago PS I also tried the below:
hive> CREATE EXTERNAL TABLE dude (key string, data struct<age:int,gender:string> ) ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe' STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ('hbase.columns.mapping' = ':key,TEST:') TBLPROPERTIES ('hbase.table.name' = 'MY_TABLE'); OK Time taken: 0.133 seconds hive> select * from dude; OK Failed with exception java.io.IOException:java.lang.ClassCastException: org.apache.hadoop.hive.hbase.ResultWritable cannot be cast to org.apache.hadoop.io.Text Time taken: 0.244 seconds
Why does your code not work with hbase? Is it the col mapping? I verified your example code works but clearly not with hbase.
HBASE create 'MY_TABLE', {NAME=>'TEST', VERSIONS => '3', COMPRESSION => 'NONE', TTL => '2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'false'}' put 'MY_TABLE' , '1' ,'TEST:data' ,'{"age":42, "gender":"male"}'
add jar hdfs:///lib/json-serde-1.3.8-jar-with-dependencies.jar; add jar hdfs:///lib/json-udf-1.3.8-jar-with-dependencies.jar; hive> CREATE EXTERNAL TABLE dude ( key string, data map<string,string>) ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe' STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ('hbase.columns.mapping' = ':key,TEST:') TBLPROPERTIES ('hbase.table.name' = 'MY_TABLE'); OK Time taken: 4.068 seconds hive> select * from dude; OK Failed with exception java.io.IOException:java.lang.ClassCastException: org.apache.hadoop.hive.hbase.ResultWritable cannot be cast to org.apache.hadoop.io.Text Time taken: 1.794 seconds hive>
I mean the data does exist:
hive> CREATE EXTERNAL TABLE uber_test (id string,colname map<string,string>) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,TEST:") TBLPROPERTIES("hbase.table.name" = "MY_TABLE"); OK Time taken: 0.116 seconds hive> select * from uber_test; OK 1 {"data":"{\"age\":42, \"gender\":\"male\"}"} Time taken: 0.206 seconds, Fetched: 1 row(s)