search

datasalt / splout-db

A web-latency SQL spout for Hadoop.
50 stars 14 forks source link

Failed to load foster storage handler #41

Closed suolemen closed 10 years ago

suolemen commented 10 years ago

commend:

hadoop jar splout-hadoop-0.2.5-hadoop-mr2.jar simple-generate -it HIVE -hdb default -htn inventory__test2 -o out-hive-simple -pby areacode -p 120 -t inventory_test2_of_me -tb inventory_test2_sqlout

create hive table commend create table inventory_test2 (phoneno string,areacode string,dialresults string,talking_times string,dialtimes string,callturn int) row format delimited fields terminated by ',' collection items terminated by '|' lines terminated by '\n' STORED AS PARQUET ;

notice : STORED AS PARQUET ; !! this formate can not supported how can slouted it ?

suolemen commented 10 years ago

Error: java.io.IOException: Failed to load foster storage handler at org.apache.hcatalog.common.HCatUtil.getStorageHandler(HCatUtil.java:421) at org.apache.hcatalog.common.HCatUtil.getStorageHandler(HCatUtil.java:387) at org.apache.hcatalog.mapreduce.HCatBaseInputFormat.createRecordReader(HCatBaseInputFormat.java:191) at com.datasalt.pangool.tuplemr.mapred.lib.input.HCatTupleInputFormat.createRecordReader(HCatTupleInputFormat.java:119) at com.datasalt.pangool.tuplemr.mapred.lib.input.DelegatingRecordReader.(DelegatingRecordReader.java:55) at com.datasalt.pangool.tuplemr.mapred.lib.input.DelegatingInputFormat.createRecordReader(DelegatingInputFormat.java:73) at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.(MapTask.java:492) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:735) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163) Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:190) at org.apache.hcatalog.mapreduce.FosterStorageHandler.(FosterStorageHandler.java:63) at org.apache.hcatalog.common.HCatUtil.getStorageHandler(HCatUtil.java:416) ... 13 more

ivanprado commented 10 years ago

I'm using Splout with PARQUET tables without any issue in CDH5, so PARQUET is not the problem. But probably the problem is that Parquet is not properly included in the classpath of your job.

Locate the needed jars from your distro, and include them in the command line statement exactly in the following position:

hadoop jar splout-hadoop-0.2.5-hadoop-mr2.jar simple-generate -libjars lib1.jar,lib2.jar -it HIVE -hdb default -htn inventory__test2 -o out-hive-simple -pby areacode -p 120 -t inventory_test2_of_me -tb inventory_test2_sqlout

See this page for more information about the libjar option: http://grepalex.com/2013/02/25/hadoop-libjars/

suolemen commented 10 years ago

thanks!!!