search

yc-huang / Hive-mongo

hive storage handler for connecting with MongoDB
Apache License 2.0
32 stars 33 forks source link

Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.hive.serde2.ColumnProjectionUtils.getReadColumnIDs(Lorg/apache/hadoop/conf/Configuration;)Ljava/util/ArrayList; #17

Closed DivyaNagarajan closed 9 years ago

DivyaNagarajan commented 9 years ago

Hi, Iam Currently Working on Hive0.14 - MonogDB 2.6.3, as per the steps I builded successfully and copied hive-mongo-0.0.3.jar,guava-r06.jar,mongo-2.6.3.jar into hive_home/lib directory and also in hadoop_home/share/hadoop/mapred,hdfs,yarn (Hadoop 2.4). I created sample external hive-mongostorage table (TBL NAME : mongodb_archives) and then i insert overwrite into hive-mongostorage table ,its sending data into mongodbs collection and also i could query inserted data in collection the problem is when ever i query hive-mongostorage table i get this execption. hive> select * from mongodb_archives; OK Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.hive.serde2.ColumnProjectionUtils.getReadColumnIDs(Lorg/apache/hadoop/conf/Configuration;)Ljava/util/ArrayList; at org.yong3.hive.mongo.MongoInputFormat.getRecordReader(MongoInputFormat.java:39) at org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:498) at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:588) at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:561) at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:138) at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1621) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:267) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:199) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:410) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:783) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:616) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) [hadoop@hadoop ~]$ This exception made hive to drop out of cli Please help me out in this issue

Thanks, Divya N

yc-huang commented 9 years ago

I check the source code of Hive 0.14, there do have a getReadColumnIDs method in ColumnProjectionUtils...

Will download a 0.14 release and try it myself...

On Friday, November 21, 2014, divyanagarajan notifications@github.com wrote:

Hi, Iam Currently Working on Hive0.14 - MonogDB 2.6.3, as per the steps I builded successfully and copied hive-mongo-0.0.3.jar,guava-r06.jar,mongo-2.6.3.jar into hive_home/lib directory and also in hadoop_home/share/hadoop/mapred,hdfs,yarn (Hadoop 2.4). I created sample external hive-mongostorage table (TBL NAME : mongodb_archives) and then i insert overwrite into hive-mongostorage table ,its sending data into mongodbs collection and also i could query inserted data in collection the problem is when ever i query hive-mongostorage table i get this execption. hive> select * from mongodb_archives; OK Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.hive.serde2.ColumnProjectionUtils.getReadColumnIDs(Lorg/apache/hadoop/conf/Configuration;)Ljava/util/ArrayList; at org.yong3.hive.mongo.MongoInputFormat.getRecordReader(MongoInputFormat.java:39) at org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:498) at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:588) at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:561) at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:138) at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1621) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:267) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:199) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:410) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:783) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:616) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) [hadoop@hadoop ~]$ This exception made hive to drop out of cli Please help me out in this issue

Thanks, Divya N

— Reply to this email directly or view it on GitHub https://github.com/yc-huang/Hive-mongo/issues/17.

DivyaNagarajan commented 9 years ago

Hi , Thanks for your reply , please let me know when you have tested. Thanks,

yc-huang commented 9 years ago

Since hive 0.13.0 release, the return type of ColumnProjectionUtils.getReadColumnIDs changed from ArrayList to List, that why this failed.

So just change the version of referenced hive related jars in pom.xml to the same as your Hive's and rebuild should fix this issue.

yc-huang commented 9 years ago

pls have a try with the new 0.0.4 build under release directory, thanks

DivyaNagarajan commented 9 years ago

Hi,

I tried out with latest release and builded successfully , iam  getting  same error .

Thanks

yc-huang commented 9 years ago

I guess you do not perform mvn clean before rebuild this project... you might try my build, which you could get under /release directory.

DivyaNagarajan commented 9 years ago

Hi, I done "mvn clean" and rebuilded the project as you said. Now its working fine with out error and i could query table in hive and see the data.Thanks for your reply,its helps me a lot.

Thanks Divya N