search

rcongiu / Hive-JSON-Serde

Read - Write JSON SerDe for Apache Hive.
Other
733 stars 391 forks source link

Error in using JSON-serde with Hive 0.13. #93

Closed balasubram closed 9 years ago

balasubram commented 9 years ago

Whenever I tried to Create a table by specifying the RowFormat as 'org.openx.data.jsonserde.JsonSerDe' I got the following error 'FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org.apache.hadoop.hive.serde2.objectinspector.primitive.AbstractPrimitiveJavaObjectInspector.(Lorg/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils$PrimitiveTypeEntry;)V'.

Logs from Hive: 2014-10-15 03:40:54,207 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=TimeToSubmit start=1413344452196 end=1413344454207 duration=2011 from=org.apache.hadoop.hive.ql.Driver> 2014-10-15 03:40:54,208 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(108)) - 2014-10-15 03:40:54,208 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(108)) - 2014-10-15 03:40:54,389 ERROR [main]: exec.DDLTask (DDLTask.java:execute(479)) - java.lang.NoSuchMethodError: org.apache.hadoop.hive.serde2.objectinspector.primitive.AbstractPrimitiveJavaObjectInspector.(Lorg/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils$PrimitiveTypeEntry;)V at org.openx.data.jsonserde.objectinspector.JsonStringJavaObjectInspector.(JsonStringJavaObjectInspector.java:30) at org.openx.data.jsonserde.objectinspector.JsonObjectInspectorFactory.(JsonObjectInspectorFactory.java:168) at org.openx.data.jsonserde.JsonSerDe.initialize(JsonSerDe.java:125) at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:339) at org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:288) at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:281) at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:631) at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:594) at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:4221) at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:282) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1503) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1270) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1088) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:911) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:901) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:275) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:227) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:430) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:803) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:697) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:636) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

2014-10-15 03:40:54,390 ERROR [main]: ql.Driver (SessionState.java:printError(545)) - FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org.apache.hadoop.hive.serde2.objectinspector.primitive.AbstractPrimitiveJavaObjectInspector.(Lorg/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils$PrimitiveTypeEntry;)V

rcongiu commented 9 years ago

How did you compiled he serde ? For hive 0.13 you have to use the cdh5 serde: mvn -P cdh5 clean package

Sent from my iPhone, please excuse typos and autoincorrect

Il giorno Oct 15, 2014, alle ore 8:33 AM, balasubram notifications@github.com ha scritto:

Whenever I tried to Create a table by specifying the RowFormat as 'org.openx.data.jsonserde.JsonSerDe' I got the following error 'FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org.apache.hadoop.hive.serde2.objectinspector.primitive.AbstractPrimitiveJavaObjectInspector.(Lorg/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils$PrimitiveTypeEntry;)V'.

Logs from Hive: 2014-10-15 03:40:54,207 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - 2014-10-15 03:40:54,208 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(108)) - 2014-10-15 03:40:54,208 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(108)) - 2014-10-15 03:40:54,389 ERROR [main]: exec.DDLTask (DDLTask.java:execute(479)) - java.lang.NoSuchMethodError: org.apache.hadoop.hive.serde2.objectinspector.primitive.AbstractPrimitiveJavaObjectInspector.(Lorg/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils$PrimitiveTypeEntry;)V at org.openx.data.jsonserde.objectinspector.JsonStringJavaObjectInspector.(JsonStringJavaObjectInspector.java:30) at org.openx.data.jsonserde.objectinspector.JsonObjectInspectorFactory.(JsonObjectInspectorFactory.java:168) at org.openx.data.jsonserde.JsonSerDe.initialize(JsonSerDe.java:125) at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:339) at org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:288) at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:281) at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:631) at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:594) at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:4221) at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:282) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1503) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1270) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1088) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:911) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:901) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:275) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:227) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:430) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:803) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:697) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:636) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

2014-10-15 03:40:54,390 ERROR [main]: ql.Driver (SessionState.java:printError(545)) - FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org.apache.hadoop.hive.serde2.objectinspector.primitive.AbstractPrimitiveJavaObjectInspector.(Lorg/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils$PrimitiveTypeEntry;)V

— Reply to this email directly or view it on GitHub.

balasubram commented 9 years ago

I used -Pcdh5 clean package to build the project. Which jar should I consider? Is it under Hive-JSON-Serde\json-serde\target or Hive-JSON-Serde\json-serde-cdh5-shim\target. I am not using Cloudera.

rcongiu commented 9 years ago

The one under json-serde which has 'with-dependencies.jar' in the name. I would check the exact name but I am on vacation and won't have access to a computer for a few days:) Also make sure you uninstall any other version you may have installed.

Sent from my iPhone, please excuse typos and autoincorrect

Il giorno Oct 15, 2014, alle ore 8:56 AM, balasubram notifications@github.com ha scritto:

I used -Pcdh5 clean package to build the project. Which jar should I consider? Is it under Hive-JSON-Serde\json-serde or Hive-JSON-Serde\json-serde-cdh5-shim\target. I am not using Cloudera.

— Reply to this email directly or view it on GitHub.

balasubram commented 9 years ago

It is working. There was a json-serde-1.1.6 jar in the hive library path. After removing it, it works. Thanks. I am closing this issue.