timesking / hive-json-serde

Automatically exported from code.google.com/p/hive-json-serde
0 stars 0 forks source link

SerDe does not work for any MapReduce job #14

Open GoogleCodeExporter opened 8 years ago

GoogleCodeExporter commented 8 years ago
What steps will reproduce the problem?
1. create any table using SerDe (for my it was the one from "Getting Started" 
section)
2. execute in hive: select count(1) from table;

What is the expected output? What do you see instead?
I expect to see number of records. Instead I got error:

hive> select count(1) from jsontest;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Starting Job = job_201205071112_0089, Tracking URL = [...]
Kill Command = /usr/lib/hadoop/bin/hadoop job  -Dmapred.job.tracker=[...] -kill 
job_201205071112_0089
2012-06-06 09:28:49,506 Stage-1 map = 0%,  reduce = 0%
2012-06-06 09:29:26,655 Stage-1 map = 100%,  reduce = 100%
Ended Job = job_201205071112_0089 with errors
FAILED: Execution Error, return code 2 from 
org.apache.hadoop.hive.ql.exec.MapRedTask

What version of the product are you using? On what operating system?
Hadoop: Hadoop 0.20.2-cdh3u3
Hive: 0.7.0-cdh3u0
OS: Ubuntu Server

Please provide any additional information below.
Full stack trace from jobtracker can be found in attached file.
"select * from jsontest" works fine.

Original issue reported on code.google.com by michalba...@gmail.com on 6 Jun 2012 at 7:39

Attachments:

GoogleCodeExporter commented 8 years ago
here you can see various options to put any custom jar file to all tasktrackers 
and solve this problem (worked for me)
http://blog.cloudera.com/blog/2011/01/how-to-include-third-party-libraries-in-yo
ur-map-reduce-job/

Original comment by mar...@vast.com on 26 Dec 2012 at 10:02