jintaoguan
commented
10 years ago I am using CDH4.4. I encountered a problem in the Hive join test.
This is the log.
ava.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"sourceip":"182.163.112.4","desturl":"nbizrgdziebsaecsecujfjcqtvnpcnxxwiopmddorcxnlijdizgoi","visitdate":"1978-10-17","adrevenue":0.332717,"useragent":"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.2)","countrycode":"BGR","languagecode":"BGR-BG","searchword":"rtzspywqgfplrlt","duration":10} at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:161) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:417) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408) at org.apache.hadoop.mapred.Child.main(Child.java:262)
This is the first MapReduce job of Hive join. The input data is generated by Hibench. I don't know if it is the same problem as what @jerryshao described.
Could someone tell me what to do with this error ?
hivebench external table is delimited by comma ',' , but in hivebench table field 'useragent' filed, there exists comma in the raw data, like "Mozilla/5.0 (Windows; U; Windows NT 5.2) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/xxx", so hive will delimit field by mistake, original filed useragent's data will wrongly delimit to the next field, and all the next left fields' data is wrong. like: 48.230.80.233 wyfctppjxtyhbcbngouswjzsekwdzqiaaapmomt 1976-04-18 0.44219965 Mozilla/5.0 (Windows; U; Windows NT 5.2) AppleWebKit/525.13 (KHTML like Gecko) Chrome/xxx LBY LBY-AR NULL