search

nexr / RHive

RHive is an R extension facilitating distributed computing via Apache Hive.
http://nexr.github.io/RHive
122 stars 63 forks source link

rhive.basic.cut failed due to ConcurrentModificationException #66

Open koje opened 9 years ago

koje commented 9 years ago

Hi,

i got a ConcurrentModificationException when I use the rhive.basic.cut operation. I'm using Hive 0.13.1 with Hadoop 1.12.

Do you have an idea why this error occurs?

Regards, koje

Here is the R-Code: rhive.basic.cut(tableName="ontime", col="arrdelay", breaks=seq(from=-1200,to=1500,by=15),right=T,forcedRef=T)

This is the exception on Hive:

`aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork) at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:82) at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:474) at org.apache.hive.com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:538) at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:61) at ...

org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:474) at org.apache.hive.com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:520) at org.apache.hadoop.hive.ql.exec.Utilities.serializeObjectByKryo(Utilities.java:895) at org.apache.hadoop.hive.ql.exec.Utilities.serializePlan(Utilities.java:799) at org.apache.hadoop.hive.ql.exec.Utilities.serializePlan(Utilities.java:811) at org.apache.hadoop.hive.ql.exec.Utilities.setBaseWork(Utilities.java:601) ... 24 more Caused by: java.util.ConcurrentModificationException at java.util.Vector$Itr.checkForComodification(Vector.java:1156) at java.util.Vector$Itr.next(Vector.java:1133) at org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:70) at org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:18) at org.apache.hive.com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:538) at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:61) ... 72 more Job Submission failed with exception 'java.lang.RuntimeException(org.apache.hive.com.esotericsoftware.kryo.KryoException: java.util.ConcurrentModificationException`

ajindia commented 9 years ago

Hey, did you ever figure out what the problem was here? I'm running into something similar

koje commented 9 years ago

No, I didn't found a solution for the Problem. I write a Hive-QL Query with the same functionality and use this query with rhive.query() function.