kayousterhout
commented
9 years ago I saw this again when running a job made up of very short tasks, which may be helpful for triggering this again in the future (it makes sense that short tasks would be more likely to trigger this, because it would be more likely that multiple tasks are modifying the job configuration).
ccanel
This is low priority and just happened once; filing this so we have a record of it in case it starts happening more persistently.
Stack trace:
java.util.ConcurrentModificationException at java.util.HashMap$HashIterator.nextEntry(HashMap.java:922) at java.util.HashMap$KeyIterator.next(HashMap.java:956) at java.util.AbstractCollection.addAll(AbstractCollection.java:341) at java.util.HashSet.(HashSet.java:117)
at org.apache.hadoop.conf.Configuration.(Configuration.java:545)
at org.apache.hadoop.mapred.JobConf.(JobConf.java:361)
at org.apache.hadoop.mapreduce.Job.getInstance(Job.java:175)
at org.apache.spark.mapreduce.SparkHadoopMapReduceUtil$class.getHadoopTaskObjects(SparkHadoopMapReduceUtil.scala:141)
at org.apache.spark.rdd.PairRDDFunctions.getHadoopTaskObjects(PairRDDFunctions.scala:73)
at org.apache.spark.rdd.PairRDDFunctions$$anonfun$12.apply(PairRDDFunctions.scala:998)
at org.apache.spark.rdd.PairRDDFunctions$$anonfun$12.apply(PairRDDFunctions.scala:996)
at org.apache.spark.monotasks.compute.ResultMonotask.getResult(ResultMonotask.scala:33)
at org.apache.spark.monotasks.compute.ExecutionMonotask.execute(ExecutionMonotask.scala:44)
at org.apache.spark.monotasks.compute.ComputeMonotask.executeAndHandleExceptions(ComputeMonotask.scala:52)
at org.apache.spark.monotasks.compute.ComputeScheduler$ConsumerThread.run(ComputeScheduler.scala:143)
at java.lang.Thread.run(Thread.java:745)