vmware-archive / geode-spark-connector

This is a fork of the Apache Geode spark connector from tags/rel/v1.1.1 - the intent is to update it to a current version of scala and geode / gemfire. This project is open source and if you would like to contribute feel free to open a pull request.
Apache License 2.0
3 stars 7 forks source link

saveToGeode("region-name") Task not serializable #1

Closed zhansen-pivotal closed 6 years ago

zhansen-pivotal commented 6 years ago

`scala> distData.saveToGeode("str_str_region") org.apache.spark.SparkException: Task not serializable at org.apache.spark.util.ClosureCleaner$.ensureSerializable(ClosureCleaner.scala:345) at org.apache.spark.util.ClosureCleaner$.org$apache$spark$util$ClosureCleaner$$clean(ClosureCleaner.scala:335) at org.apache.spark.util.ClosureCleaner$.clean(ClosureCleaner.scala:159) at org.apache.spark.SparkContext.clean(SparkContext.scala:2292) at org.apache.spark.SparkContext.runJob(SparkContext.scala:2022) at org.apache.spark.SparkContext.runJob(SparkContext.scala:2048) at org.apache.spark.SparkContext.runJob(SparkContext.scala:2080) at org.apache.geode.spark.connector.GeodePairRDDFunctions.saveToGeode(GeodePairRDDFunctions.scala:49) ... 51 elided Caused by: java.io.NotSerializableException: org.apache.log4j.Logger Serialization stack:

zhansen-pivotal commented 6 years ago

fixed with commit 23e385f653f95a5f14b723962b7d95543b4ad684