search

Open-EO / openeo-geotrellis-extensions

Java/Scala extensions for Geotrellis, for use with OpenEO GeoPySpark backend.
Apache License 2.0
5 stars 3 forks source link

raster_to_vector process causes Stackoverflow #312

Open JeroenVerstraelen opened 1 month ago

JeroenVerstraelen commented 1 month ago

The raster_to_vector process causes a stackoverflow error on the executors when processing the collect() call at the following line:

    val collectedFeatures: Array[(String, List[PolygonFeature[Map[String, Int]]])] = vectorizedValuesWithMap.map(kv => {
      val key: K = kv._1
      val value: List[PolygonFeature[Map[String, Int]]] = kv._2
      val bandStr = "band0"
      key match {
        case stk: SpaceTimeKey =>
          val id = bandStr + "_" + stk.time.format(DateTimeFormatter.ofPattern("yyyyMMdd"))
          (id, value)
        case _ =>
          (bandStr, value)
      }
    }).collect()

This causes the vectorization process in the parcel delination notebook to fail: j-240731df876f46188b3737ed69e4b582

java.lang.StackOverflowError
at com.esotericsoftware.kryo.Kryo.getRegistration(Kryo.java:488)
at com.esotericsoftware.kryo.util.DefaultClassResolver.writeClass(DefaultClassResolver.java:97)
at com.esotericsoftware.kryo.Kryo.writeClass(Kryo.java:540)
at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:645)
at com.twitter.chill.TraversableSerializer.$anonfun$write$1(Traversable.scala:29)
at com.twitter.chill.TraversableSerializer.$anonfun$write$1$adapted(Traversable.scala:27)
at scala.collection.immutable.Map$Map1.foreach(Map.scala:193)
at com.twitter.chill.TraversableSerializer.write(Traversable.scala:27)
at com.twitter.chill.TraversableSerializer.write(Traversable.scala:21)
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:575)
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:79)
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:508)
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:575)
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:79)
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:508)
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:575)
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:79)
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:508)
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:575)
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:79)
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:508)
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:575)
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:79)
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:508)
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:575)
...