gbif / occurrence

Occurrence store, download, search
Apache License 2.0
22 stars 15 forks source link

K8s: Spark ContainsUdf fails because of WKTReader and JtsShapeFactory serialization issue #329

Open muttcg opened 1 year ago

muttcg commented 1 year ago

WKTReader and JtsShapeFactory are not serializable and cannot be created outside the call method. Initializing these classes within the call method may(or not) potentially impact performance. Extending or wrapping them into serializable classes did not work, and neither did Spark broadcasting or wrapping with broadcasting features