This project is very helpful for us to deploy iceberg with spark sql support.
When we test join with icebergSQL, we found serialization issue as below. After we make DelegatedMethod0 extend Serializable, the problem is still there, but this time the unserializable method is java.lang.reflect.Method, which is unchangeable. Is there any way to workaround?
select * from store_sales_out a join store_sales b on a.ss_item_sk=b.ss_item_sk where a.ss_item_sk=17908
object not serializable (class: java.lang.reflect.Method, value: public abstract org.apache.spark.rdd.RDD org.apache.spark.sql.execution.SparkPlan.doExecute())
field (class: org.apache.spark.sql.iceberg.utils.DelegationUtils$DelegatedMethod0, name: method, type: class java.lang.reflect.Method)
This project is very helpful for us to deploy iceberg with spark sql support. When we test
join
with icebergSQL, we found serialization issue as below. After we make DelegatedMethod0 extend Serializable, the problem is still there, but this time the unserializable method is java.lang.reflect.Method, which is unchangeable. Is there any way to workaround?Caused by: java.io.NotSerializableException: org.apache.spark.sql.iceberg.utils.DelegationUtils$DelegatedMethod0 Serialization stack:
After we make DelegatedMethod0 extend Serializable
Caused by: java.io.NotSerializableException: java.lang.reflect.Method Serialization stack: