Spark 3.4: Support Spark Column Stats

apache / iceberg

Apache Iceberg

https://iceberg.apache.org/

Apache License 2.0

6.49k stars 2.24k forks source link

Spark 3.4: Support Spark Column Stats #11532

Closed saitharun15 closed 1 week ago

saitharun15 commented 1 week ago

Backport of #10659 & #11035

Currently only Spark Action to Analyze table(#11106) is backported to 3.4, this PR backports changes related to reporting back Column Stats to Spark 3.4

@huaxingao @karuppayya @szehon-ho can u help review this

huaxingao commented 1 week ago

@saitharun15 https://github.com/apache/iceberg/pull/11035 fixed SparkScan to only read Apache DataSketches. Could you please back-port these changes too?