delta-io / delta

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
https://delta.io
Apache License 2.0
7.63k stars 1.71k forks source link

[Spark] Makes DataSkippingReader encoders lazy to prevent initialization failures #3733

Closed LukasRupprecht closed 1 month ago

LukasRupprecht commented 1 month ago

Which Delta project/connector is this regarding?

Description

This is a small fix that changes the sizeCollectorInputEncoders in DataSkippingReader to a lazy val. We are already doing this for other encoders in the codebases (e.g. see here) in order to prevent initialization failures of those encoders during JVM startup.

How was this patch tested?

Existing tests are sufficient as this does make any logical changes.

Does this PR introduce any user-facing changes?

No