[Spark] Relax protocol reader feature requirements to allow backward compatibility - Githubissues

delta-io / delta

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs

https://delta.io

Apache License 2.0

7.62k stars 1.71k forks source link

[Spark] Relax protocol reader feature requirements to allow backward compatibility #3747

Closed andreaschat-db closed 1 month ago

andreaschat-db commented 1 month ago

Which Delta project/connector is this regarding?

[x] Spark
[ ] Standalone
[ ] Flink
[ ] Kernel
[ ] Other (fill in here)

Description

This is a followup of https://github.com/delta-io/delta/commit/920f185a04382ff466e47e75240dfda48efe40d3. We relax the protocol reader feature requirements to allow backward compatibility with older writers. This is because before https://github.com/delta-io/delta/commit/920f185a04382ff466e47e75240dfda48efe40d3 was possible to serialize invalid (2, 7) protocols without the reader features set.

How was this patch tested?

Added new UTs.

Does this PR introduce any user-facing changes?

No.