delta-io / delta

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
https://delta.io
Apache License 2.0
7.62k stars 1.71k forks source link

[Spark] Nicer error when failing to read table due to type change not support… #3728

Closed johanl-db closed 1 month ago

johanl-db commented 1 month ago

Description

Delta 3.2/3.3 only supports a limited subset of type changes that will become available with Delta 4.0 / Spark 4.0. This changes improves the error returned when reading a table with an unsupported type change to tell user to upgrade to Delta 4.0 in case the type change will be supported in that version.

How was this patch tested?

Added tests to cover the error path.

Does this PR introduce any user-facing changes?

Updates error thrown on unsupported type change when reading a table.