[Spark][3.2] Support RESTORE for clustered table

delta-io / delta

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs

https://delta.io

Apache License 2.0

7.22k stars 1.62k forks source link

[Spark][3.2] Support RESTORE for clustered table #3239

Closed zedtang closed 2 weeks ago

zedtang commented 2 weeks ago

Which Delta project/connector is this regarding?

[x] Spark
[ ] Standalone
[ ] Flink
[ ] Kernel
[ ] Other (fill in here)

Description

Backported from https://github.com/delta-io/delta/commit/ef1def9e3c97d80518dcb9b5cd48f58e8e859bd8

Support RESTORE for clustered tables by adding a new domain metadata to overwrite the existing one so that clustering columns are correctly restored.

How was this patch tested?

New unit tests.

Does this PR introduce any user-facing changes?

felipepessoto commented 11 hours ago

Hi @zedtang do you have plans to release a 3.2.1 version soon to include the latest fixes? I think we have at least two more fixes in 3.2.1-SNAPSHOT.

Thanks