delta-io / delta

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
https://delta.io
Apache License 2.0
6.96k stars 1.58k forks source link

[Spark] ALTER TABLE ALTER COLUMN SYNC IDENTITY SQL support #3005

Open c27kwan opened 2 weeks ago

c27kwan commented 2 weeks ago

Which Delta project/connector is this regarding?

Description

This PR is part of https://github.com/delta-io/delta/issues/1959 In this PR, we add SQL support for ALTER TABLE ALTER COLUMN SYNC IDENTITY.

This is used for GENERATED BY DEFAULT Identity Columns, where a user may want to manually update the identity column high watermark.

How was this patch tested?

This PR adds a new test suite IdentityColumnSyncSuite.

Does this PR introduce any user-facing changes?

Yes. We introduce the SQL syntax ALTER TABLE (ALTER| CHANGE) COLUMN? <colName> SYNC IDENTITY into Delta. This will update the high watermark stored in the metadata for that specific identity column. Example Usage

ALTER TABLE ALTER COLUMN id SYNC IDENTITY
ALTER TABLE CHANGE COLUMN id SYNC IDENTITY
ALTER TABLE ALTER id SYNC IDENTITY
ALTER TABLE CHANGE id SYNC IDENTITY