delta-io / delta

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
https://delta.io
Apache License 2.0
7.22k stars 1.62k forks source link

[Spark] Backport fix to clustering column validation #3279

Closed chirag-s-db closed 1 week ago

chirag-s-db commented 1 week ago

Currently, clustering columns are validating by parsing a list of clustering columns. This is super brittle, and breaks when any clustering column has a comma in the name. Fix that by passing a list of clustering columns directly.

This fix resolves https://github.com/delta-io/delta/issues/3265

This fix is backported from https://github.com/delta-io/delta/commit/8c7b62e1a39a27e7108208a8d921e8de07b60ff2