delta-io / delta-rs

A native Rust library for Delta Lake, with bindings into Python

https://delta-io.github.io/delta-rs/

Apache License 2.0

1.97k stars 365 forks source link

fix(rust): inconsistent order of partitioning columns (#2494) #2614

Open aditanase opened 1 week ago

aditanase commented 1 week ago

Description

We need stable order between logical and physical schemas, but the order of partitioning columns is not always the same in the json schema and the array. This is visible only when you have more than one partitioning column.

This can surface in different ways:

select a partitioning column and get the values from the other
filter by a partitioning column and get the wrong results (#2494)

Related Issue(s)

closes #2494

github-actions[bot] commented 1 week ago

ACTION NEEDED

delta-rs follows the Conventional Commits specification for release automation.

The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification.