delta-io / delta-rs

A native Rust library for Delta Lake, with bindings into Python

https://delta-io.github.io/delta-rs/

Apache License 2.0

2.33k stars 411 forks source link

fix(python, rust): use input schema to get correct schema in cdf reads #2723

Closed ion-elgreco closed 3 months ago

ion-elgreco commented 3 months ago

Description

The arrow schema method wraps partition values, but these then incorrectly get read as Categorical/Dictionary values. Using the input schema fixes this.