delta-io / delta-rs

A native Rust library for Delta Lake, with bindings into Python
https://delta-io.github.io/delta-rs/
Apache License 2.0
2.2k stars 395 forks source link

fix: enable field_with_name to support nested fields with '.' delimiter #2519

Closed alexwilcoxson-rel closed 4 months ago

alexwilcoxson-rel commented 4 months ago

Description

field_with_name on schema type does not handle nested field names. This comes up when setting the data skipping stats column property. For example when I set this value to parent.child, the stats handling code in table load/scan throws an error that it cannot find a field named parent.child.

Area where we would see the error: https://github.com/delta-io/delta-rs/blob/fae1406e91c7ac5188b64827eba11dbbb2acce59/crates/core/src/kernel/snapshot/mod.rs#L296