search

delta-incubator / delta-kernel-rs

A native Delta implementation for integration with any query engine
Apache License 2.0
121 stars 32 forks source link

We were passing the wrong offset value when computing list offsets #327

Closed nicklan closed 2 weeks ago

nicklan commented 3 weeks ago

Fix it, and add a test

codecov[bot] commented 3 weeks ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Please upload report for BASE (main@1459b30). Learn more about missing BASE report. Report is 3 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #327 +/- ## ======================================= Coverage ? 73.77% ======================================= Files ? 43 Lines ? 8061 Branches ? 8061 ======================================= Hits ? 5947 Misses ? 1735 Partials ? 379 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

nicklan commented 2 weeks ago

LGTM so found_fields.len() != parquet_index when we are skipping something in parquet schema (like requesting subset in the test?)

Yeah, found_fields just indicates which fields we were looking for and have found so far. So unless you're looking for all the fields that precede the list, the value would be incorrect.