jorgecarleitao / arrow2

Transmute-free Rust library to work with the Arrow format
Apache License 2.0
1.07k stars 221 forks source link

fix: fix deserialization of parquets with large string list columns causing stack overflow #1575

Closed ByteNybbler closed 9 months ago

ByteNybbler commented 9 months ago

This mirrors a fix I did within polars: https://github.com/pola-rs/polars/pull/11471

codecov[bot] commented 9 months ago

Codecov Report

Attention: 2 lines in your changes are missing coverage. Please review.

Comparison is base (6271f48) 83.06% compared to head (d3cf22c) 83.07%. Report is 1 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1575 +/- ## ======================================= Coverage 83.06% 83.07% ======================================= Files 391 391 Lines 42898 42904 +6 ======================================= + Hits 35635 35641 +6 Misses 7263 7263 ``` | [Files](https://app.codecov.io/gh/jorgecarleitao/arrow2/pull/1575?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jorge+Leitao) | Coverage Δ | | |---|---|---| | [src/io/parquet/read/deserialize/binary/nested.rs](https://app.codecov.io/gh/jorgecarleitao/arrow2/pull/1575?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jorge+Leitao#diff-c3JjL2lvL3BhcnF1ZXQvcmVhZC9kZXNlcmlhbGl6ZS9iaW5hcnkvbmVzdGVkLnJz) | `81.19% <88.88%> (+0.49%)` | :arrow_up: | ... and [5 files with indirect coverage changes](https://app.codecov.io/gh/jorgecarleitao/arrow2/pull/1575/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jorge+Leitao)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

sundy-li commented 9 months ago

lint is unhappy.