Closed jaychia closed 11 months ago
Also happy to move the int96_to_i64_*s
logic over into the parquet2 repo!
Also happy to move the int96_toi64*s
I believe that all arrow datatype related conversions are done here, so that is fine.
Patch coverage: 100.00%
and project coverage change: -0.66%
:warning:
Comparison is base (
b09e580
) 83.73% compared to head (b749735
) 83.07%. Report is 49 commits behind head on main.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
Second PR is up! #1533
@ritchie46 any chance you could retrigger CI - Seems to be failing on some flaky token issues?
{'detail': ErrorDetail(string='Unable to locate build via Github Actions API. Please upload with the Codecov repository upload token to resolve issue.', code='not_found')}
Thanks for the stamp @sundy-li!
I had to make a new PR to fix lints since they were re-enabled last week. Would appreciate some help with launching CI :)
Merged, will create a new pr to fix the clippy.
Addresses the first part of issue #1527
Instead of always naively parsing Parquet Int96 timestamps into
TimeUnit::Nanosecond
, we match on the requested timeunit and perform timeunit-specific parsingThis makes parsing safer when reading Int96 timestamps that are outside of the range of
timestamp[ns]
(e.g. timestamps with dates like the years 1000 or 3000) instead of the current behavior which is to always parse the timestamps with the Nanosecond timeunit, and overflow.