[Kernel][Parquet Writer] Fix an issue with writing decimal as binary

delta-io / delta

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs

https://delta.io

Apache License 2.0

7.22k stars 1.62k forks source link

[Kernel][Parquet Writer] Fix an issue with writing decimal as binary #3233

Closed vkorukanti closed 2 weeks ago

vkorukanti commented 2 weeks ago

Description

The number of bytes needed to calculate the max buffer size needed when writing the decimal type to Parquet is off by one.

Resolved #3152

How was this patch tested?

Added unit tests that read and write decimals with various precision and scales.