apache / parquet-java

Apache Parquet Java
https://parquet.apache.org/
Apache License 2.0
2.65k stars 1.41k forks source link

MINOR: Optimize bit-packing decoder performance by increasing unpackCount in Packer16 #3062

Closed 1111nit closed 4 days ago

1111nit commented 1 week ago

Rationale for this change The current Packer16 decoder implementation using JDK Vector API has performance limitations with unpackCount = 16 for bit-packing decoding.

What changes are included in this PR? Modified ByteBitPacking512VectorLE.java to increase the processing batch size: // Before private static final int UNPACK_COUNT = 16; // After private static final int UNPACK_COUNT = 32;

Testing Details Testing environment:

Performance results:

wgtmac commented 1 week ago

cc @jatin-bhateja