apache / drill

Apache Drill is a distributed MPP query layer for self describing data
https://drill.apache.org/
Apache License 2.0
1.93k stars 980 forks source link

DRILL-8416: Memory leak when the async Parquet reader skips empty pages #2784

Closed jnturton closed 1 year ago

jnturton commented 1 year ago

DRILL-8416: Memory leak when the async Parquet reader skips empty pages

Description

A regression introduced by the Parquet reader clean-up released in Drill 1.20 has meant that buffers used for (non-empty) compressed data holding empty dictionary or data pages which are skipped are not freed. Because empty pages are uncommon in real data this bug went undetected for a long time.

Documentation

N/A

Testing

New unit test.