datafuselabs / databend

๐——๐—ฎ๐˜๐—ฎ, ๐—”๐—ป๐—ฎ๐—น๐˜†๐˜๐—ถ๐—ฐ๐˜€ & ๐—”๐—œ. Modern alternative to Snowflake. Cost-effective and simple for massive-scale analytics. https://databend.com
https://docs.databend.com
Other
7.29k stars 701 forks source link

fix: check all partitions for small files instead of the last only. #15360

Closed youngsofun closed 2 weeks ago

youngsofun commented 2 weeks ago

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

Although ParquetFilesParts are always staying at the end of parts when do_read_partitions, but parts are reshuffled when redistribute_source_fragment, so let us check all of them.

Tests

Type of change


This change isโ€‚Reviewable