chore(storage): hook recluster avoid scan all segments - Githubissues

databendlabs / databend

𝗗𝗮𝘁𝗮, 𝗔𝗻𝗮𝗹𝘆𝘁𝗶𝗰𝘀 & 𝗔𝗜. Modern alternative to Snowflake. Cost-effective and simple for massive-scale analytics. https://databend.com

https://docs.databend.com

Other

7.71k stars 732 forks source link

chore(storage): hook recluster avoid scan all segments #16498

Closed zhyass closed 1 day ago

zhyass commented 3 days ago

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

avoid scan all segments when do hook recluster
recluster will compact the small block and segments.
insert will trigger compact and recluster.

Tests

[ ] Unit Test
[x] Logic Test
[ ] Benchmark Test
[ ] No Test - Explain why

Type of change

[ ] Bug Fix (non-breaking change which fixes an issue)
[ ] New Feature (non-breaking change which adds functionality)
[ ] Breaking Change (fix or feature that could cause existing functionality not to work as expected)
[ ] Documentation Update
[ ] Refactoring
[ ] Performance Improvement
[x] Other (please describe):

This change is

zhyass commented 3 days ago

The new commit, PTAL @dantengsky https://github.com/datafuselabs/databend/pull/16498/commits/260874a24772ce6f48195343d119aecb83e8b182