search

databendlabs / databend

𝗗𝗮𝘁𝗮, 𝗔𝗻𝗮𝗹𝘆𝘁𝗶𝗰𝘀 & 𝗔𝗜. Modern alternative to Snowflake. Cost-effective and simple for massive-scale analytics. https://databend.com
https://docs.databend.com
Other
7.85k stars 750 forks source link

Feature: adaptive hash join runtime filter #16635

Open Dousir9 opened 4 weeks ago

Dousir9 commented 4 weeks ago

Summary

Enable hash join runtime filter adaptively during execution instead of when generating the physical plan.

xudong963 commented 4 weeks ago

In fact, runtime filter is generated during hash join runtime phase by the data of hash join build side.

Min-max index will always be generated (the main runtime filter type) Inlist index will be generated according to the data size of build side. Bloom filter will be generated if the join selectivity is ideal (it's not the main runtime filter, because it's not pushed down storage to prune parts)

So what kind of runtime filter type do you mean?