apache / doris

Apache Doris is an easy-to-use, high performance and unified analytics database.
https://doris.apache.org
Apache License 2.0
12.83k stars 3.3k forks source link

[Bug] runtime filter reports 'sync filter size meet error' when enable_pipeline_x_exec is off. #44730

Open dh-cloud opened 6 days ago

dh-cloud commented 6 days ago

Search before asking

Version

2.1.6

What's Wrong?

When enable_pipeline_exec is on but enable_pipeline_x_exec is off, two table join with in_or_bloom runtime filter will report error e.g.:

ERROR 1105(HY000):errCode = 2, detailMessage = (cdb-doris-benode-2.cdb-doris-benode.cloudspace.svc.cluster.local)[CANCELLED]sync filter size meet error,filter:RuntimeFilter:(id = 1,type = in_or_bloomfilter, need_local_merge: false, is_broadcast:false, build_bf_cardinality:true)

What You Expected?

no error report

How to Reproduce?

No response

Anything Else?

TIP

When enable_pipeline_exec is on but enable_pipeline_x_exec is on:

HashJoinBuildSinkOperatorX::sink() calls send_filter_size() when eos is on.

When enable_pipeline_exec is on but enable_pipeline_x_exec is off:

HashJoinNode::sink() does not call send_filter_size() when eos is on.

Are you willing to submit PR?

Code of Conduct

ixzc commented 1 day ago

you can avoid this by setting this set global runtime_filter_type = 'IN,MIN_MAX'; or upgrade your doris to 2.1.7 version

dh-cloud commented 1 day ago

or upgrade your doris to 2.1.7 version

@ixzc I see doris 2.17 has been forcibly set enable_pipeline_x_engine to true. Does that mean those code of pipeline on, pipeline_x off will not be maintained?