Open morningman opened 4 days ago
Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR.
Please clearly describe your PR:
run buildall
PR approved by at least one committer and no changes requested.
PR approved by anyone and no changes requested.
run buildall
run buildall
run buildall
PR approved by at least one committer and no changes requested.
What problem does this PR solve?
Problem Summary: When there is a
limit
cluse in SQL, if FE has obtained data with more than thelimit
number of rows, it should send a cancel command to BE to cancel the query to prevent BE from reading more data. However, this function has problems in the current code and does not work. Especially in external table query, this may result in lots of unnecessary network io read.isBlockQuery
In the old optimizer, if a query statement contains a
sort
oragg
node,isBlockQuery
will be marked as true, otherwise it will be false. In the new optimizer, this value is always true.Regardless of the old or new optimizer, this logic is wrong. But only when
isBlockQuery = false
will the reach limit logic be triggered.Calling problem of reach limit logic
The reach limit logic judgment will only be performed when
eos = true
in the rowBatch returned by BE. This is wrong. Because forlimit N
queries, each BE's ownlimit
is N. But for FE, as long as the total number of rows returned by all BEs exceeds N, the reach limit logic can be triggered. So it should not be processed only wheneos = true
.The PR mainly changes:
Remove
isBlockQuery
isBlockQuery
is only used in the reach limit logic. And it is not needed. Remove it completely.Modify the judgment position of reach limit.
When the number of rows obtained by FE is greater than the limit, it will check the reach limit logic.
fix wrong
limitRows
inQueryProcessor
the limitRows should be got from the first fragment, not last.
Release note
fix query should be cancelled if limit reached
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)