pingcap / tiflash

The analytical engine for TiDB and TiDB Cloud. Try free: https://tidbcloud.com/free-trial
https://docs.pingcap.com/tidb/stable/tiflash-overview
Apache License 2.0
944 stars 410 forks source link

Storages: Skip filtering for filter column #9361

Closed JinheLin closed 1 month ago

JinheLin commented 1 month ago

What problem does this PR solve?

Issue Number: close #9348, ref https://github.com/pingcap/tiflash/issues/9103

Problem Summary:

If Min-Max index returns RSResult::All, we will create a EMPTY filter column to make some checks on block structure happy. https://github.com/pingcap/tiflash/blob/f0a4087d599a19645820b0bc9c48d9b80906541f/dbms/src/DataStreams/FilterTransformAction.cpp#L77-L85

In LateMaterializationBlockInputStream, filter_column_stream does not execute projecting to remove filter column, and will execute filtering in https://github.com/pingcap/tiflash/blob/f0a4087d599a19645820b0bc9c48d9b80906541f/dbms/src/Storages/DeltaMerge/LateMaterializationBlockInputStream.cpp#L72-L75

This result issue #9348.

What is changed and how it works?

Skip filtering for filter column as https://github.com/pingcap/tiflash/blob/f0a4087d599a19645820b0bc9c48d9b80906541f/dbms/src/Storages/DeltaMerge/LateMaterializationBlockInputStream.cpp#L113-L117

Add add unit-test for this case.

Check List

Tests

Side effects

Documentation

Release note

None
ti-chi-bot[bot] commented 1 month ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: CalvinNeo, Lloyd-Pottiger

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/pingcap/tiflash/blob/master/OWNERS)~~ [CalvinNeo,Lloyd-Pottiger] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
ti-chi-bot[bot] commented 1 month ago

[LGTM Timeline notifier]

Timeline:

Lloyd-Pottiger commented 1 week ago

/cherry-pick release-7.5

ti-chi-bot commented 1 week ago

@Lloyd-Pottiger: new pull request created to branch release-7.5: #9480.

In response to [this](https://github.com/pingcap/tiflash/pull/9361#issuecomment-2378351535): >/cherry-pick release-7.5 Instructions for interacting with me using PR comments are available [here](https://prow.tidb.net/command-help). If you have questions or suggestions related to my behavior, please file an issue against the [ti-community-infra/tichi](https://github.com/ti-community-infra/tichi/issues/new?title=Prow%20issue:) repository.
Lloyd-Pottiger commented 1 week ago

/cherry-pick release-8.1

Lloyd-Pottiger commented 1 week ago

/cherry-pick release-7.1

ti-chi-bot commented 1 week ago

@Lloyd-Pottiger: new pull request created to branch release-8.1: #9481.

In response to [this](https://github.com/pingcap/tiflash/pull/9361#issuecomment-2378352850): >/cherry-pick release-8.1 Instructions for interacting with me using PR comments are available [here](https://prow.tidb.net/command-help). If you have questions or suggestions related to my behavior, please file an issue against the [ti-community-infra/tichi](https://github.com/ti-community-infra/tichi/issues/new?title=Prow%20issue:) repository.
ti-chi-bot commented 1 week ago

@Lloyd-Pottiger: new pull request created to branch release-7.1: #9482.

In response to [this](https://github.com/pingcap/tiflash/pull/9361#issuecomment-2378352907): >/cherry-pick release-7.1 Instructions for interacting with me using PR comments are available [here](https://prow.tidb.net/command-help). If you have questions or suggestions related to my behavior, please file an issue against the [ti-community-infra/tichi](https://github.com/ti-community-infra/tichi/issues/new?title=Prow%20issue:) repository.