pingcap / tiflash

The analytical engine for TiDB and TiDB Cloud. Try free: https://tidbcloud.com/free-trial
https://docs.pingcap.com/tidb/stable/tiflash-overview
Apache License 2.0
936 stars 409 forks source link

auto pass through hashagg #9167

Open guo-shaoge opened 1 week ago

guo-shaoge commented 1 week ago

What problem does this PR solve?

Issue Number: close #xxx

Problem Summary:

What is changed and how it works?

This PR mainly includes four modifications:

  1. AutoPassThroughHashAggContext: Includes the state control logic, with five states: Init, Adjust, Selective, PreHashAgg, and PassThrough. State transitions occur based on the pre-aggregation effect of each block.
  2. AutoPassThroughHashAggBlockInputStream/AutoPassThroughAggregateTransform: Executor logic that call the AutoPassThroughHashAggContext. It can support sending blocks in advance.
  3. ExchangeSenderWriter: Includes modifications of HashPartitionWriter and FineGrainedShuffleWriter to support processing block.info.selective, which means only sending rows specified in the selective information.
  4. Column: Added support for selective versions of IColumn::updateWeakHash32 and IColumn::scatter for use by ExchangeSenderWriter.

Check List

Tests

Side effects

Documentation

Release note

support auto pass through hashagg
ti-chi-bot[bot] commented 1 week ago

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Once this PR has been reviewed and has the lgtm label, please ask for approval from guo-shaoge, ensuring that each of them provides their approval before proceeding. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files: - **[OWNERS](https://github.com/pingcap/tiflash/blob/master/OWNERS)** Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
ti-chi-bot[bot] commented 14 hours ago

[FORMAT CHECKER NOTIFICATION]

Notice: To remove the do-not-merge/needs-linked-issue label, please provide the linked issue number on one line in the PR body, for example: Issue Number: close #123 or Issue Number: ref #456.

:open_book: For more info, you can check the "Contribute Code" section in the development guide.