pingcap / tiflash

The analytical engine for TiDB and TiDB Cloud. Try free: https://tidbcloud.com/free-trial
https://docs.pingcap.com/tidb/stable/tiflash-overview
Apache License 2.0
937 stars 409 forks source link

Storages: Splitting DMFileReaderPool to reduce lock contention #9126

Closed JinheLin closed 1 month ago

JinheLin commented 1 month ago

What problem does this PR solve?

Issue Number: close #9125

Problem Summary:

What is changed and how it works?

Reduce the lock contention on DMFileReaderPool::mtx by splitting the dmfiles by hash(file_path) of 16

Check List

Tests

Concurrency   Master This PR  
1 27 25
2 28 29
4 46 46
8 72 77
16 124 97
32 232 160
64 450 327

Side effects

Documentation

Release note

Improve the short queries performance under high concurrency by reducing the lock contention
ti-chi-bot[bot] commented 1 month ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: JaySon-Huang, Lloyd-Pottiger

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/pingcap/tiflash/blob/master/OWNERS)~~ [JaySon-Huang,Lloyd-Pottiger] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
ti-chi-bot[bot] commented 1 month ago

[LGTM Timeline notifier]

Timeline:

ti-chi-bot commented 1 month ago

In response to a cherrypick label: new pull request created to branch release-8.1: #9129.

ti-chi-bot commented 1 month ago

In response to a cherrypick label: new pull request created to branch release-7.5: #9130.