pingcap / tidb

TiDB is an open-source, cloud-native, distributed, MySQL-Compatible database for elastic scale and real-time analytics. Try AI-powered Chat2Query free at : https://www.pingcap.com/tidb-serverless/
https://pingcap.com
Apache License 2.0
36.32k stars 5.72k forks source link

*: implement global goroutine pool (#53299) #53346

Closed you06 closed 1 week ago

you06 commented 2 weeks ago

cherry pick #53299 to release-8.1 branch

What problem does this PR solve?

Issue Number: close #52887

Problem Summary:

What changed and how does it work?

Goroutines with short lifetime often release its stack soon after several stack grows, this made newstack a performance issue. This PR create a global goroutine pool, reuse the stack, which shows a great improvement, high QPS with lower CPU utilization.

Check List

Tests

Sysbench read only(with --point-selects=0)

thread nightly this pr diff
100 105052 qps 110537 qps +5.2%
200 100057 qps 121466 qps +21.4%

TPC-C

nightly this pr diff
106864 tpmC 115495 tpmC +8.1%

Side effects

Documentation

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

Optimize performance by a global goroutine pool. 
ti-chi-bot[bot] commented 2 weeks ago

This cherry pick PR is for a release branch and has not yet been approved by triage owners. Adding the do-not-merge/cherry-pick-not-approved label.

To merge this cherry pick:

  1. It must be approved by the approvers firstly.
  2. AFTER it has been approved by approvers, please wait for the cherry-pick merging approval from triage owners.
Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
ti-chi-bot[bot] commented 2 weeks ago

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Once this PR has been reviewed and has the lgtm label, please assign tangenta for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files: - **[OWNERS](https://github.com/pingcap/tidb/blob/release-8.1/OWNERS)** - **[pkg/resourcemanager/OWNERS](https://github.com/pingcap/tidb/blob/release-8.1/pkg/resourcemanager/OWNERS)** Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
tiprow[bot] commented 2 weeks ago

Hi @you06. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
codecov[bot] commented 2 weeks ago

Codecov Report

Attention: Patch coverage is 48.33333% with 31 lines in your changes are missing coverage. Please review.

Please upload report for BASE (release-8.1@d4fe60e). Learn more about missing BASE report.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## release-8.1 #53346 +/- ## ================================================ Coverage ? 71.1591% ================================================ Files ? 1466 Lines ? 421464 Branches ? 0 ================================================ Hits ? 299910 Misses ? 101087 Partials ? 20467 ``` | [Flag](https://app.codecov.io/gh/pingcap/tidb/pull/53346/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap) | Coverage Δ | | |---|---|---| | [unit](https://app.codecov.io/gh/pingcap/tidb/pull/53346/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap) | `71.1591% <48.3333%> (?)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap#carryforward-flags-in-the-pull-request-comment) to find out more. | [Components](https://app.codecov.io/gh/pingcap/tidb/pull/53346/components?src=pr&el=components&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap) | Coverage Δ | | |---|---|---| | [dumpling](https://app.codecov.io/gh/pingcap/tidb/pull/53346/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap) | `53.9957% <0.0000%> (?)` | | | [parser](https://app.codecov.io/gh/pingcap/tidb/pull/53346/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap) | `∅ <0.0000%> (?)` | | | [br](https://app.codecov.io/gh/pingcap/tidb/pull/53346/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap) | `40.9637% <0.0000%> (?)` | |
you06 commented 1 week ago

Close thie PR because we won't backport it to 8.1, supress the regression by #53429.