search

pingcap / tidb

TiDB - the open-source, cloud-native, distributed SQL database designed for modern applications.
https://pingcap.com
Apache License 2.0
37.29k stars 5.85k forks source link

6.0% performance regression in taobench in master branch #55785

Open owlsome2501 opened 2 months ago

owlsome2501 commented 2 months ago

Bug Report

1. Minimal reproduce step (Required)

  1. Deploy a cluster with 3 TiDB and 3 Tikv.
  2. Run benchmark taobench (workload_a).

2. What did you expect to see? (Required)

No performance regression.

3. What did you see instead (Required)

QPS regression 5.98% in benchmark taobench (workload_a) compared to baseline 24060.14.

description tidb tikv pd QPS degradation ratio
last success pingcap/tidb@d1a966dc7448e72381077344af94b67b8b155974 tikv/tikv@222e0a43215c68d4e6485cd24f8e902560ebb983 tikv/pd@09e272f5d932bffae5b5c9e8f908994a829b3b4c 22964.33 -4.6%
first regression pingcap/tidb@4c23efb10b32318de7dbec7c9bec80b43b396c78 tikv/tikv@222e0a43215c68d4e6485cd24f8e902560ebb983 tikv/pd@09e272f5d932bffae5b5c9e8f908994a829b3b4c 22620.72 -6.0%

4. What is your TiDB version? (Required)

pingcap/tidb@4c23efb10b32318de7dbec7c9bec80b43b396c78

D3Hunter commented 2 months ago

the regression is the accumulation result of multiple changes

D3Hunter commented 2 months ago
i test it again, the overall qps degradation compared with version 8.3.0 is 0.92%, and #55634 contributed 0.36% "bench_type" "bench_sub_type" "tidb_version" "thread" "duration(s)" "tps" "qps" "avg_latency(ms)" "P99_latency(ms)" "P999_latency(ms)"
taobench workload_a 8.4.0-4c23e (#55634) 15 2206 8922 13637 13.1 138 1159
taobench workload_a 8.4.0-d1a96(prev commit of #55634) 15 2207 8953 13687 13.1 138 1155
taobench workload_a 8.3.0 15 2207 9005 13764 13.0 136 1143
D3Hunter commented 2 months ago

compared with previous commit, https://github.com/pingcap/tidb/pull/55634 introduces 2 memory allocation, and cost of read from map, this mount of degradation is expected