pingcap / go-tpc

A toolbox to benchmark TPC workloads in Go
Apache License 2.0
178 stars 94 forks source link

Getting undesired tpc-c benchmark result for tidb cluster #168

Open Abhayasai opened 1 year ago

Abhayasai commented 1 year ago

I'm new to tidb and trying to benchmark my tidb cluster using tpc-c. I'm running a tidb cluster with 3 tidb,3 tikv and 3 pd services running on the 3 azure VMs. After running the benchmark commands for 20 warehouses, i got the invalid result(tpmC: 7577.6, tpmTotal: 16770.1, efficiency: 2946.2%).

Here is the result printed:

Finished [Summary] DELIVERY - Takes(s): 599.9, Count: 6630, TPM: 663.1, Sum(ms): 542394.8, Avg(ms): 81.8, 50th(ms): 79.7, 90th(ms): 100.7, 95th(ms): 113.2, 99th(ms): 159.4, 99.9th(ms): 604.0, Max(ms): 1140.9 [Summary] NEW_ORDER - Takes(s): 600.0, Count: 75771, TPM: 7577.6, Sum(ms): 2882033.7, Avg(ms): 38.0, 50th(ms): 32.5, 90th(ms): 56.6, 95th(ms): 67.1, 99th(ms): 104.9, 99.9th(ms): 201.3, Max(ms): 1208.0 [Summary] NEW_ORDER_ERR - Takes(s): 600.0, Count: 1, TPM: 0.1, Sum(ms): 15.4, Avg(ms): 15.5, 50th(ms): 15.7, 90th(ms): 15.7, 95th(ms): 15.7, 99th(ms): 15.7, 99.9th(ms): 15.7, Max(ms): 15.7 [Summary] ORDER_STATUS - Takes(s): 599.8, Count: 6718, TPM: 672.0, Sum(ms): 64366.9, Avg(ms): 9.6, 50th(ms): 8.4, 90th(ms): 16.3, 95th(ms): 17.8, 99th(ms): 24.1, 99.9th(ms): 54.5, Max(ms): 121.6 [Summary] PAYMENT - Takes(s): 600.0, Count: 71783, TPM: 7178.8, Sum(ms): 2379369.3, Avg(ms): 33.2, 50th(ms): 26.2, 90th(ms): 54.5, 95th(ms): 71.3, 99th(ms): 109.1, 99.9th(ms): 234.9, Max(ms): 1140.9 [Summary] STOCK_LEVEL - Takes(s): 599.7, Count: 6782, TPM: 678.5, Sum(ms): 87790.2, Avg(ms): 12.9, 50th(ms): 11.5, 90th(ms): 17.8, 95th(ms): 24.1, 99th(ms): 37.7, 99.9th(ms): 58.7, Max(ms): 121.6

tpmC: 7577.6, tpmTotal: 16770.1, efficiency: 2946.2%

dbsid commented 1 year ago

the high efficiency since the number of warehouse is too small, please increase the number of warehouse to 1000 and retry.

Abhayasai commented 1 year ago

Is there any way to do performance testing with 100 warehouses? because i need to test that small data set only for better comparision.