search

apache / doris

Apache Doris is an easy-to-use, high performance and unified analytics database.
https://doris.apache.org
Apache License 2.0
11.8k stars 3.11k forks source link

[branch-2.0][improvement](jdbc catalog) Modify the maximum number of connections in the connection pool to 30 by default #37024

Closed zy-kkk closed 14 hours ago

zy-kkk commented 3 days ago

pick (#36720)

In many cases, we found that users would use JDBC Catalog to perform a large number of queries, which resulted in the maximum of 10 connections being insufficient, so I adjusted it to 30, which covered most needs.

doris-robot commented 3 days ago

Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website. See Doris Document.

zy-kkk commented 3 days ago

run buildall

doris-robot commented 3 days ago
TPC-H: Total hot run time: 49779 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit c2c934f6d3d4cbbed543081ca7f3c70428da6a44, data reload: false ------ Round 1 ---------------------------------- q1 18050 4545 4325 4325 q2 2071 158 145 145 q3 10483 1894 1968 1894 q4 10347 1289 1342 1289 q5 8413 3878 3987 3878 q6 239 157 131 131 q7 2095 1641 1580 1580 q8 9552 2754 2717 2717 q9 13836 10317 10308 10308 q10 8630 3543 3542 3542 q11 415 249 246 246 q12 468 310 309 309 q13 18339 3957 4052 3957 q14 362 323 323 323 q15 523 473 465 465 q16 663 570 573 570 q17 1140 935 892 892 q18 7264 6798 6958 6798 q19 1804 1694 1607 1607 q20 519 317 307 307 q21 4454 4132 4057 4057 q22 530 456 439 439 Total cold run time: 120197 ms Total hot run time: 49779 ms ----- Round 2, with runtime_filter_mode=off ----- q1 4326 4368 4326 4326 q2 329 215 214 214 q3 4194 4150 4145 4145 q4 2754 2755 2754 2754 q5 7169 7075 7048 7048 q6 245 122 127 122 q7 3237 2820 2925 2820 q8 4354 4432 4512 4432 q9 16987 16774 16683 16683 q10 4261 4307 4305 4305 q11 767 690 706 690 q12 1032 875 859 859 q13 7127 3746 3756 3746 q14 452 423 419 419 q15 508 464 458 458 q16 736 683 686 683 q17 3857 3810 3884 3810 q18 8796 8725 8766 8725 q19 1745 1665 1672 1665 q20 2357 2113 2158 2113 q21 8452 8475 8601 8475 q22 1044 963 1005 963 Total cold run time: 84729 ms Total hot run time: 79455 ms ```
doris-robot commented 3 days ago
TPC-DS: Total hot run time: 205576 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit c2c934f6d3d4cbbed543081ca7f3c70428da6a44, data reload: false query1 940 430 382 382 query2 6541 2934 2627 2627 query3 6921 206 210 206 query4 20582 18008 17985 17985 query5 19726 6477 6578 6477 query6 301 225 236 225 query7 4164 309 312 309 query8 441 423 424 423 query9 3144 2684 2635 2635 query10 402 324 293 293 query11 11388 10696 10785 10696 query12 129 78 75 75 query13 5602 699 693 693 query14 17718 13601 13954 13601 query15 361 246 239 239 query16 6471 308 268 268 query17 1719 1446 889 889 query18 2309 413 414 413 query19 215 149 152 149 query20 80 78 83 78 query21 195 97 99 97 query22 5261 5107 5131 5107 query23 32740 31970 32019 31970 query24 7071 6530 6510 6510 query25 556 450 440 440 query26 609 167 164 164 query27 1993 294 301 294 query28 6113 2355 2303 2303 query29 2842 2811 2742 2742 query30 247 168 166 166 query31 887 722 746 722 query32 68 63 60 60 query33 400 263 270 263 query34 839 476 487 476 query35 1116 929 933 929 query36 1307 1306 1113 1113 query37 88 62 60 60 query38 3055 2910 2902 2902 query39 1378 1336 1340 1336 query40 199 97 96 96 query41 50 43 44 43 query42 82 82 89 82 query43 726 819 682 682 query44 1139 707 716 707 query45 248 230 237 230 query46 1219 955 955 955 query47 1873 1801 1774 1774 query48 1027 722 708 708 query49 615 374 379 374 query50 868 622 602 602 query51 4738 4631 4627 4627 query52 103 81 87 81 query53 454 334 335 334 query54 2670 2427 2500 2427 query55 85 80 77 77 query56 246 223 218 218 query57 1229 1119 1206 1119 query58 228 190 209 190 query59 4609 4337 3956 3956 query60 222 218 200 200 query61 98 93 95 93 query62 792 484 492 484 query63 497 345 346 345 query64 2591 1528 1401 1401 query65 3658 3576 3579 3576 query66 813 399 387 387 query67 15460 15653 15516 15516 query68 8673 676 645 645 query69 571 339 352 339 query70 1527 1491 1451 1451 query71 405 301 307 301 query72 6479 3509 3452 3452 query73 727 323 320 320 query74 6272 5820 5850 5820 query75 5361 3771 3710 3710 query76 5413 1193 1197 1193 query77 853 250 266 250 query78 12621 12824 13638 12824 query79 12246 650 638 638 query80 916 396 397 396 query81 500 234 233 233 query82 637 99 100 99 query83 181 135 134 134 query84 259 75 69 69 query85 752 317 313 313 query86 337 308 307 307 query87 3237 3061 2999 2999 query88 6015 2328 2338 2328 query89 378 290 278 278 query90 2595 208 218 208 query91 171 140 148 140 query92 58 52 51 51 query93 4442 553 607 553 query94 1187 208 208 208 query95 1090 1060 1072 1060 query96 642 321 332 321 query97 6478 6373 6374 6373 query98 198 178 171 171 query99 2709 791 871 791 Total cold run time: 316388 ms Total hot run time: 205576 ms ```
doris-robot commented 3 days ago
ClickBench: Total hot run time: 30.51 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit c2c934f6d3d4cbbed543081ca7f3c70428da6a44, data reload: false query1 0.02 0.02 0.02 query2 0.07 0.03 0.03 query3 0.25 0.05 0.04 query4 1.79 0.07 0.07 query5 0.54 0.52 0.51 query6 1.24 0.62 0.61 query7 0.01 0.01 0.01 query8 0.03 0.03 0.02 query9 0.52 0.49 0.47 query10 0.57 0.55 0.53 query11 0.12 0.09 0.08 query12 0.12 0.10 0.09 query13 0.62 0.62 0.61 query14 0.80 0.78 0.80 query15 0.78 0.76 0.75 query16 0.36 0.38 0.36 query17 1.00 1.00 1.03 query18 0.24 0.25 0.25 query19 1.94 1.83 1.90 query20 0.02 0.01 0.01 query21 15.46 0.56 0.55 query22 2.16 2.18 1.44 query23 17.03 1.08 0.95 query24 7.13 0.87 0.98 query25 0.37 0.07 0.05 query26 0.78 0.16 0.15 query27 0.04 0.05 0.04 query28 6.25 0.73 0.77 query29 12.62 2.32 2.30 query30 0.61 0.53 0.56 query31 2.81 0.39 0.38 query32 3.38 0.49 0.49 query33 3.05 3.10 3.08 query34 15.26 4.81 4.78 query35 4.87 4.84 4.86 query36 1.04 1.01 1.01 query37 0.06 0.04 0.05 query38 0.03 0.02 0.02 query39 0.02 0.01 0.01 query40 0.16 0.14 0.14 query41 0.06 0.01 0.01 query42 0.02 0.01 0.01 query43 0.02 0.02 0.02 Total cold run time: 104.27 s Total hot run time: 30.51 s ```
doris-robot commented 3 days ago

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit c2c934f6d3d4cbbed543081ca7f3c70428da6a44 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.3 seconds inserted 10000000 Rows, about 469K ops/s