search

apache / doris

Apache Doris is an easy-to-use, high performance and unified analytics database.
https://doris.apache.org
Apache License 2.0
12.38k stars 3.22k forks source link

[fix](nereids) derive column stats for 'expr and A is not null' #37235 #37236

Closed englefly closed 2 months ago

englefly commented 3 months ago

Proposed changes

pick #37235 Issue Number: close #xxx

doris-robot commented 3 months ago

Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website. See Doris Document.

englefly commented 3 months ago

run buildall

doris-robot commented 3 months ago
TPC-H: Total hot run time: 49851 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 6d67e2e6a9668e336dcc231852e65ccc7a9eede2, data reload: false ------ Round 1 ---------------------------------- q1 17734 4384 4360 4360 q2 2037 156 144 144 q3 10437 1922 1933 1922 q4 10343 1223 1319 1223 q5 8929 3909 3920 3909 q6 231 126 129 126 q7 2046 1624 1577 1577 q8 9287 2719 2701 2701 q9 10600 10207 10200 10200 q10 8607 3487 3479 3479 q11 425 247 253 247 q12 461 313 308 308 q13 18366 3960 4013 3960 q14 352 323 329 323 q15 507 462 458 458 q16 669 567 562 562 q17 1138 966 966 966 q18 7248 6896 6932 6896 q19 1783 1665 1664 1664 q20 541 320 293 293 q21 4422 4113 4099 4099 q22 514 434 438 434 Total cold run time: 116677 ms Total hot run time: 49851 ms ----- Round 2, with runtime_filter_mode=off ----- q1 4338 4492 4309 4309 q2 321 227 221 221 q3 4200 4138 4143 4138 q4 2768 2745 2741 2741 q5 7171 7097 7084 7084 q6 235 118 118 118 q7 3253 2834 2886 2834 q8 4369 4434 4505 4434 q9 16844 16749 16814 16749 q10 4240 4193 4322 4193 q11 725 706 697 697 q12 1046 856 843 843 q13 7064 3761 3727 3727 q14 440 430 420 420 q15 506 470 454 454 q16 754 684 675 675 q17 3822 3904 3919 3904 q18 8752 8796 8704 8704 q19 1732 1713 1670 1670 q20 2409 2145 2121 2121 q21 8537 8420 8425 8420 q22 1040 981 961 961 Total cold run time: 84566 ms Total hot run time: 79417 ms ```
doris-robot commented 3 months ago
TPC-DS: Total hot run time: 203534 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 6d67e2e6a9668e336dcc231852e65ccc7a9eede2, data reload: false query1 940 427 375 375 query2 6534 2651 2776 2651 query3 6921 209 203 203 query4 20833 18123 18071 18071 query5 19742 6538 6519 6519 query6 298 223 237 223 query7 4153 306 299 299 query8 422 406 448 406 query9 3117 2658 2619 2619 query10 422 309 294 294 query11 11418 10846 10779 10779 query12 119 81 76 76 query13 5618 704 692 692 query14 18179 13482 13249 13249 query15 364 231 254 231 query16 6457 299 264 264 query17 1691 1460 876 876 query18 2304 418 428 418 query19 215 150 156 150 query20 81 80 78 78 query21 192 94 94 94 query22 5395 4955 5012 4955 query23 32355 31968 31971 31968 query24 6921 6510 6582 6510 query25 527 431 442 431 query26 527 170 160 160 query27 1896 293 295 293 query28 6140 2379 2327 2327 query29 2916 2771 2868 2771 query30 248 166 166 166 query31 920 746 729 729 query32 74 62 61 61 query33 389 241 254 241 query34 853 469 480 469 query35 1112 928 946 928 query36 1293 1155 1202 1155 query37 86 61 61 61 query38 3133 2921 2936 2921 query39 1373 1349 1327 1327 query40 201 94 94 94 query41 45 45 47 45 query42 80 83 91 83 query43 760 644 662 644 query44 1156 723 721 721 query45 252 240 241 240 query46 1237 950 957 950 query47 2036 1738 1685 1685 query48 1010 731 721 721 query49 622 363 377 363 query50 851 630 594 594 query51 4864 4734 4620 4620 query52 90 87 78 78 query53 444 334 318 318 query54 2673 2463 2458 2458 query55 96 85 86 85 query56 246 218 218 218 query57 1232 1209 1100 1100 query58 219 193 192 192 query59 3996 4313 4015 4015 query60 208 211 203 203 query61 98 91 95 91 query62 704 468 480 468 query63 488 343 346 343 query64 2419 1541 1523 1523 query65 3613 3555 3599 3555 query66 801 372 368 368 query67 16201 16049 14961 14961 query68 9539 638 641 638 query69 582 362 352 352 query70 1698 1407 1537 1407 query71 428 301 323 301 query72 6591 3484 3502 3484 query73 730 319 325 319 query74 6356 5922 5880 5880 query75 5463 3716 3668 3668 query76 5956 1153 1196 1153 query77 1019 262 253 253 query78 12657 11684 11832 11684 query79 6668 630 646 630 query80 982 401 401 401 query81 494 234 235 234 query82 1304 103 100 100 query83 178 138 139 138 query84 260 75 70 70 query85 884 335 334 334 query86 326 291 300 291 query87 3253 3037 2994 2994 query88 4719 2312 2279 2279 query89 356 281 299 281 query90 1957 210 216 210 query91 171 137 149 137 query92 60 55 59 55 query93 4436 603 587 587 query94 701 210 213 210 query95 1111 1072 1049 1049 query96 642 326 331 326 query97 6445 6301 6336 6301 query98 194 179 166 166 query99 2972 861 826 826 Total cold run time: 311952 ms Total hot run time: 203534 ms ```
doris-robot commented 3 months ago
ClickBench: Total hot run time: 31.33 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 6d67e2e6a9668e336dcc231852e65ccc7a9eede2, data reload: false query1 0.02 0.02 0.02 query2 0.07 0.02 0.02 query3 0.24 0.06 0.05 query4 1.77 0.07 0.08 query5 0.53 0.52 0.52 query6 1.26 0.63 0.61 query7 0.02 0.01 0.01 query8 0.03 0.03 0.02 query9 0.53 0.49 0.48 query10 0.56 0.53 0.55 query11 0.12 0.09 0.09 query12 0.12 0.09 0.09 query13 0.62 0.61 0.61 query14 0.77 0.81 0.78 query15 0.78 0.75 0.78 query16 0.36 0.38 0.36 query17 1.01 0.98 1.00 query18 0.22 0.23 0.27 query19 1.93 1.86 1.86 query20 0.02 0.01 0.01 query21 15.46 0.55 0.54 query22 1.67 2.29 1.96 query23 17.09 0.98 0.91 query24 3.73 1.22 1.52 query25 0.36 0.09 0.05 query26 0.53 0.16 0.15 query27 0.04 0.04 0.05 query28 8.87 0.72 0.73 query29 12.66 2.33 2.38 query30 0.54 0.54 0.52 query31 2.81 0.38 0.37 query32 3.40 0.50 0.49 query33 3.08 3.07 3.07 query34 15.23 4.77 4.80 query35 4.86 4.84 4.84 query36 1.04 1.01 1.02 query37 0.06 0.04 0.04 query38 0.03 0.02 0.02 query39 0.02 0.01 0.02 query40 0.16 0.15 0.14 query41 0.07 0.01 0.02 query42 0.02 0.02 0.01 query43 0.02 0.02 0.02 Total cold run time: 102.73 s Total hot run time: 31.33 s ```
doris-robot commented 3 months ago

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 6d67e2e6a9668e336dcc231852e65ccc7a9eede2 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.2 seconds inserted 10000000 Rows, about 471K ops/s