search

apache / doris

Apache Doris is an easy-to-use, high performance and unified analytics database.
https://doris.apache.org
Apache License 2.0
12.77k stars 3.28k forks source link

[enhance](catalog)External partition prune return partitionName instead of partitionId #44415

Open zddr opened 13 hours ago

zddr commented 13 hours ago

What problem does this PR solve?

The partition ID of external data sources is meaningless, and some data sources only have partition names, so the return result of partition pruning is replaced with name instead of ID Issue Number: close #xxx

Related PR: #xxx

Problem Summary: External partition prune return partitionName instead of partitionId

Release note

External partition prune return partitionName instead of partitionId

Check List (For Author)

Check List (For Reviewer who merge this PR)

doris-robot commented 13 hours ago

Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?
zddr commented 13 hours ago

run buildall

doris-robot commented 12 hours ago
TPC-H: Total hot run time: 40189 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 5e4e33e50dcbfa259ed7480bd84b5639ce5f1e75, data reload: false ------ Round 1 ---------------------------------- q1 17580 7549 7349 7349 q2 2050 175 186 175 q3 10581 1101 1221 1101 q4 10525 777 770 770 q5 7631 2777 2788 2777 q6 246 152 152 152 q7 978 629 622 622 q8 9234 1877 1907 1877 q9 6682 6489 6453 6453 q10 7001 2310 2356 2310 q11 462 259 258 258 q12 430 218 211 211 q13 17790 3021 3046 3021 q14 253 208 213 208 q15 576 539 526 526 q16 656 579 593 579 q17 984 547 583 547 q18 7308 6681 6707 6681 q19 1341 1025 1069 1025 q20 465 180 182 180 q21 3988 3055 3250 3055 q22 382 318 312 312 Total cold run time: 107143 ms Total hot run time: 40189 ms ----- Round 2, with runtime_filter_mode=off ----- q1 7349 7345 7354 7345 q2 333 230 229 229 q3 2915 2836 2977 2836 q4 2095 1826 1910 1826 q5 5696 5724 5646 5646 q6 233 148 144 144 q7 2289 1822 1814 1814 q8 3468 3529 3531 3529 q9 8850 8922 8932 8922 q10 3609 3558 3562 3558 q11 606 502 493 493 q12 840 592 635 592 q13 11135 3248 3186 3186 q14 310 270 279 270 q15 573 513 516 513 q16 692 646 661 646 q17 1879 1644 1621 1621 q18 8286 7810 7793 7793 q19 1688 1653 1648 1648 q20 2131 1895 1877 1877 q21 5560 5516 5274 5274 q22 636 568 560 560 Total cold run time: 71173 ms Total hot run time: 60322 ms ```
doris-robot commented 12 hours ago
TPC-DS: Total hot run time: 198249 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 5e4e33e50dcbfa259ed7480bd84b5639ce5f1e75, data reload: false query1 1254 931 959 931 query2 6295 2085 2091 2085 query3 10790 3974 3982 3974 query4 67778 28780 23606 23606 query5 4895 467 465 465 query6 401 187 179 179 query7 5552 293 287 287 query8 316 233 228 228 query9 8754 2685 2678 2678 query10 470 284 251 251 query11 17270 15121 15999 15121 query12 159 101 103 101 query13 1463 436 430 430 query14 10326 7715 7697 7697 query15 214 182 187 182 query16 7152 476 489 476 query17 1227 561 560 560 query18 1833 303 310 303 query19 212 154 154 154 query20 120 121 112 112 query21 207 111 104 104 query22 4814 4697 4755 4697 query23 34988 34545 34548 34545 query24 5483 2510 2473 2473 query25 493 406 385 385 query26 666 148 153 148 query27 2027 285 282 282 query28 4194 2478 2454 2454 query29 708 447 420 420 query30 217 153 160 153 query31 1006 840 877 840 query32 67 58 54 54 query33 412 292 293 292 query34 967 532 535 532 query35 854 767 765 765 query36 1105 955 956 955 query37 128 75 75 75 query38 4463 4398 4412 4398 query39 1613 1502 1492 1492 query40 203 97 131 97 query41 44 43 45 43 query42 110 99 100 99 query43 528 515 505 505 query44 1221 866 850 850 query45 178 166 168 166 query46 1161 726 730 726 query47 2065 1951 1932 1932 query48 458 321 322 321 query49 743 408 425 408 query50 841 403 406 403 query51 7397 7238 7095 7095 query52 96 87 93 87 query53 264 181 183 181 query54 536 399 385 385 query55 80 114 76 76 query56 262 228 237 228 query57 1318 1181 1182 1181 query58 225 214 212 212 query59 3267 3012 2982 2982 query60 270 255 244 244 query61 107 106 103 103 query62 825 673 680 673 query63 211 195 186 186 query64 1416 686 625 625 query65 3297 3205 3266 3205 query66 708 310 308 308 query67 16074 15957 15817 15817 query68 3719 589 576 576 query69 438 253 264 253 query70 1201 1100 1162 1100 query71 360 253 258 253 query72 6404 4100 4138 4100 query73 773 365 365 365 query74 10240 9072 9079 9072 query75 3409 2721 2681 2681 query76 1875 1034 1173 1034 query77 470 272 273 272 query78 10495 9482 9430 9430 query79 2158 607 604 604 query80 1373 428 430 428 query81 523 235 232 232 query82 1273 117 121 117 query83 261 152 147 147 query84 278 74 70 70 query85 1037 354 297 297 query86 432 296 291 291 query87 4816 4574 4612 4574 query88 3667 2261 2202 2202 query89 417 301 300 300 query90 1961 196 191 191 query91 140 102 100 100 query92 67 52 47 47 query93 2904 535 543 535 query94 850 314 296 296 query95 354 253 253 253 query96 630 279 279 279 query97 2868 2646 2700 2646 query98 220 199 197 197 query99 1650 1285 1290 1285 Total cold run time: 322061 ms Total hot run time: 198249 ms ```
doris-robot commented 12 hours ago
ClickBench: Total hot run time: 32.16 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 5e4e33e50dcbfa259ed7480bd84b5639ce5f1e75, data reload: false query1 0.04 0.04 0.03 query2 0.07 0.03 0.03 query3 0.25 0.07 0.07 query4 1.62 0.10 0.11 query5 0.43 0.42 0.41 query6 1.14 0.67 0.64 query7 0.02 0.02 0.02 query8 0.04 0.03 0.04 query9 0.58 0.51 0.51 query10 0.56 0.58 0.56 query11 0.14 0.10 0.10 query12 0.14 0.12 0.11 query13 0.60 0.60 0.59 query14 2.71 2.72 2.75 query15 0.89 0.81 0.82 query16 0.39 0.39 0.38 query17 1.01 1.06 1.00 query18 0.23 0.21 0.21 query19 1.97 1.81 1.91 query20 0.02 0.01 0.01 query21 15.37 0.58 0.58 query22 2.47 2.77 1.70 query23 17.23 0.94 0.77 query24 2.95 0.84 0.75 query25 0.15 0.11 0.21 query26 0.36 0.14 0.14 query27 0.04 0.04 0.04 query28 11.09 1.11 1.08 query29 12.53 3.32 3.34 query30 0.24 0.06 0.06 query31 2.87 0.38 0.38 query32 3.26 0.45 0.46 query33 3.10 3.04 3.05 query34 17.13 4.40 4.41 query35 4.48 4.50 4.43 query36 0.68 0.50 0.47 query37 0.09 0.07 0.06 query38 0.04 0.04 0.03 query39 0.04 0.02 0.03 query40 0.16 0.13 0.12 query41 0.07 0.02 0.02 query42 0.03 0.02 0.02 query43 0.04 0.03 0.03 Total cold run time: 107.27 s Total hot run time: 32.16 s ```
zddr commented 11 hours ago

run p0

zddr commented 11 hours ago

run cloud_p0