apache / doris

Apache Doris is an easy-to-use, high performance and unified analytics database.
https://doris.apache.org
Apache License 2.0
11.8k stars 3.11k forks source link

[profile](fe)update format of min/hour/sec unit in profile #37010

Closed englefly closed 10 hours ago

englefly commented 3 days ago

Proposed changes

it helps us to find the point in profile which operator consumes long time

Issue Number: close #xxx

doris-robot commented 3 days ago

Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website. See Doris Document.

englefly commented 3 days ago

run buildall

doris-robot commented 3 days ago
TPC-H: Total hot run time: 40705 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 0218ce8c4072350520dba7f676cb7230b026b8e3, data reload: false ------ Round 1 ---------------------------------- q1 19384 4641 4399 4399 q2 2884 197 200 197 q3 11738 1200 1151 1151 q4 10576 843 817 817 q5 7829 2724 2632 2632 q6 219 134 138 134 q7 945 604 605 604 q8 9225 2086 2081 2081 q9 8892 6512 6538 6512 q10 8849 3719 3700 3700 q11 452 242 247 242 q12 406 236 226 226 q13 17778 2979 2957 2957 q14 273 225 228 225 q15 518 478 473 473 q16 518 372 385 372 q17 971 692 629 629 q18 8273 7468 7394 7394 q19 6913 1395 1477 1395 q20 667 327 339 327 q21 4842 3893 3984 3893 q22 396 348 345 345 Total cold run time: 122548 ms Total hot run time: 40705 ms ----- Round 2, with runtime_filter_mode=off ----- q1 4420 4258 4264 4258 q2 375 269 272 269 q3 3029 2712 2684 2684 q4 1905 1606 1571 1571 q5 5291 5308 5301 5301 q6 225 129 130 129 q7 2135 1790 1738 1738 q8 3179 3370 3358 3358 q9 8409 8372 8338 8338 q10 3925 3716 3633 3633 q11 580 484 476 476 q12 771 627 618 618 q13 16257 2983 2982 2982 q14 288 251 275 251 q15 532 482 476 476 q16 478 412 420 412 q17 1777 1477 1433 1433 q18 7712 7453 7364 7364 q19 1939 1670 1601 1601 q20 2015 1762 1776 1762 q21 4899 4819 4672 4672 q22 628 535 524 524 Total cold run time: 70769 ms Total hot run time: 53850 ms ```
doris-robot commented 3 days ago
TPC-DS: Total hot run time: 173204 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 0218ce8c4072350520dba7f676cb7230b026b8e3, data reload: false query1 916 381 370 370 query2 6448 2349 2485 2349 query3 6647 203 205 203 query4 19569 17499 17361 17361 query5 4187 472 458 458 query6 279 162 155 155 query7 4600 307 289 289 query8 305 289 302 289 query9 8602 2457 2443 2443 query10 614 308 281 281 query11 10625 10127 10204 10127 query12 132 89 86 86 query13 1648 376 368 368 query14 10152 7578 6705 6705 query15 263 186 201 186 query16 8181 275 268 268 query17 1884 563 542 542 query18 2108 285 275 275 query19 194 154 154 154 query20 91 83 87 83 query21 218 127 127 127 query22 4345 4016 4029 4016 query23 33887 33188 32822 32822 query24 11021 2903 2848 2848 query25 645 374 376 374 query26 1379 158 152 152 query27 2966 319 317 317 query28 7560 2134 2132 2132 query29 920 625 625 625 query30 278 161 153 153 query31 972 733 742 733 query32 96 54 58 54 query33 795 302 292 292 query34 935 457 476 457 query35 771 617 618 617 query36 1102 937 946 937 query37 156 72 71 71 query38 2889 2799 2835 2799 query39 853 805 783 783 query40 222 128 125 125 query41 57 54 52 52 query42 126 100 100 100 query43 601 585 556 556 query44 1167 743 741 741 query45 205 162 164 162 query46 1087 761 723 723 query47 1869 1822 1802 1802 query48 370 305 300 300 query49 1106 429 435 429 query50 778 391 381 381 query51 6815 6707 6786 6707 query52 107 91 98 91 query53 359 308 288 288 query54 936 467 457 457 query55 75 75 76 75 query56 302 278 284 278 query57 1146 1034 1041 1034 query58 262 267 261 261 query59 3358 3195 3004 3004 query60 306 288 285 285 query61 93 91 92 91 query62 664 450 443 443 query63 316 290 290 290 query64 9013 2265 1762 1762 query65 3211 3130 3109 3109 query66 1155 355 334 334 query67 15862 14963 14866 14866 query68 7074 559 598 559 query69 684 457 423 423 query70 1179 1174 1106 1106 query71 516 266 276 266 query72 9093 5765 5609 5609 query73 796 331 324 324 query74 5969 5557 5622 5557 query75 4906 2634 2696 2634 query76 4832 944 906 906 query77 773 296 301 296 query78 10571 9763 9850 9763 query79 8152 532 529 529 query80 1021 477 481 477 query81 555 217 218 217 query82 762 107 103 103 query83 364 174 170 170 query84 268 86 87 86 query85 1420 297 271 271 query86 454 306 285 285 query87 3319 3121 3147 3121 query88 4787 2395 2372 2372 query89 515 383 386 383 query90 1937 193 194 193 query91 136 103 106 103 query92 65 50 52 50 query93 6712 533 529 529 query94 1215 197 189 189 query95 412 320 319 319 query96 606 272 269 269 query97 3203 3039 3100 3039 query98 216 194 198 194 query99 1253 815 852 815 Total cold run time: 294566 ms Total hot run time: 173204 ms ```
doris-robot commented 3 days ago
ClickBench: Total hot run time: 31.54 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 0218ce8c4072350520dba7f676cb7230b026b8e3, data reload: false query1 0.04 0.03 0.04 query2 0.08 0.03 0.04 query3 0.22 0.04 0.05 query4 1.67 0.08 0.07 query5 0.51 0.50 0.49 query6 1.14 0.73 0.74 query7 0.02 0.02 0.01 query8 0.06 0.04 0.04 query9 0.54 0.47 0.50 query10 0.56 0.54 0.54 query11 0.16 0.12 0.11 query12 0.16 0.13 0.13 query13 0.59 0.60 0.60 query14 0.77 0.81 0.76 query15 0.85 0.81 0.82 query16 0.37 0.37 0.37 query17 1.04 0.98 1.08 query18 0.23 0.24 0.22 query19 1.77 1.70 1.71 query20 0.01 0.01 0.01 query21 15.44 0.74 0.65 query22 3.89 6.54 3.02 query23 18.33 1.44 1.28 query24 2.17 0.23 0.22 query25 0.16 0.09 0.08 query26 0.27 0.18 0.18 query27 0.08 0.09 0.08 query28 13.21 1.01 0.99 query29 12.65 3.31 3.26 query30 0.26 0.06 0.05 query31 2.87 0.38 0.39 query32 3.27 0.47 0.47 query33 2.93 2.86 2.90 query34 17.08 4.41 4.38 query35 4.58 4.47 4.52 query36 0.65 0.48 0.46 query37 0.18 0.15 0.14 query38 0.16 0.15 0.14 query39 0.05 0.03 0.04 query40 0.18 0.14 0.16 query41 0.09 0.05 0.05 query42 0.06 0.05 0.04 query43 0.05 0.04 0.03 Total cold run time: 109.4 s Total hot run time: 31.54 s ```
englefly commented 17 hours ago

run buildall

doris-robot commented 16 hours ago
TPC-H: Total hot run time: 40399 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 90ea6ca20deeb7c5efffb762697a62f0660f9d1d, data reload: false ------ Round 1 ---------------------------------- q1 17640 4308 4286 4286 q2 2019 189 205 189 q3 10448 1131 1106 1106 q4 10196 826 843 826 q5 7504 2641 2590 2590 q6 219 142 142 142 q7 962 610 621 610 q8 9250 2078 2055 2055 q9 8646 6490 6476 6476 q10 8884 3709 3757 3709 q11 441 242 246 242 q12 413 240 242 240 q13 17773 2958 2957 2957 q14 262 219 228 219 q15 534 490 479 479 q16 508 383 387 383 q17 950 658 706 658 q18 7977 7315 7389 7315 q19 6927 1462 1558 1462 q20 675 327 319 319 q21 4841 3797 3890 3797 q22 394 339 346 339 Total cold run time: 117463 ms Total hot run time: 40399 ms ----- Round 2, with runtime_filter_mode=off ----- q1 4445 4226 4282 4226 q2 381 262 270 262 q3 3019 2862 2861 2861 q4 1995 1674 1757 1674 q5 5664 5442 5478 5442 q6 237 132 131 131 q7 2242 1894 1826 1826 q8 3247 3410 3381 3381 q9 8709 8772 8765 8765 q10 4197 3808 3820 3808 q11 635 505 498 498 q12 803 680 659 659 q13 16145 3174 3208 3174 q14 297 264 278 264 q15 534 507 485 485 q16 485 443 443 443 q17 1803 1501 1485 1485 q18 8081 7943 7902 7902 q19 1781 1719 1595 1595 q20 2065 1856 1870 1856 q21 8320 4858 4887 4858 q22 627 547 588 547 Total cold run time: 75712 ms Total hot run time: 56142 ms ```
doris-robot commented 16 hours ago
TPC-DS: Total hot run time: 174940 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 90ea6ca20deeb7c5efffb762697a62f0660f9d1d, data reload: false query1 926 382 384 382 query2 6461 2422 2474 2422 query3 6645 206 214 206 query4 19645 17563 17349 17349 query5 3677 481 477 477 query6 269 156 176 156 query7 4590 291 298 291 query8 312 318 291 291 query9 8573 2413 2389 2389 query10 563 285 300 285 query11 10645 9961 10153 9961 query12 116 84 82 82 query13 1652 365 376 365 query14 9605 7787 7640 7640 query15 251 199 190 190 query16 7956 264 272 264 query17 1897 536 534 534 query18 2067 272 270 270 query19 213 145 159 145 query20 89 80 80 80 query21 218 126 128 126 query22 4395 4353 4221 4221 query23 33837 33804 33668 33668 query24 10628 2874 2931 2874 query25 568 391 372 372 query26 699 154 153 153 query27 2237 342 324 324 query28 5872 2141 2137 2137 query29 900 650 648 648 query30 241 166 157 157 query31 1015 809 793 793 query32 96 52 53 52 query33 677 317 278 278 query34 875 488 493 488 query35 759 632 634 632 query36 1145 977 977 977 query37 139 75 78 75 query38 2965 2922 2833 2833 query39 907 842 855 842 query40 209 124 130 124 query41 58 52 51 51 query42 119 109 109 109 query43 621 538 559 538 query44 1135 724 731 724 query45 188 157 162 157 query46 1078 714 722 714 query47 1832 1761 1783 1761 query48 370 296 288 288 query49 842 408 418 408 query50 768 388 398 388 query51 6938 6914 6778 6778 query52 97 96 90 90 query53 362 299 286 286 query54 870 431 435 431 query55 74 74 75 74 query56 290 271 267 267 query57 1157 1065 1050 1050 query58 236 234 241 234 query59 3357 3277 3302 3277 query60 296 279 280 279 query61 89 86 95 86 query62 589 437 452 437 query63 314 298 290 290 query64 8458 2263 1750 1750 query65 3197 3122 3100 3100 query66 739 323 322 322 query67 15308 14952 14929 14929 query68 4501 537 529 529 query69 474 406 343 343 query70 1151 1157 1161 1157 query71 376 276 280 276 query72 6912 5661 5152 5152 query73 746 325 324 324 query74 5872 5469 5537 5469 query75 3371 2694 2692 2692 query76 2096 929 890 890 query77 435 294 296 294 query78 10344 9744 9869 9744 query79 2576 520 521 520 query80 1597 468 481 468 query81 594 223 219 219 query82 742 111 110 110 query83 278 175 182 175 query84 261 88 94 88 query85 1672 351 270 270 query86 468 315 325 315 query87 3296 3125 3109 3109 query88 4355 2360 2374 2360 query89 477 388 421 388 query90 1672 192 189 189 query91 128 97 97 97 query92 58 46 48 46 query93 1938 504 499 499 query94 1000 186 190 186 query95 404 326 315 315 query96 589 272 270 270 query97 3204 3037 3040 3037 query98 223 198 203 198 query99 1145 853 840 840 Total cold run time: 266871 ms Total hot run time: 174940 ms ```
doris-robot commented 16 hours ago
ClickBench: Total hot run time: 30.67 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 90ea6ca20deeb7c5efffb762697a62f0660f9d1d, data reload: false query1 0.04 0.04 0.03 query2 0.08 0.04 0.05 query3 0.22 0.05 0.06 query4 1.70 0.07 0.08 query5 0.50 0.49 0.48 query6 1.13 0.73 0.72 query7 0.02 0.01 0.01 query8 0.06 0.05 0.04 query9 0.55 0.49 0.48 query10 0.54 0.54 0.52 query11 0.14 0.11 0.12 query12 0.14 0.12 0.13 query13 0.59 0.59 0.60 query14 0.79 0.78 0.78 query15 0.84 0.82 0.82 query16 0.36 0.36 0.37 query17 0.95 0.95 0.98 query18 0.23 0.23 0.25 query19 1.79 1.72 1.71 query20 0.01 0.01 0.01 query21 15.44 0.77 0.66 query22 3.75 7.17 2.12 query23 18.23 1.38 1.26 query24 2.17 0.22 0.21 query25 0.16 0.08 0.09 query26 0.26 0.17 0.18 query27 0.08 0.07 0.07 query28 13.22 1.02 0.99 query29 12.64 3.26 3.29 query30 0.27 0.06 0.06 query31 2.87 0.40 0.38 query32 3.26 0.47 0.48 query33 2.90 2.90 2.94 query34 16.84 4.37 4.41 query35 4.46 4.52 4.47 query36 0.66 0.50 0.51 query37 0.17 0.16 0.15 query38 0.15 0.15 0.14 query39 0.05 0.03 0.04 query40 0.18 0.15 0.16 query41 0.09 0.05 0.04 query42 0.06 0.05 0.04 query43 0.04 0.03 0.04 Total cold run time: 108.63 s Total hot run time: 30.67 s ```
github-actions[bot] commented 11 hours ago

PR approved by anyone and no changes requested.

github-actions[bot] commented 10 hours ago

PR approved by at least one committer and no changes requested.