StarRocks / starrocks

The world's fastest open query engine for sub-second analytics both on and off the data lakehouse. With the flexibility to support nearly any scenario, StarRocks provides best-in-class performance for multi-dimensional analytics, real-time analytics, and ad-hoc queries. A Linux Foundation project.
https://starrocks.io
Apache License 2.0
8.9k stars 1.78k forks source link

The execution plan results do not correctly display the CPU and memory indicator values. #49268

Open DanielYu82 opened 3 months ago

DanielYu82 commented 3 months ago

Steps to reproduce the behavior (Required)

  1. CREATE TABLE
    
    CREATE TABLE `test_ind_order` (
    `create_time` datetime NOT NULL ,                                  
    `order_code` varchar(50) NOT NULL ,                                
    `refund_order_code` varchar(50) NOT NULL ,                         
    `merchant_id` varchar(50) NULL ,                                   
    `merchant_name` varchar(50) NULL ,                                 
    `shop_id` varchar(50) NULL ,                                       
    `shop_name` varchar(50) NULL ,                                     
    `shop_property` int(11) NULL ,                                     
    `shop_property_name` varchar(50) NULL ,                            
    `region_id` varchar(50) NULL ,                                     
    `region_name` varchar(50) NULL ,                                   
    `area_id` varchar(50) NULL ,                                       
    `area_name` varchar(50) NULL ,                                     
    `parent_order_code` varchar(50) NULL ,                             
    `member_id` varchar(50) NULL ,                                     
    `member_type` int(11) NULL ,                                       
    `member_type_name` varchar(50) NULL ,                              
    `applet_mobile` varchar(50) NULL ,                                 
    `member_name` varchar(50) NULL ,                                   
    `member_card` varchar(50) NULL ,                                   
    `member_level` int(11) NULL ,                                      
    `member_level_name` varchar(50) NULL ,                             
    `svip_flag` int(11) NULL ,                                         
    `order_type` int(11) NULL ,                                        
    `order_type_name` varchar(50) NULL ,                               
    `shipped_type` int(11) NULL ,                                      
    `shipped_type_name` varchar(50) NULL ,                             
    `goods_sku` varchar(50) NULL ,                                     
    `goods_name` varchar(50) NULL ,                                    
    `goods_category` varchar(50) NULL ,                                
    `goods_category_name` varchar(50) NULL ,                           
    `pay_type` int(11) NULL ,                                          
    `pay_type_name` varchar(50) NULL ,                                 
    `order_channel` int(11) NULL ,                                     
    `order_channel_name` varchar(50) NULL ,                            
    `refund_type` int(11) NULL ,                                       
    `refund_type_name` varchar(50) NULL ,                              
    `order_status` int(11) NULL ,                                      
    `pay_status` int(11) NULL ,                                        
    `pay_status_name` varchar(50) NULL ,                               
    `shipping_status` int(11) NULL ,                                   
    `refund_status` int(11) NULL ,                                     
    `refund_status_name` varchar(50) NULL ,                            
    `filter_order_status` int(11) NULL ,                               
    `filter_order_status_name` varchar(50) NULL ,                      
    `biz_item_id` varchar(50) NULL ,                                   
    `biz_item_name` varchar(50) NULL ,                                 
    `creator_id` varchar(50) NULL ,                                    
    `creator_name` varchar(50) NULL ,                                  
    `cashier_id` varchar(50) NULL ,                                    
    `cashier_name` varchar(50) NULL ,                                  
    `total_amount` decimal(20, 6) NULL ,                               
    `payable_amount` decimal(20, 6) NULL ,                             
    `discount_amount` decimal(20, 6) NULL ,                            
    `refund_amount` decimal(20, 6) NULL ,                              
    `package_amount` decimal(20, 6) NULL ,                             
    `total_ticket_amount` decimal(20, 6) NULL ,                        
    `payable_ticket_amount` decimal(20, 6) NULL ,                      
    `discount_ticket_amount` decimal(20, 6) NULL ,                     
    `refund_ticket_amount` decimal(20, 6) NULL ,                       
    `total_point_amount` decimal(20, 6) NULL ,                         
    `payable_point_amount` decimal(20, 6) NULL ,                       
    `refund_point_amount` decimal(20, 6) NULL ,                        
    `order_num` decimal(20, 0) NULL ,                                  
    `goods_num` decimal(20, 0) NULL ,                                  
    `goods_payable_amount` decimal(20, 6) NULL ,                       
    `order_gift_coin_num` decimal(20, 0) NULL ,                        
    `order_gift_temp_coin_num` decimal(20, 0) NULL ,                   
    `order_gift_voucher_num` decimal(20, 0) NULL ,                     
    `sale_give_coin_num` decimal(20, 0) NULL ,                         
    `give_coin_num` decimal(20, 0) NULL ,                              
    `sale_give_temp_coin_num` decimal(20, 0) NULL ,                    
    `give_temp_coin_num` decimal(20, 0) NULL ,                         
    `sale_give_coupon_round` decimal(20, 0) NULL ,                     
    `give_coupon_round` decimal(20, 0) NULL ,                          
    `rent_merchant_name` varchar(50) NULL ,                            
    `small_shop_rent_amount` decimal(20, 6) NULL ,                     
    `small_shop_deposit_amount` decimal(20, 6) NULL ,                  
    `machine_rent_amount` decimal(20, 6) NULL ,                        
    `machine_rent_deposit_amount` decimal(20, 6) NULL ,                
    `entry_amount` decimal(20, 6) NULL ,                               
    `create_year` varchar(50) NULL ,                                   
    `create_month` varchar(50) NULL ,                                  
    `create_day` varchar(50) NULL ,                                    
    `create_at` int(11) NULL ,                                         
    `refund_time` datetime NULL ,                                      
    `refund_finish_time` datetime NULL ,                               
    `refund_finish_year` varchar(50) NULL ,                            
    `refund_finish_month` varchar(50) NULL ,                           
    `refund_finish_day` varchar(50) NULL                  
    ) ENGINE=OLAP 
    PRIMARY KEY(`create_time`, `order_code`, `refund_order_code`)
    PARTITION BY date_trunc('month', create_time)
    DISTRIBUTED BY HASH(`order_code`)
    ORDER BY(`order_code`)
    PROPERTIES (
    "compression" = "LZ4",
    "enable_persistent_index" = "true",
    "fast_schema_evolution" = "true",
    "replicated_storage" = "true",
    "replication_num" = "2"
    );

CREATE TABLE test_ind_shop ( stat_date date NOT NULL ,
shop_id varchar(50) NOT NULL ,
merchant_id varchar(50) NULL ,
shop_name varchar(50) NULL ,
province_code int(11) NULL ,
province_name varchar(50) NULL ,
city_code int(11) NULL ,
city_name varchar(50) NULL ,
area_code int(11) NULL ,
area_name varchar(50) NULL ,
shop_level int(11) NULL ,
shop_level_name varchar(50) NULL ,
shop_property int(11) NULL ,
shop_property_name varchar(50) NULL ,
group_wechat_sale_coin_num decimal(20, 0) NULL ,
group_wechat_gift_coin_num decimal(20, 0) NULL ,
group_koubei_sale_coin_num decimal(20, 0) NULL ,
group_koubei_gift_coin_num decimal(20, 0) NULL ,
group_meituan_sale_coin_num decimal(20, 0) NULL ,
group_meituan_gift_coin_num decimal(20, 0) NULL ,
group_tmall_sale_coin_num decimal(20, 0) NULL ,
group_tmall_gift_coin_num decimal(20, 0) NULL ,
group_douyin_sale_coin_num decimal(20, 0) NULL ,
group_douyin_gift_coin_num decimal(20, 0) NULL ,
coin_order_sale_coin_num decimal(20, 0) NULL ,
coin_order_gift_coin_num decimal(20, 0) NULL ,
temp_coin_order_sale_coin_num decimal(20, 0) NULL ,
temp_coin_order_gift_coin_num decimal(20, 0) NULL ,
exchange_sale_coin_num decimal(20, 0) NULL ,
exchange_gift_coin_num decimal(20, 0) NULL ,
mem_mkt_gift_coin_num decimal(20, 0) NULL ,
smp_gift_coin_num decimal(20, 0) NULL ,
coupon_gift_coin_num decimal(20, 0) NULL ,
ecode_gift_coin_num decimal(20, 0) NULL ,
hand_coin_sale_coin_num decimal(20, 0) NULL ,
sale_gift_coin_num decimal(20, 0) NULL ,
begin_deposit_coin_num decimal(20, 0) NULL ,
end_deposit_coin_num decimal(20, 0) NULL ,
machine_receive_coin_num decimal(20, 0) NULL ,
coin_order_amount decimal(20, 6) NULL ,
verification_group_amount decimal(20, 6) NULL ,
per_coin_price decimal(20, 6) NULL ,
create_time datetime NULL ,
create_at int(11) NULL ,
update_at datetime NULL ,
issue_card_gift_coin_num decimal(20, 0) NULL ,
region_name varchar(50) NULL ,
precinct_name varchar(50) NULL ,
exchange_save_lottery_num decimal(20, 0) NULL ,
machine_ele_lottery_num decimal(20, 0) NULL ,
non_ele_lottery_num decimal(20, 0) NULL ,
self_machine_lottery_num decimal(20, 0) NULL ,
activity_gift_lottery_num decimal(20, 0) NULL ,
exchange_redeem_lottery_num decimal(20, 0) NULL ,
jingdong_redeem_lottery_num decimal(20, 0) NULL ,
wechat_redeem_lottery_num decimal(20, 0) NULL ,
invalid_lottery_num decimal(20, 0) NULL ,
expire_lottery_num decimal(20, 0) NULL ,
no_machine_coin_num decimal(20, 0) NULL ,
virtual_machine_coin_num decimal(20, 0) NULL ,
goods_num decimal(20, 0) NULL ,
exch_coin_num decimal(20, 0) NULL ,
total_order_amount decimal(20, 6) NULL ,
total_order_num decimal(20, 0) NULL ,
mem_first_order_num decimal(20, 0) NULL ,
mem_first_order_amount decimal(20, 6) NULL ,
mem_repeat_order_num decimal(20, 0) NULL ,
mem_repeat_order_amount decimal(20, 6) NULL ,
segment0 decimal(20, 0) NULL ,
segment1 decimal(20, 0) NULL ,
segment2 decimal(20, 0) NULL ,
segment3 decimal(20, 0) NULL ,
segment4 decimal(20, 0) NULL ,
segment5 decimal(20, 0) NULL ,
segment6 decimal(20, 0) NULL ,
segment7 decimal(20, 0) NULL ,
mem_mem_num int(11) NULL ,
active_mem_num int(11) NULL ,
mem_order_amount decimal(20, 6) NULL ,
mem_order_num decimal(20, 0) NULL ) ENGINE=OLAP PRIMARY KEY(stat_date, shop_id) DISTRIBUTED BY HASH(shop_id) BUCKETS 100 ORDER BY(shop_id) PROPERTIES ( "colocate_with" = "shop_group", "compression" = "LZ4", "enable_persistent_index" = "true", "fast_schema_evolution" = "true", "replicated_storage" = "true", "replication_num" = "2" );


3. EXPLAIN 
```sql
    explain logical
select
    date_format(t1.create_time, '%y-%m-%d') as stat_date,
    t1.shop_id,
    sum(t1.order_gift_coin_num),
    sum(t1.give_coin_num),
    sum(t1.discount_ticket_amount),
    ifnull(sum(t2.group_meituan_sale_coin_num), 0),
    ifnull(sum(t2.coin_order_amount), 0)
from
    test_ind_order t1
left join (
    select
        t3.stat_date,
        t3.shop_id,
        sum(t3.group_meituan_sale_coin_num) group_meituan_sale_coin_num,
        sum(t3.coin_order_amount) coin_order_amount
    from
        test_ind_shop t3
    where
        t3.stat_date >= '2020-01-01'
        and t3.stat_date < '2020-02-01'
        and t3.shop_id = '9102a162-ee65-48e7-a543-c2d6ed0275e6'
    group by
            t3.stat_date,
        t3.shop_id) t2 on
    t1.shop_id = t2.shop_id
    and t2.stat_date = date_format(t1.create_time, '%y-%m-%d')
where
     t1.create_time >= '2020-01-01'
    and t1.create_time < '2020-02-01'
    and t1.shop_id = '9102a162-ee65-48e7-a543-c2d6ed0275e6'
group by
    date_format(t1.create_time, '%y-%m-%d'),
    t1.shop_id
order by
    stat_date,t1.shop_id desc limit 20

Expected behavior (Required)

The result should contain cpu and memory metric values

Real behavior (Required)

Explain String                                                                                                                                                                                                        |
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
- Output => [173:date_format, 6:shop_id, 174:sum, 175:sum, 176:sum, 179:ifnull, 180:ifnull]                                                                                                                           |
    - TOP-20(FINAL)[173: date_format ASC NULLS FIRST, 6: shop_id DESC NULLS LAST]                                                                                                                                     |
            Estimates: {row: 20, cpu: ?, memory: ?, network: ?, cost: 1.29943575E9}                                                                                                                                   |
        - TOP-20(PARTIAL)[173: date_format ASC NULLS FIRST, 6: shop_id DESC NULLS LAST]                                                                                                                               |
                Estimates: {row: 20, cpu: ?, memory: ?, network: ?, cost: 1.29942551E9}                                                                                                                               |
            - AGGREGATE(GLOBAL) [173:date_format, 6:shop_id] => [179:ifnull, 180:ifnull, 6:shop_id, 173:date_format, 174:sum, 175:sum, 176:sum]                                                                       |
                    Estimates: {row: 854437, cpu: ?, memory: ?, network: ?, cost: 1.05334175E9}                                                                                                                       |
                    177:sum := sum(177:sum)                                                                                                                                                                           |
                    178:sum := sum(178:sum)                                                                                                                                                                           |
                    174:sum := sum(174:sum)                                                                                                                                                                           |
                    175:sum := sum(175:sum)                                                                                                                                                                           |
                    176:sum := sum(176:sum)                                                                                                                                                                           |
                    179:ifnull := ifnull(177:sum, 0)                                                                                                                                                                  |
                    180:ifnull := ifnull(178:sum, 0)                                                                                                                                                                  |
                - EXCHANGE(SHUFFLE) [173, 6]                                                                                                                                                                          |
                        Estimates: {row: 1627500, cpu: ?, memory: ?, network: ?, cost: 7.1612375E8}                                                                                                                   |
                    - AGGREGATE(LOCAL) [173:date_format, 6:shop_id]                                                                                                                                                   |
                            Estimates: {row: 1627500, cpu: ?, memory: ?, network: ?, cost: 6.7966775E8}                                                                                                               |
                            177:sum := sum(171:sum)                                                                                                                                                                   |
                            178:sum := sum(172:sum)                                                                                                                                                                   |
                            174:sum := sum(67:order_gift_coin_num)                                                                                                                                                    |
                            175:sum := sum(71:give_coin_num)                                                                                                                                                          |
                            176:sum := sum(59:discount_ticket_amount)                                                                                                                                                 |
                        - HASH/LEFT OUTER JOIN [6:shop_id = 92:shop_id AND 182:cast = 181:cast] => [67:order_gift_coin_num, 6:shop_id, 71:give_coin_num, 59:discount_ticket_amount, 171:sum, 172:sum, 173:date_format]|
                                Estimates: {row: 3100000, cpu: ?, memory: ?, network: ?, cost: 6.1965175E8}                                                                                                           |
                                173:date_format := date_format(1:create_time, '%y-%m-%d')                                                                                                                             |
                            - SCAN [test_ind_order] => [1:create_time, 67:order_gift_coin_num, 182:cast, 6:shop_id, 71:give_coin_num, 59:discount_ticket_amount]                                                      |
                                    Estimates: {row: 3100000, cpu: ?, memory: ?, network: ?, cost: 1.24E8}                                                                                                            |
                                    partitionRatio: 1/39, tabletRatio: 4/4                                                                                                                                            |
                                    182:cast := cast(date_format(1:create_time, '%y-%m-%d') as datetime)                                                                                                              |
                            - EXCHANGE(BROADCAST)                                                                                                                                                                     |
                                    Estimates: {row: 182375, cpu: ?, memory: ?, network: ?, cost: 9.009325E7}                                                                                                         |
                                - SCAN [test_ind_shop] => [181:cast, 171:sum, 92:shop_id, 172:sum]                                                                                                                    |
                                        Estimates: {row: 182375, cpu: ?, memory: ?, network: ?, cost: 8389250.0}                                                                                                      |
                                        partitionRatio: 1/1, tabletRatio: 100/100                                                                                                                                     |
                                        181:cast := cast(91:stat_date as datetime)                                                                                                                                    |
                                        171:sum := cast(109:group_meituan_sale_coin_num as decimal(38, 0))                                                                                                            |
                                        172:sum := cast(130:coin_order_amount as decimal(38, 6))                                                                                                                      |
                                        predicate: 91:stat_date >= 2020-01-01 AND 91:stat_date < 2020-02-01                                                                                                           |

StarRocks version (Required)

DanielYu82 commented 3 months ago

test_ind order 100 million records test_ind shop 800,000 records