StarRocks / starrocks

The world's fastest open query engine for sub-second analytics both on and off the data lakehouse. With the flexibility to support nearly any scenario, StarRocks provides best-in-class performance for multi-dimensional analytics, real-time analytics, and ad-hoc queries. A Linux Foundation project.
https://starrocks.io
Apache License 2.0
9.03k stars 1.82k forks source link

Short key index not utilized as expected #48422

Closed taiyang-li closed 3 months ago

taiyang-li commented 4 months ago

Steps to reproduce the behavior (Required)

  1. create olap table
    CREATE TABLE `mytable` (
    `reportTime` bigint(20) NULL COMMENT "",
    `appId` bigint(20) NULL COMMENT "",
    `day` date NULL AS str2date(from_unixtime(reportTime), '%Y-%m-%d %H:%i:%s') COMMENT "",
    `rtime` datetime NULL AS str_to_date(from_unixtime(reportTime), '%Y-%m-%d %H:%i:%s') COMMENT ""
    ) ENGINE=OLAP 
    DUPLICATE KEY(`reportTime`, `appId`)
    PARTITION BY date_trunc('day', day)
    DISTRIBUTED BY HASH(`reportTime`, `appId`)
    PROPERTIES (
    "compression" = "LZ4",
    "partition_live_number" = "3",
    "replicated_storage" = "true",
    "replication_num" = "2"
    );
  2. insert some random data
  3. set enable_profile = true;
  4. SELECT COUNT() FROM default.mytable WHERE day='2024-07-10' and reportTime >= 1720583999;
  5. login fe webui, refer to corresponding profile of current query.
                 - RemainingRowsAfterShortKeyFilter: 376.326M (376325576)
                   - __MAX_OF_RemainingRowsAfterShortKeyFilter: 2.056M (2056416)
                   - __MIN_OF_RemainingRowsAfterShortKeyFilter: 1.049M (1048576)
                 - SegmentRuntimeZoneMapFilterRows: 0
                 - SegmentZoneMapFilterRows: 0
                 - ShortKeyFilter: 1.143us
                   - __MAX_OF_ShortKeyFilter: 9.063us
                   - __MIN_OF_ShortKeyFilter: 354ns
                 - ShortKeyFilterRows: 0
                 - ShortKeyRangeNumber: 0
                 - ZoneMapIndexFilterRows: 0
                 - ZoneMapIndexFiter: 3.258ms
                   - __MAX_OF_ZoneMapIndexFiter: 118.163ms
                   - __MIN_OF_ZoneMapIndexFiter: 183.467us

Expected behavior (Required)

ShortKeyFilterRows is not zero, and short key index is utilized

Real behavior (Required)

ShortKeyFilterRows is zero, and short key index is not utilized

StarRocks version (Required)

StarRocks Version: 3.3.0-19a3f66 (it was also found in 3.2.0)

taiyang-li commented 3 months ago

No one replied, let's close it.