search

StarRocks / starrocks

The world's fastest open query engine for sub-second analytics both on and off the data lakehouse. With the flexibility to support nearly any scenario, StarRocks provides best-in-class performance for multi-dimensional analytics, real-time analytics, and ad-hoc queries. A Linux Foundation project.
https://starrocks.io
Apache License 2.0
8.87k stars 1.78k forks source link

[BugFix] Dynamic partition tables that are partitioned by hours need to set the partition field type to datetime #44564

Open blanklin030 opened 6 months ago

blanklin030 commented 6 months ago

Steps to reproduce the behavior (Required)

CREATE TABLE `db`.`test_111` (
  `project_id` bigint(20) NOT NULL DEFAULT "0" COMMENT "",
  `dt` date NOT NULL DEFAULT "1970-01-01" COMMENT "",
  `project_code` varchar(65533) NOT NULL DEFAULT "" COMMENT "",
  `project_name` varchar(65533) NOT NULL DEFAULT "" COMMENT "",
  `owners` varchar(65533) NOT NULL DEFAULT "" COMMENT ""
) ENGINE=OLAP
PRIMARY KEY(`project_id`, `dt`)
COMMENT "xxx"
PARTITION BY RANGE(`dt`)
(PARTITION p20240324 VALUES [("2024-03-24"), ("2024-03-25")),
PARTITION p20240325 VALUES [("2024-03-25"), ("2024-03-26")),
PARTITION p20240326 VALUES [("2024-03-26"), ("2024-03-27")),
PARTITION p20240327 VALUES [("2024-03-27"), ("2024-03-28")),
PARTITION p20240328 VALUES [("2024-03-28"), ("2024-03-29")),
PARTITION p20240329 VALUES [("2024-03-29"), ("2024-03-30")),
PARTITION p20240330 VALUES [("2024-03-30"), ("2024-03-31")),
PARTITION p20240331 VALUES [("2024-03-31"), ("2024-04-01")),
PARTITION p20240401 VALUES [("2024-04-01"), ("2024-04-02")))
DISTRIBUTED BY HASH(`project_id`) BUCKETS 5
PROPERTIES (
"replication_num" = "1",
"dynamic_partition.enable" = "true",
"dynamic_partition.time_unit" = "HOUR",
"dynamic_partition.time_zone" = "Asia/Shanghai",
"dynamic_partition.start" = "-360",
"dynamic_partition.end" = "1",
"dynamic_partition.prefix" = "p",
"dynamic_partition.buckets" = "100",
"dynamic_partition.history_partition_num" = "0",
"in_memory" = "false",
"storage_format" = "DEFAULT",
"enable_persistent_index" = "true",
"compression" = "LZ4"
);

Expected behavior (Required)

create table fail, and give some tips like "if the table's time unit be hour, then the partition column type need to be DateTime"

Real behavior (Required)

create table success, but DynamicPartitionScheduler fail

2024-04-22 23:32:46,058 ERROR (DynamicPartitionScheduler|51) [Daemon.run():117] daemon thread got exception. name: DynamicPartitionScheduler
java.lang.StringIndexOutOfBoundsException: String index out of range: 10
        at java.lang.String.substring(String.java:1963) ~[?:1.8.0_77]
        at com.starrocks.common.util.DynamicPartitionUtil.getFormattedPartitionName(DynamicPartitionUtil.java:447) ~[starrocks-fe.jar:?]
        at com.starrocks.clone.DynamicPartitionScheduler.getAddPartitionClause(DynamicPartitionScheduler.java:234) ~[starrocks-fe.jar:?]
        at com.starrocks.clone.DynamicPartitionScheduler.executeDynamicPartition(DynamicPartitionScheduler.java:380) ~[starrocks-fe.jar:?]
        at com.starrocks.clone.DynamicPartitionScheduler.runAfterCatalogReady(DynamicPartitionScheduler.java:597) ~[starrocks-fe.jar:?]
        at com.starrocks.common.util.LeaderDaemon.runOneCycle(LeaderDaemon.java:60) ~[starrocks-fe.jar:?]
        at com.starrocks.common.util.Daemon.run(Daemon.java:115) ~[starrocks-fe.jar:?]

StarRocks version (Required)

github-actions[bot] commented 4 days ago

We have marked this issue as stale because it has been inactive for 6 months. If this issue is still relevant, removing the stale label or adding a comment will keep it active. Otherwise, we'll close it in 10 days to keep the issue queue tidy. Thank you for your contribution to StarRocks!