StarRocks / starrocks

StarRocks, a Linux Foundation project, is a next-generation sub-second MPP OLAP database for full analytics scenarios, including multi-dimensional analytics, real-time analytics, and ad-hoc queries.
https://starrocks.io
Apache License 2.0
8.65k stars 1.75k forks source link

Timeout when load `orc` file with broker from hdfs. #5683

Closed yongbingwang closed 2 years ago

yongbingwang commented 2 years ago

Steps to reproduce the behavior (Required)

  1. 'CREATE DATABASE'
    create database test_broker_load_50d95a02_c787_11ec_97ad_00163e0e489a;
    use test_broker_load_50d95a02_c787_11ec_97ad_00163e0e489a;
  2. CREATE TABLE '...'
    CREATE TABLE `duplicate_table_with_null` ( `k1`  date, `k2`  datetime, `k3`  char(20), `k4`  varchar(20), `k5`  boolean, `k6`  tinyint, `k7`  smallint, `k8`  int, `k9`  bigint, `k10` largeint, `k11` float, `k12` double, `k13` decimal(27,9) ) ENGINE=OLAP DUPLICATE KEY(`k1`, `k2`, `k3`, `k4`, `k5`) COMMENT "OLAP" DISTRIBUTED BY HASH(`k1`, `k2`, `k3`) BUCKETS 3 PROPERTIES ( "replication_num" = "1", "storage_format" = "v2" );
    admin set frontend config("vectorized_load_enable" = "true");
  3. Load file '....'
    LOAD LABEL test_broker_load_50d95a02_c787_11ec_97ad_00163e0e489a.duplicate_table_with_null (DATA INFILE( "hdfs://172.26.92.141:9002/user/nose-ut-data/data.orc") INTO TABLE `duplicate_table_with_null` FORMAT AS "orc" (k1, k2, k3, k4, k5, k6, k7, k8, k9, k10, k11, k12, k13) ,DATA INFILE( "hdfs://172.26.92.141:9002/user/nose-ut-data/data.parquet") INTO TABLE `duplicate_table_with_null` FORMAT AS "parquet" (k1, k2, k3, k4, k5, k6, k7, k8, k9, k10, k11, k12, k13) ,DATA INFILE( "hdfs://xxx:9002/user/nose-ut-data/data") INTO TABLE `duplicate_table_with_null` FORMAT AS "csv" (k1, k2, k3, k4, k5, k6, k7, k8, k9, k10, k11, k12, k13) )  WITH BROKER hdfs_broker ("username"="sr","password"="xxx") PROPERTIES("timeout"="3600");

    Expected behavior (Required)

    run load task correctly.

    Real behavior (Required)

    timeout.

    StarRocks version (Required)

    • You can get the StarRocks version by executing SQL select current_version()
    • MAIN-RELEASE 9ad652c
yongbingwang commented 2 years ago

Because of the wrong parameter setting in that cluster.