StarRocks / starrocks

StarRocks, a Linux Foundation project, is a next-generation sub-second MPP OLAP database for full analytics scenarios, including multi-dimensional analytics, real-time analytics, and ad-hoc queries.
https://starrocks.io
Apache License 2.0
8.74k stars 1.75k forks source link

Importing data using trim_space and escape parameters results is incorrect #42945

Open huoyue-git opened 6 months ago

huoyue-git commented 6 months ago

Steps to reproduce the behavior (Required)

  1. CREATE TABLE '...'
    
    CREATE TABLE `duplicate_table_with_null4` (
    `k0` varchar(65533) NULL COMMENT "",
    `v1` varchar(65533) NULL COMMENT "",
    `v2` varchar(65533) NULL COMMENT "",
    `v3` varchar(65533) NULL COMMENT "",
    `v4` varchar(65533) NULL COMMENT "",
    `v5` varchar(65533) NULL COMMENT ""
    ) ENGINE=OLAP
    DUPLICATE KEY(`k0`)
    COMMENT "OLAP"
    DISTRIBUTED BY HASH(`k0`) BUCKETS 3
    PROPERTIES (
    "replication_num" = "3",
    "in_memory" = "false",
    "enable_persistent_index" = "false",
    "replicated_storage" = "true",
    "fast_schema_evolution" = "true",
    "compression" = "LZ4"
    );

data

1 , 9999-12-31 , 9999-12-31 23:59:59 , "beijintgaergertte" , "haindian'sdvgerwwge" , "asrew er:""hgeth " 2 , 0001-01-01 , 0001-01-01 00:00:01 , " beij ing " , " haidtnr&_#@#$)!_ian " , " saefwd "" " 3 , 2020-01-01 , 2020-01-01 00:00:01 , "beij ting" , "haidi""tnr&_#@#$)!_ian" , "saefwd "" "

curl --location-trusted -u "root:" -T ../para_trim_space_enclose.csv -XPUT -H "label:stream_load_1710837374414_jhx" -H "column_separator:," -H "trim_space:true" -H 'enclose:"' http://xxx.xx.xx.xxx:xxxx/api/test_db/duplicate_table_with_null4/_stream_load

select *, length(k0), length(v1), length(v2), length(v3), length(v4), length(v5) from duplicate_table_with_null4;



### Expected behavior (Required)

### Real behavior (Required)
![img_v3_0294_17457b17-0a0e-4311-8f74-d035e07be56g](https://github.com/StarRocks/starrocks/assets/91584973/4ca1e547-4756-422b-9cd1-6fbf9bb41d62)

### StarRocks version (Required)
 - You can get the StarRocks version by executing SQL `select current_version()`
github-actions[bot] commented 1 week ago

We have marked this issue as stale because it has been inactive for 6 months. If this issue is still relevant, removing the stale label or adding a comment will keep it active. Otherwise, we'll close it in 10 days to keep the issue queue tidy. Thank you for your contribution to StarRocks!