apache / doris

Apache Doris is an easy-to-use, high performance and unified analytics database.
https://doris.apache.org
Apache License 2.0
12.7k stars 3.28k forks source link

[Bug] [streamLoader] There are null values ​​in the imported data #38173

Open liugddx opened 3 months ago

liugddx commented 3 months ago

Search before asking

Version

2.0.3

What's Wrong?

There are null values ​​in the imported data

What You Expected?

-

How to Reproduce?

create database e2e_sink;

CREATE TABLE doris_e2e_unique_table ( F_ID bigint(20) NULL, F_INT int(11) NULL, F_BIGINT bigint(20) NULL, F_TINYINT tinyint(4) NULL, F_SMALLINT smallint(6) NULL, F_DECIMAL DECIMAL(18, 6) NULL, F_LARGEINT largeint(40) NULL, F_BOOLEAN boolean NULL, F_DOUBLE double NULL, F_FLOAT float NULL, F_CHAR char(1) NULL, F_VARCHAR_11 varchar(11) NULL, F_STRING text NULL, F_DATETIME_P datetime(6) NULL, F_DATETIME datetime NULL, F_DATE date NULL, MAP_VARCHAR_BOOLEAN MAP<text,boolean> NULL, MAP_CHAR_TINYINT MAP<text,tinyint(4)> NULL, MAP_STRING_SMALLINT MAP<text,smallint(6)> NULL, MAP_INT_INT MAP<int(11),int(11)> NULL, MAP_TINYINT_BIGINT MAP<tinyint(4),bigint(20)> NULL, MAP_SMALLINT_LARGEINT MAP<smallint(6),decimalv3(20, 0)> NULL, MAP_BIGINT_FLOAT MAP<bigint(20),float> NULL, MAP_LARGEINT_DOUBLE MAP<decimalv3(20, 0),double> NULL, MAP_STRING_DECIMAL MAP<text,decimalv3(10, 2)> NULL, MAP_DECIMAL_DATE MAP<decimalv3(10, 2),datev2> NULL, MAP_DATE_DATETIME MAP<datev2,datetimev2(6)> NULL, MAP_DATETIME_CHAR MAP<datetimev2(6),text> NULL, MAP_CHAR_VARCHAR MAP<text,text> NULL, MAP_VARCHAR_STRING MAP<text,text> NULL ) ENGINE=OLAP UNIQUE KEY(F_ID) COMMENT 'OLAP' DISTRIBUTED BY HASH(F_ID) BUCKETS 10 PROPERTIES ( "replication_allocation" = "tag.location.default: 1", "is_being_synced" = "false", "storage_format" = "V2", "light_schema_change" = "true", "disable_auto_compaction" = "false", "enable_single_replica_compaction" = "false" );

- data
[data.json](https://github.com/user-attachments/files/16323004/data.json)
- shell

curl --location-trusted -u root: \ -H "Expect:100-continue" \ -H "format:json" \ -H "strip_outer_array:true" \ -T data.json \ -XPUT http://localhost:8040/api/e2e_sink/doris_e2e_unique_table/_stream_load



### Anything Else?

-

### Are you willing to submit PR?

- [x] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
liugddx commented 3 months ago

If largeint value is scientific notation, A null value will be inserted

Hisoka-X commented 3 months ago

cc @CalvinKirs