apache / doris

Apache Doris is an easy-to-use, high performance and unified analytics database.
https://doris.apache.org
Apache License 2.0
12.64k stars 3.26k forks source link

[Bug] decimal类型超出精度结果不对,streamload导入结果不对 #39864

Open chenlinzhong opened 2 months ago

chenlinzhong commented 2 months ago

Search before asking

Version

create database  if not exists test_db;
CREATE TABLE  test_db.test_tbl (
k1 decimal(1,0) NULL,
v1 decimal(1,0) NULL
) ENGINE=OLAP
DUPLICATE KEY(k1)
DISTRIBUTED BY HASH(k1) BUCKETS 15
PROPERTIES (
"replication_allocation" = "tag.location.default: 1"
);
#数据 data.csv
1,1
28,2,
38,3
400,4
5000,5
6000,65
#streamload导入
curl --location-trusted -u ${user}:${password} -H "strict_mode: true" -T data.csv -H "column_separator:,"  http://{host}:{port}/api/test_db/test_tbl/_stream_load

What's Wrong?

2.0+ after load the result is wrong

# 2.1.5-rc02 版本,结果不符合预期,应该是要报错的,目前看插入的时候是按照M的长度从左到右截取的
MySQL [(none)]> select * from test_db.test_tbl;
+------+------+
| k1   | v1   |
+------+------+
|    2 |    2 |
|    4 |    4 |
|    5 |    5 |
|    1 |    1 |
|    3 |    3 |
+------+------+

如果用 insert into 导入是符合预期的,直接报错
MySQL [test_db]> insert into test_tbl values(1,1),(28,2),(38,3),(400,4),(5000,5);
ERROR 1105 (HY000): errCode = 2, detailMessage = (10.81.85.75)[E-255]Arithmetic overflow when converting value 28 from type Int8 to type Decimal(1, 0)

# 1.1.5 版本  符合预期
MySQL [(none)]> select * from test_db.test_tbl;
+------+------+
| k1   | v1   |
+------+------+
|    1 |    1 |
+------+------+

Reason: decimal value is not valid for definition, column=k1, value=28, precision=1, scale=0; . src line []; 
Reason: decimal value is not valid for definition, column=k1, value=38, precision=1, scale=0; . src line []; 
Reason: decimal value is not valid for definition, column=k1, value=400, precision=1, scale=0; . src line []; 
Reason: decimal value is not valid for definition, column=k1, value=5000, precision=1, scale=0; . src line []; 
Reason: decimal value is not valid for definition, column=k1, value=6000, precision=1, scale=0; . src line [];

#1.2.8版本  符合预期
MySQL [(none)]> select * from test_db.test_tbl;
+------+------+
| k1   | v1   |
+------+------+
|    1 |    1 |
+------+------+
Reason: decimal value is not valid for definition, column=k1, value=28, precision=1, scale=0; . src line []; 
Reason: decimal value is not valid for definition, column=k1, value=38, precision=1, scale=0; . src line []; 
Reason: decimal value is not valid for definition, column=k1, value=400, precision=1, scale=0; . src line []; 
Reason: decimal value is not valid for definition, column=k1, value=5000, precision=1, scale=0; . src line []; 
Reason: decimal value is not valid for definition, column=k1, value=6000, precision=1, scale=0; . src line [];

总结下当前streamload 对decimal超出宽度的行为

What You Expected?

行为和1.2之前的保持一致

How to Reproduce?

No response

Anything Else?

No response

Are you willing to submit PR?

Code of Conduct

jacktengg commented 2 months ago

我们正在梳理数字类型cast的逻辑,会对行为进行统一。