apache / doris

Apache Doris is an easy-to-use, high performance and unified analytics database.
https://doris.apache.org
Apache License 2.0
12.03k stars 3.15k forks source link

[Bug] doris发送错误的fieldpacket,导致mysql驱动将数据库的largeInt解析成String #21217

Closed wxlfight closed 11 months ago

wxlfight commented 1 year ago

Search before asking

Version

doris version: 5.7.37 image

What's Wrong?

【问题描述】: doris数据库的largeint字段(截图中的op_userid字段)通过该mysql驱动读取过来(驱动在unpackField的过程),变成String类型,如图所示: image MysqlType中254对应的是String类型: image

What You Expected?

使用mysql驱动从doris数据库读取largeInt字段时,doris数据库应该发送正确的数据结构,使mysql驱动解析成UNSIGNED BIGINT。对应文档:https://doris.apache.org/docs/dev/lakehouse/multi-catalog/jdbc?_highlight=jdbc image

How to Reproduce?

【mysql 驱动版本】: `

mysql
        <artifactId>mysql-connector-java</artifactId>
        <version>8.0.28</version>

【数据库样例数据】 CREATE DATABASE doris; USE doris; CREATE TABLE IF NOT EXISTS doris.example_tbl ( timestamp DATE NOT NULL COMMENT "['0000-01-01', '9999-12-31']", type TINYINT NOT NULL COMMENT "[-128, 127]", error_code INT COMMENT "[-2147483648, 2147483647]", error_msg VARCHAR(300) COMMENT "[1-65533]", op_id BIGINT COMMENT "[-9223372036854775808, 9223372036854775807]", op_time DATETIME COMMENT "['0000-01-01 00:00:00', '9999-12-31 23:59:59']", target float COMMENT "4 字节", source double COMMENT "8 字节", lost_cost decimal(20,4) COMMENT "", remark string COMMENT "1m size", op_userid LARGEINT COMMENT "[-2^127 + 1 ~ 2^127 - 1]", plate SMALLINT COMMENT "[-32768, 32767]", iscompleted boolean COMMENT "true 或者 false" ) DUPLICATE KEY(timestamp, type) DISTRIBUTED BY HASH(type) BUCKETS 1 PROPERTIES ( "replication_allocation" = "tag.location.default: 1" );

INSERT INTO doris.example_tbl (timestamp, type, error_code, error_msg, op_id, op_time, target, source, lost_cost, remark, op_userid, plate, iscompleted) VALUES ('2022-01-01', 1, 1001, 'error message 1', 1234567890123, '2022-01-01 00:00:00', 1.23, 4.56, 123456789012345.6789, 'remark 1', 9223372036854775807, 32767, true), ('2022-01-02', 2, 1002, 'error message 2', 9876543210987, '2022-01-02 12:34:56', 4.56, 7.89, 98765432109876.5432, 'remark 2', -9223372036854775808, -32768, false), ('2022-01-03', 3, 1003, 'error message 3', 4567890123456, '2022-01-03 23:59:59', 7.89, 1.23, 11111111111111.1111, 'remark 3', 1234567890123, 0, true); select * from doris.example_tbl ` 写一个查询脚本就可以开始调试。

Anything Else?

我额外对mysql-connector-java-5.1.47.jar做了测试,也是同样的问题 image

Are you willing to submit PR?

Code of Conduct

stalary commented 1 year ago

Please show frontends look doris version

wxlfight commented 1 year ago

Please show frontends look doris version

Git : file://40f4eb9bc650@Unknown

Version : doris-1.2.1-rc01

BuildInfo : 40f4eb9bc650

BuildTime : Sat, 31 Dec 2022 13:46:08 UTC

stalary commented 1 year ago

try it in 1.2.5

yiguolei commented 12 months ago

It is a very serious problem, we will fix it.

jacktengg commented 12 months ago

MySQL client/sever protocol does not support 128 bit integer. Seems the common way is to parse out-of-range numbers as string type and the client side parse it back to numbers.

jacktengg commented 12 months ago

Then type mapping you mentioned above is for mapping MySQL data types to Doris data types, not from Doris to MySQL.

wxlfight commented 11 months ago

22835 has fix this problem