matrixorigin / matrixone

Hyperconverged cloud-edge native database
https://docs.matrixorigin.cn/en
Apache License 2.0
1.76k stars 274 forks source link

[Bug]: streaming: tinyint smallint and bigint type read data failed #13015

Closed heni02 closed 9 months ago

heni02 commented 9 months ago

Is there an existing issue for the same bug?

Environment

- Version or commit-id (e.g. v0.1.0 or 8b23a93):07d71e8492cb03bf2cd24d80486b1a518c9aa8cb
- Hardware parameters:
- OS type:
- Others:

Actual Behavior

stream ddl: create stream stream_8(c1 int,c2 tinyint,c3 smallint,c4 bigint,c5 int unsigned ,c6 tinyint unsigned,c7 smallint unsigned,c8 bigint unsigned)with( "type"='kafka', "topic"= 'test', "partion" = '1', "value"= 'json', "bootstrap.servers"='127.0.0.1:9092'
)

json data: {"c1": -2147483648,"c2":20,"c3": -3,"c4":8,"c5":425,"c6":55,"c7":65,"c8":1845} select stream tinyint and other int type result error , only support int type?

企业微信截图_b4868615-0fb1-4352-a7be-a3daa33b346d

Expected Behavior

No response

Steps to Reproduce

No response

Additional information

No response

gavinyue commented 9 months ago

https://github.com/matrixorigin/matrixone/commit/87b1ce95033cbaa6bb91cf8b378e8cb49848f0ca

please verify

heni02 commented 9 months ago

int类型为边界值时,查询显示null

企业微信截图_98fd3c24-2b50-449a-947d-1ab883fa593c 企业微信截图_41bfb714-4605-4e10-b9f4-2e8538b96769
gavinyue commented 9 months ago

现在的策略就是 超出边界值,显示为null。

就是无法正确被convert的, 都是null.

gavinyue commented 9 months ago

https://github.com/matrixorigin/matrixone/pull/13310

已经提了PR

gavinyue commented 9 months ago

tinyint, smallint, bigint均已经支持。

极值的问题 和https://github.com/matrixorigin/matrixone/issues/13074 重复。

heni02 commented 9 months ago

和#13074不是同一问题,该问题是json都是各数值类型的极值,但没超出范围,但是source没有读到 json:

企业微信截图_e80cd3b6-10d2-4269-9651-6e83c85941ce 企业微信截图_973c52c0-4f58-4279-98b1-7cfdb710f881

mo 数值类型支持的范围:

企业微信截图_676a38cb-a68d-48d1-9c45-e493b5c2c35b
gavinyue commented 9 months ago

https://github.com/matrixorigin/matrixone/commit/24306f4b997ae8dd7a63fb578e07d4d7b38d4305

问题是我们之前把他们当成了string, 但是float转strign的时候用了scientific notation, 就无法转化成int了

heni02 commented 9 months ago

confirm,closed commit:2a761a0c8fec04129107bd03a3222f356a23f1c6 ddl: create source stream_8(c1 int,c2 tinyint,c3 smallint,c4 bigint,c5 int unsigned ,c6 tinyint unsigned,c7 smallint unsigned,c8 bigint unsigned)with( "type"='kafka', "topic"= 'test', "partition" = '0', "value"= 'json', "bootstrap.servers"='127.0.0.1:9092' ) json: {"c1": -2147483648,"c2":20,"c3": -3,"c4":8,"c5":425,"c6":55,"c7":65,"c8":1845} {"c1": -2147483648,"c2":127,"c3": -32768,"c4":-9223372036854775808,"c5":4294967295,"c6":255,"c7":65535,"c8":18446744073709551615}

企业微信截图_36f406ad-b11d-4c58-ab7f-1022d68095fc