[Bug]: streaming: json data greater than char or varchar type size，source can read data

heni02 commented 11 months ago

Is there an existing issue for the same bug?

[X] I have checked the existing issues.

Environment

- Version or commit-id (e.g. v0.1.0 or 8b23a93):c4b255cbd0ee319c50210706a94eaf5d6084aadf
- Hardware parameters:
- OS type:
- Others:

Actual Behavior

json data greater than char or varchar type size，it is expect read null json：{"c1": "aaaaaaaa","c2":"bbbbbbbb"}

ddl： create source stream_8(c1 int,c2 tinyint,c3 smallint,c4 bigint,c5 int unsigned ,c6 tinyint unsigned,c7 smallint unsigned,c8 bigint unsigned)with( "type"='kafka', "topic"= 'test', "partition" = '0', "value"= 'json', "bootstrap.servers"='127.0.0.1:9092' )

Expected Behavior

No response

Steps to Reproduce

No response

Additional information

No response

gavinyue commented 11 months ago

这个字符串的行为应该是和外表的行为一致。超出范围的字符串是不是设为null 还是 truncate.

需要产品帮助一下 @LiSong0214

gavinyue commented 11 months ago

在征求pm 意见中

LiSong0214 commented 11 months ago

按脏数据处理

gavinyue commented 11 months ago

明天解决

gavinyue commented 11 months ago

Pending

gavinyue commented 11 months ago

Pending

gavinyue commented 10 months ago

Pending

gavinyue commented 10 months ago

Pending

gavinyue commented 10 months ago

Pending

gavinyue commented 10 months ago

Pending

fengttt commented 10 months ago

adaptor.go should just reuse code in external.go, there is no reason to have so many ways to parse json.

qingxinhome commented 9 months ago

We will handle it later

qingxinhome commented 9 months ago

We will handle it later

qingxinhome commented 9 months ago

Pending processing

qingxinhome commented 9 months ago

Pending processing

qingxinhome commented 9 months ago

Pending processing