apache / inlong

Apache InLong - a one-stop, full-scenario integration framework for massive data
https://inlong.apache.org/
Apache License 2.0
1.4k stars 530 forks source link

[Improve][SDK] Optimization of ultra-long field processing in InlongSDK #10463

Closed vernedeng closed 1 month ago

vernedeng commented 5 months ago

Description

SDK does not limit the maximum report data size, while dataproxy only supports processing a maximum of 500k bytes for each message.

image

Location: inlong-sdk/dataproxy-sdk/

Requirements:

  1. Support automatic ultra-long data truncation in SDK
  2. User-configurable automatic truncation
  3. Provide the default value of the allowed data length

This issue is a dedicated issue for the 2024 Tencent Rhino-bird Open Source Talent Training event and is only available for college students to claim 【Claim the issue】Sign up for the corresponding issue in columns M~R of the Research and Study Base "issue Camp" to be considered as successfully claimed. 【Complete the issue】For students who have claimed the issue, please also reply "Successfully claimed this issue" in the comment section of this issue; if there is no progress submitted within 7 days, including but not limited to comments, commits, or Pull Requests, it will be considered as voluntarily giving up the issue and the committee will release the issue to the next person in line.

InLong Component

InLong SDK

Are you willing to submit PR?

Code of Conduct

Hackerismydream commented 4 months ago

Successfully claimed this issue

ratSquealer commented 4 months ago

已领取该问题

ratSquealer commented 4 months ago

初步写好处理代码,但是在测试TcpClientExample.java文件(根据调用修改后),报链接错误: Connect Manager error, message: Unsupported or unrecognized SSL message QQ图片20240714224212 QQ图片20240714224423 在调查问题后,手动修改为http后,可以连接,但从代码角度来看,要修改TcpClientExample.java和ProxyClientConfig.java文件(重点),不知道是否进行,希望能得到导师指导。 我是在win11下面部署的inlong QQ图片20240714224325

github-actions[bot] commented 2 months ago

This issue is stale because it has been open for 60 days with no activity.