search

apache / inlong

Apache InLong - a one-stop, full-scenario integration framework for massive data
https://inlong.apache.org/
Apache License 2.0
1.4k stars 530 forks source link

[INLONG-344] Add broker receiving time in message property #940

Closed dockerzhang closed 3 years ago

dockerzhang commented 3 years ago

When using the offset setting function of tube, there is a problem that there is no reference time coordinate. The business data usually uses DT in tdmsg as the business time, but it also needs a broker receiving time as the reference coordinate to mark the offset I need to set back. Basic requirements: the time and offset of messages are in order, that is, the order of two messages on the same partition in the offset and the order of the broker's receiving time should be consistent.

 

----------------------------------------------------------------------------------------------------------------------------------------

当使用tube的offset回设功能时，会遇到一个没有基准时间坐标的问题，业务数据通常使用tdmsg中的dt作为业务时间，但是还需要一个broker接收时间作为基准坐标，用来标记我需要回设的offset。基本要求：消息的时间和offset保序性，即同一个分区上的两条消息，他们在offset的顺序和broker接收时间的顺序要保持一致。

JIRA link - [INLONG-344] created by steventian

gosonzhang commented 3 years ago

Closed due to long periods of inactivity

Please reopen it if needed