OpenAtomFoundation / pika

Pika is a Redis-Compatible database developed by Qihoo's infrastructure team.
BSD 3-Clause "New" or "Revised" License
5.83k stars 1.19k forks source link

binlog_sender 工具支持持续拉取 #2820

Open haoxinyuanlalala opened 1 month ago

haoxinyuanlalala commented 1 month ago

Which PikiwiDB functionalities are relevant/related to the feature request?

No response

Description

当前的 binlog_sender 工具支持拉取二进制日志并应用,但只能拉取发起任务时刻的所有 binlog,无法实现持续拉取。这限制了工具在需要实时或持续监控 binlog 更新的场景中的应用。为了提升工具的实用性和功能性,建议改进 binlog_sender,使其能够支持持续性拉取 binlog

Proposed solution

对 binlog_sender 进行改进,使其能够在初次拉取指定时刻的 binlog 后,能够持续监听并拉取后续产生的 binlog。可以通过以下方式实现: 引入持续性监听机制,检测到新的 binlog 时自动拉取。 提供相应的配置选项,允许用户指定持续拉取模式。

Alternatives considered

目前的替代方案是每隔一定时间手动发起 binlog 拉取任务,但这种方式在实时性和自动化程度上存在较大不足,无法满足实际应用中的高效性要求。因此,迫切需要对现有工具进行改进。

Issues-translate-bot commented 1 month ago

Bot detected the issue body's language is not English, translate it automatically.


Title: binlog_sender tool supports continuous pulling

Which PikiwiDB functionalities are relevant/related to the feature request?

No response

Description

The current binlog_sender tool supports pulling binary logs and applying them, but it can only pull all binlogs at the time when the task is initiated, and cannot achieve continuous pulling. This limits the application of the tool in scenarios where real-time or continuous monitoring of binlog updates is required. In order to improve the utility and functionality of the tool, it is recommended to improve binlog_sender so that it can support continuous pulling of binlog

Proposed solution

Improve binlog_sender so that it can continue to monitor and pull subsequent binlogs after pulling the binlog at a specified time for the first time. This can be achieved by: Introducing a continuous monitoring mechanism to automatically pull new binlogs when they are detected. Provide corresponding configuration options to allow users to specify continuous pull mode.

Alternatives considered

The current alternative is to manually initiate binlog pull tasks at regular intervals, but this method has major shortcomings in real-time performance and automation, and cannot meet the efficiency requirements in practical applications. Therefore, there is an urgent need to improve existing tools.