julien-duponchelle / python-mysql-replication

Pure Python Implementation of MySQL replication protocol build on top of PyMYSQL
2.33k stars 679 forks source link

May I ask whether BinLogStreamReader can control the detection time of binlog events? For example, it is June 29 today. I want to monitor events from 6.1, can it be realized by parameter skip_to_timestamp #404

Open legend-xu opened 1 year ago

legend-xu commented 1 year ago

May I ask whether BinLogStreamReader can control the detection time of binlog events? For example, it is June 29 today. I want to monitor events from 6.1, can it be realized by parameter skip_to_timestamp,look forward to your reply

nkr-404-NotFound commented 1 year ago

试试

    skip_to_timestamp = (datetime.datetime.now() - datetime.timedelta(minutes = 1)).timestamp()

你想退多久 delta 里面就设置多久

nkr-404-NotFound commented 1 year ago

我们只退 1分钟 =。=

legend-xu commented 1 year ago

@nankingrider 你好,我按照这样写法试了,但是只对当天有效果,我试着用skip_to_timestamp = (datetime.datetime.now() - datetime.timedelta(days= 30)).timestamp(),依然是从当天最早的时间开始监测,请问是什么原因,我的BinLogStreamReader配置如下: stream = BinLogStreamReader( connection_settings=mysql_settings, server_id=1024, blocking=True, # 没有变更时阻塞

resume_stream=True, # 重启后从最新的位置开始, 和skip_to_timestamp不同时使用

    only_events=[WriteRowsEvent, UpdateRowsEvent],  # 增改
    only_schemas=schema_list,  # 只查看当前schema
    only_tables=table_list,  # 只查看当前表
    skip_to_timestamp=(datetime.datetime.now() - datetime.timedelta(days=30)).timestamp(),
    # 这里传入时间戳(10位)
    # log_file=start_file,  # 指定binlog文件
    # log_pos=end_file,    # 指定
    # fail_on_table_metadata_unavailable=True,
)
nkr-404-NotFound commented 1 year ago

如果我没看错的话, 他只能从 current binlog 开始

nkr-404-NotFound commented 1 year ago
image

或许你们的 master db 每天都会 rotate 一个新的 binlog 。。。 所以你总是只能从当天的binlog开始获取

nkr-404-NotFound commented 1 year ago

或许你可以传入 binlog file name 和 binlog pos。

legend-xu commented 1 year ago

@nankingrider 好的,非常感谢

dongwook-chan commented 1 year ago

@legend-xu Thank you for your question. My team and I will certainly look into this matter. Our initial approach will be to examine all events starting from the first file (instead of current file) within the available binary log files and apply a filter based on the date/time parameter that you have supplied. We will continue to keep you updated as we progress in addressing this issue.