search

julien-duponchelle / python-mysql-replication

Pure Python Implementation of MySQL replication protocol build on top of PyMYSQL
2.31k stars 678 forks source link

How to use auto_position ? #576

Closed larzwow closed 8 months ago

larzwow commented 10 months ago

Because I want to ensure the function of breakpoint continuation, I used this parameter.

autoposition: Use master Auto_ Position gtid to set position

Is this parameter written directly in the following format?

Auto_ Position="e53937be cdd7-11ea-840f-b02628fc286a: 1-10593571"

Can gtid be dynamically obtained

jaehyeonpy commented 10 months ago

Hello, I want to check if this is your intention:

  1. giving the string format to auto_position

  2. getting gtid as python-mysql-replication fetches event from database

dongwook-chan commented 10 months ago

@larzwow

Is this parameter written directly in the following format?

Yes, but without any spaces. "e53937becdd7-11ea-840f-b02628fc286a:1-10593571"

Can gtid be dynamically obtained?

  1. Using a MySQL client SHOW MASTER STATUS command displays the current GTID.

  2. From instances of GtidEvent A GtidEvent is the first event in every transaction. This event has gtid as instance variable. Subsequent events within the same transaction share the same GTID.

I want to ensure the function of breakpoint continuation

GTID corresponds to a transaction not an event. Transactions typically comprise multiple events. If you want to resume a replication stream without processing any duplicate events, it's important to track both the GTID and the event sequence number within the transaction. This way, you can determine not only which transaction to resume but also which event in the transaction to resume.

larzwow commented 10 months ago

@dongwook-chan Good afternoon, thank you for your help.

When I use gtid, after shutting down and restarting, this issue is reported. What I can confirm is that the number of days the binglog log server is retained is 30 days.

pymysql.err.OperationalError: (1236, "Cannot replicate because the master purged required binary logs. Replicate the missing transactions from elsewhere, or provision a new slave from backup. Consider increasing the master's binary log expiration period. The GTID set sent by the slave is 'c76d62ce-8075-11eb-a0f8-b02628fc286a:13590524910', and the missing transactions are 'c76d62ce-8075-11eb-a0f8-b02628fc286a:1-12585696247,\ne53937be-cdd7-11ea-840f-b02628fc286a:1-10593571'")

larzwow commented 8 months ago

已解决,so easy