bitpoke / mysql-operator

Asynchronous MySQL Replication on Kubernetes using Percona Server and Openark's Orchestrator.
https://www.bitpoke.io/docs/mysql-operator/getting-started/
Apache License 2.0
1.03k stars 276 forks source link

replica's binlog would be reset, once pod is reconstructed. #643

Closed jianhaiqing closed 3 years ago

jianhaiqing commented 3 years ago

https://github.com/presslabs/mysql-operator/blob/47436cc5edd3d974239e45c994ccf7e22afa65e0/pkg/sidecar/appconf.go#L196

jianhaiqing commented 3 years ago

The logic of FirstPodInSet should be removed. @AMecea

AMecea commented 3 years ago

Hi @jianhaiqing, can you explain a little more? Thanks!

jianhaiqing commented 3 years ago
// the slave pod doesn't need to back up sys_operator.status, Xtrabackup might have some bugs that
// cause the table to be unclean. We can do this cleanup before the slave pod starts to avoid accidents.
if !cfg.IsFirstPodInSet() {
queries = append(queries, fmt.Sprintf("DROP TABLE IF EXISTS %s.%s",
    constants.OperatorDbName, constants.OperatorStatusTableName))
}

The logic is used, replica is created, and xtrabackup's "--tables-exclude=sys_operator.status" can't work properly. But, once the replica is running for a while, and there's a rolling update for each pod. Then the status would be dropped. again, and reset master; set global gtid_purged='' would be executed again. master-replica io_thread or sql_thread might be stopped due to 'binlog is purged or binlog is replayed already.

jianhaiqing commented 3 years ago

how about renaming or removing xtrabackup_binlog_info after set global gtid_purged='@gtid' is run ? remove --tables-exclude from xtrabackup ?

jianhaiqing commented 3 years ago

It really confuse me, could you explain the logical more detailed for me. In my option, it's fatal issue, replicas' binlog might be reset again once it's reconstructed again.

pchy93 commented 3 years ago

this problem is sloved ? i still has this issues