go-mysql-org / go-mysql

a powerful mysql toolset with Go
MIT License
4.52k stars 967 forks source link

a problem about mysql dump zombie process #889

Open deadspacewii opened 1 month ago

deadspacewii commented 1 month ago

when i use go-mysql to sync data from mysql, it well occur Binlog Dump zombie process.how can i resolve it image

                cfg := canal.NewDefaultConfig()
        cfg.Addr = conf.Database.Addr
        cfg.User = conf.Database.Username
        cfg.Flavor = "mysql"
        cfg.Password = conf.Database.Password
        // We only care table canal_test in test db
        cfg.Dump.TableDB = s.Rule.GetSourceDataBase()
        if conf.Database.Mysqldump != "" {
            cfg.Dump.ExecutionPath = conf.Database.Mysqldump
        }
        cfg.Dump.Tables = []string{s.Rule.GetSourceTable()}

        c, err := canal.NewCanal(cfg)
        if err != nil {
            logger.Error(fmt.Sprintf("init mysql canal fail: %s, instanceId: %s", err.Error(), s.GetInstanceId()))
            return nil, err
        }
               c.RunFrom(mysql.Position{Name: filename, Pos: uint32(pos)})
dveeden commented 1 month ago

What MySQL version are you using? What is in the MySQL logs (with log_error_verbosity=3)

lance6716 commented 1 month ago

I remember this library should be able to KILL the zombie connection

https://github.com/go-mysql-org/go-mysql/blob/2e5c5ac5becaebcd49b34165fb64a9b72957bfe8/replication/binlogsyncer.go#L910

deadspacewii commented 1 month ago

What MySQL version are you using? What is in the MySQL logs (with log_error_verbosity=3)

mysql version is 8.0.36 and my mysql is running in docker image

deadspacewii commented 1 month ago

I remember this library should be able to KILL the zombie connection

https://github.com/go-mysql-org/go-mysql/blob/2e5c5ac5becaebcd49b34165fb64a9b72957bfe8/replication/binlogsyncer.go#L910

but it actual happen

deadspacewii commented 1 month ago

log_error_verbosity=3

there are logs: 2024-06-14T03:16:13.867457Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel. 2024-06-14T03:16:13.905644Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Bind-address: '::' port: 33060 2024-06-14T03:16:13.905761Z 0 [System] [MY-010931] [Server] C:\Program Files\MySQL\MySQL Server 8.0\bin\mysqld.exe: ready for connections. Version: '8.0.35' socket: '' port: 3306 MySQL Community Server - GPL. 2024-06-14T07:05:11.976428Z 8 [Warning] [MY-013360] [Server] Plugin mysql_native_password reported: ''mysql_native_password' is deprecated and will be removed in a future release. Please use caching_sha2_password instead' 2024-06-14T07:05:24.600115Z 9 [Warning] [MY-013360] [Server] Plugin mysql_native_password reported: ''mysql_native_password' is deprecated and will be removed in a future release. Please use caching_sha2_password instead' 2024-06-14T07:05:38.109822Z 10 [Warning] [MY-013360] [Server] Plugin mysql_native_password reported: ''mysql_native_password' is deprecated and will be removed in a future release. Please use caching_sha2_password instead' 2024-06-14T07:05:41.714596Z 11 [Warning] [MY-013360] [Server] Plugin mysql_native_password reported: ''mysql_native_password' is deprecated and will be removed in a future release. Please use caching_sha2_password instead' 2024-06-14T07:05:48.800317Z 12 [Warning] [MY-013360] [Server] Plugin mysql_native_password reported: ''mysql_native_password' is deprecated and will be removed in a future release. Please use caching_sha2_password instead' 2024-06-14T07:11:46.711720Z 13 [Warning] [MY-013360] [Server] Plugin mysql_native_password reported: ''mysql_native_password' is deprecated and will be removed in a future release. Please use caching_sha2_password instead' 2024-06-14T07:12:02.544257Z 14 [Warning] [MY-013360] [Server] Plugin mysql_native_password reported: ''mysql_native_password' is deprecated and will be removed in a future release. Please use caching_sha2_password instead' 2024-06-14T07:12:02.596410Z 15 [Warning] [MY-013360] [Server] Plugin mysql_native_password reported: ''mysql_native_password' is deprecated and will be removed in a future release. Please use caching_sha2_password instead' 2024-06-14T07:14:12.390574Z 16 [Warning] [MY-013360] [Server] Plugin mysql_native_password reported: ''mysql_native_password' is deprecated and will be removed in a future release. Please use caching_sha2_password instead' 2024-06-14T07:14:12.393994Z 17 [Warning] [MY-013360] [Server] Plugin mysql_native_password reported: ''mysql_native_password' is deprecated and will be removed in a future release. Please use caching_sha2_password instead' 2024-06-14T07:16:37.628869Z 18 [Warning] [MY-013360] [Server] Plugin mysql_native_password reported: ''mysql_native_password' is deprecated and will be removed in a future release. Please use caching_sha2_password instead' 2024-06-14T07:16:37.631618Z 19 [Warning] [MY-013360] [Server] Plugin mysql_native_password reported: ''mysql_native_password' is deprecated and will be removed in a future release. Please use caching_sha2_password instead' 2024-06-14T07:24:00.607178Z 18 [Note] [MY-010914] [Server] Aborted connection 18 to db: 'unconnected' user: 'anyops' host: 'localhost' (Got an error reading communication packets). 2024-06-14T07:24:39.918805Z 20 [Warning] [MY-013360] [Server] Plugin mysql_native_password reported: ''mysql_native_password' is deprecated and will be removed in a future release. Please use caching_sha2_password instead' 2024-06-14T07:24:39.921726Z 21 [Warning] [MY-013360] [Server] Plugin mysql_native_password reported: ''mysql_native_password' is deprecated and will be removed in a future release. Please use caching_sha2_password instead' 2024-06-14T07:24:39.928300Z 21 [Note] [MY-010462] [Repl] Start binlog_dump to source_thread_id(21) replica_server(1704), pos(WIN-OKO0DC0TPJV-bin.000011, 1361)

deadspacewii commented 1 month ago

What MySQL version are you using? What is in the MySQL logs (with log_error_verbosity=3)

after I KILL source_thread_id(21) 。source_thread_id(22) occur immediately

deadspacewii commented 1 month ago

What MySQL version are you using? What is in the MySQL logs (with log_error_verbosity=3)

this is another log: image

lance6716 commented 1 month ago

Please provide a runnable code for us to reproduce.