go-sql-driver / mysql

Go MySQL Driver is a MySQL driver for Go's (golang) database/sql package
https://pkg.go.dev/github.com/go-sql-driver/mysql
Mozilla Public License 2.0
14.46k stars 2.3k forks source link

too may ram usage when lost connection #1549

Closed parsibox closed 7 months ago

parsibox commented 7 months ago

hi i use go 1.20 with mysql 8.0.33-0ubuntu0.20.04.2
i use pool connection i have a PingContext 40*time.Second and randomly i get context deadline exceeded image this is db.Stats output at error time

{
    "Idle": 0,
    "InUse": 15,
    "MaxIdleClosed": 0,
    "MaxLifetimeClosed": 57,
    "MaxOpenConnections": 15,
    "OpenConnections": 15,
    "WaitCount": 141,
    "WaitDuration": "2m39.604195004s",
}

i have no error log on my mysql server and other applications with that mysql server work find at that time i can not understand what happen also at this time ram usage goes up from 50.3M to 1.6 G

methane commented 7 months ago

I can't compile jpeg. Post a runnable source code.

parsibox commented 7 months ago
func mysql_check_connection() {
    logger.Info("mysql_check_connection call")
    //db.Execute("select 42")
    for {
        var errMy error
        ctx, _ := context.WithTimeout(context.Background(), 40*time.Second)
        errMy = db.PingContext(ctx)
        if errMy != nil {
            stats := db.Stats()
            fields := logrus.Fields{
                "MaxOpenConnections": stats.MaxOpenConnections,
                "OpenConnections":    stats.OpenConnections,
                "InUse":              stats.InUse,
                "Idle":               stats.Idle,
                "WaitCount":          stats.WaitCount,
                "WaitDuration":       stats.WaitDuration.String(),
                "MaxIdleClosed":      stats.MaxIdleClosed,
                "MaxLifetimeClosed":  stats.MaxLifetimeClosed,
                "err":                errMy,
            }
            logger.ErrorWithFields(fields, "ConMysql error on mysql_check_connection Err : ")
            Heartbeat = false
            //fmt.Println(" Heartbeat change to false :", Heartbeat)
            /**/
            mutex.Lock()
            db.Close()
            start()
            mutex.Unlock()

            //go start()
            return

        }
        Heartbeat = true
        time.Sleep(3 * time.Second)
    }
}
parsibox commented 7 months ago

how can increase total time blocked waiting for a new connection?

methane commented 7 months ago

Oh! You don't call cancel func. You are leaking many contexts.

parsibox commented 7 months ago

do you mean this change?

        ctx, cancel := context.WithTimeout(context.Background(), 40*time.Second)
        defer cancel() // Call cancel function to release resources when the function returns
methane commented 7 months ago

Don't use defer in a loop. Just call cancel() after PingContext returns.

parsibox commented 7 months ago

i change it

        ctx, cancel := context.WithTimeout(context.Background(), 40*time.Second)
        errMy = db.PingContext(ctx)
        cancel()

how can increase total time blocked waiting for a new connection?

methane commented 7 months ago

I cannot understand what you mean. If you want to be blocked long time, set max open connection to 1 and query "SELECT SLEEP(100)" from 100 goroutines. 99 goroutines need to wait 100 sec. 98 goroutines need to wait another 100 sec. ...

parsibox commented 7 months ago

no please see this db stat

{
    "Idle": 0,
    "InUse": 15,
    "MaxIdleClosed": 0,
    "MaxLifetimeClosed": 57,
    "MaxOpenConnections": 15,
    "OpenConnections": 15,
    "WaitCount": 141,
    "WaitDuration": "2m39.604195004s",
}

my MaxOpenConnections was equaled to OpenConnections and WaitDuration is 2m39 how can i increase WaitDuration to 10 m ?

methane commented 7 months ago

Maybe, you misunderstand it. Wait duration in db stat is what I said. Total time for waiting connection from pool.

parsibox commented 7 months ago

ok but is there any time limit for waiting connection from pool?

methane commented 7 months ago

It is unlimited by default. So increasing context timeout also increase timeout to get connection from pool.

methane commented 7 months ago

Anyway, here is issue tracker, not user forum.