upper / db

Data Access Layer (DAL) for PostgreSQL, CockroachDB, MySQL, SQLite and MongoDB with ORM-like features.
https://upper.io/
MIT License
3.54k stars 235 forks source link

Memory leak with .Count() #628

Closed aj3423 closed 4 months ago

aj3423 commented 3 years ago

I was using sqlite3 and gorm and found a memory leak problem, then I tried upper-db, the same problem. Maybe it is caused by the driver library mattn/go-sqlite3? But from the pprof backtrace, most leaks are ended in code of upper, not in driver code.

To reproduce, run the following code, then open this URL in browser: http://localhost:3344/debug/pprof.

package main

import (
    "fmt"
    "runtime"

    "github.com/upper/db/adapter/sqlite"

    "net/http"
    _ "net/http/pprof"
)

type Person struct {
    Age int `db:"age"`
}

func leak_upper() {
    sess, e := sqlite.Open(sqlite.ConnectionURL{Database: `test.db`})
    if e != nil {
        panic(e)
    }
    sess.SQL().Exec(`create table "person" ("age" int);`)

    P := sess.Collection(`person`)

    for i := 0; i < 10000; i++ {
        P.Find(`age`, 123).Count()
    }
    //for i := 0; i < 10000; i++ {
        //P.Find(`age`, 123).Count()
    //}
    //for i := 0; i < 10000; i++ {
        //P.Find(`age`, 123).Count()
    //}
}
func main() {
    go func() { // set up pprof server, for analyzing heap leak
        fmt.Println("open in browser: http://localhost:3344/debug/pprof")
        e := http.ListenAndServe("0.0.0.0:3344", nil)
        if e != nil {
            panic(e)
        }
    }()

    leak_upper()

    runtime.GC()

    fmt.Println("press enter to exit")
    fmt.Scanln()
}

The heap grows when .Count() called, in my case the number is near 70+. Click it for details, all these heap allocations point to sqladapter.(*Result).Count

a sample trace: image

How to solve this? Thanks.