Memory leak with many sequential lookups to the same reader?

[danmia]

I'm seeing resident memory grow continuously with a high rate (~12k/sec) of sequential lookups to the same reader(multiple workers.. each have their own reader). It looks like certain IP blocks for one reason or another seem to exacerbate it more than others .. I can't put my finger on the why yet but here is as trivial as an example as I could do. Worth noting that closing the reader does in fact clean everything up but it's a serious performance hit. Perhaps it's my use?

import (
    "errors"
    "fmt"
    "log"
    "net"
    "os"
    "github.com/oschwald/geoip2-golang"
)

type Asn struct {
    Number uint   `json:"asnumber"`
    Org    string `json:"asorg"`
}

type AsnDB struct {
    DB *geoip2.Reader
}

func (a *AsnDB) AsnDBInit(path string) error {

    db, geoerr := geoip2.Open(path)
    if geoerr != nil {
        fmt.Printf("Error opening maxmind database:  %s\n", geoerr)
        return errors.New("Error opening maxmind database: " + geoerr.Error())
    }

    a.DB = db
    return nil
}

func (a *AsnDB) GetAsn(ip net.IP) (Asn, error) {
    var as Asn
    asn, serr := a.DB.ASN(ip)

    if serr != nil {
        fmt.Printf("Error looking up %s in maxmind ASN database:  %s\n", ip.String(), serr.Error())
        asn = nil
        return as, errors.New("ASN DB Error looking up ip: " + ip.String() + " error: " + serr.Error())
    }

    as.Number = asn.AutonomousSystemNumber
    as.Org = asn.AutonomousSystemOrganization

    return  as, nil
}

func main ()  {
    done := make(chan bool, 1)

    for i:=0; i < 1000; i++ {
        go testgeo()
    }   
    <-done
}

func testgeo()  {
    cidrs := []string{"192.0.0.0/8", "10.0.0.0/24", "192.168.0.0/16"}
        var db AsnDB
        aserr := db.AsnDBInit("/path/to/db")
        if aserr != nil {
                fmt.Println("Could not open asn db: " + aserr.Error())
                os.Exit(1)
        }

        for {
            for i:=0; i < len(cidrs); i++ {
                ip, ipnet, err := net.ParseCIDR(cidrs[i])
                if err != nil {
                        log.Fatal(err)
                }
                for ip := ip.Mask(ipnet.Mask); ipnet.Contains(ip); inc(ip) {
                    org,_ := db.GetAsn(ip)
                    fmt.Println(ip.String() + " org: " + org.Org)
                }
            }
        }
}

func inc(ip net.IP) {
    for j := len(ip)-1; j>=0; j-- {
    ip[j]++
    if ip[j] > 0 {
        break
    }
    }
}

[danmia]

addendum.. go version go1.10.3 linux/amd64 4.14.51-66.38.amzn2.x86_64

[oschwald]

I ran the script and everything seemed normal to me. I have also been using this in production for several years in a high performance web service and the max response time for the service is quite low. Does the same thing happen if you load the database into memory rather than memory mapping it? Perhaps you are hitting a page that is not in cache.

[danmia]

That very well could be it ... That never even crossed my mind, but constantly paging in as it needs different data from different workers could be it. Is there preload ability in the api?

Good catch by the way and thanks for the quick response... I starred at this for way too long. I also haven't really dealt with mmap much. It never occurred to me that pages read in would add to process RES memory.

[oschwald]

You can preload by reading the file into memory and then using FromBytes to create the reader.

On Tue, Aug 7, 2018, 8:34 PM Dan Murphy notifications@github.com wrote:

That very well could be it ... That never even crossed my mind, but constantly paging in as it needs different data from different workers could be it. Is there preload ability in the api?

Good catch by the way and thanks for the quick response... I starred at this for way too long. I also haven't really dealt with mmap much. It never occurred to me that pages read in would add to process RES memory.

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/oschwald/geoip2-golang/issues/38#issuecomment-411273664, or mute the thread https://github.com/notifications/unsubscribe-auth/AARBM6Yv_gjFD7RnvJC0H1t8KKxiE76Jks5uOlxggaJpZM4VzLk3 .

[danmia]

Nice. Thanks again for the help.