Memory leak in version 1.3.2

[michael-bouvy]

First of all thanks for your module!

I use the following command within my Dockerfile to install it:

opm get anjia0532/lua-resty-maxminddb

However, in the past days, version has bumped from 1.3.0 to 1.3.2, introducing an issue with memory usage, which constantly raises up to several gigabytes.

When downgrading to 1.3.0, I do not have the issue anymore.

Might this be caused by https://github.com/anjia0532/lua-resty-maxminddb/commit/ea529cd3e9da758111ba73e1f1cefe375a4b9d72 ?

My LUA script loaded within a content_by_lua_file directive looks like the following:

local geo = require 'resty.maxminddb'

if not geo.initted() then
    geo.init("/path/to/GeoLite2-Country.mmdb")
end

local ip = ngx.var.arg_ip or headers["x-real-ip"] or headers["x-forwarded-for"] or ngx.var.remote_addr
local res, err = geo.lookup(ip)

[anjia0532]

maybe ,I have been busy recently , There is no time to solve this problem . This issues may help you .

rollback ea529cd and use wrk/ab/gatling to test it.

[Hoops]

I wonder, could you try replacing this line:

local entry_data_list = ffi_cast('MMDB_entry_data_list_s **const',ffi_new("MMDB_entry_data_list_s"))

With:

local entry_data_list = ffi_new("MMDB_entry_data_list_s*[1]")

And see if that makes any difference to the memory behaviour?

[anjia0532]

@Hoops @michael-bouvy

In my test, it couldn't be reproduced.

vagrant+virtualbox for this repo https://github.com/anjia0532/lua-resty-maxminddb-vagrant.git

ref https://github.com/anjia0532/lua-resty-maxminddb/issues/9#issuecomment-404742603

[michael-bouvy]

Maybe is it related to OpenResty version, but I have to way finding what version I was using at the time. I'll try to reproduce.

[anjia0532]

@michael-bouvy

My Test Env

1. ubuntu 16.04.7 LTS (Xenial Xerus) amd64 ( cat /etc/os-release )
2. wrk version 4.10
3. openresty version openresty/1.17.8.2 ( openresty -v )
4. lua-resty-maxminddb version 1.3.2 ( sudo opm --install-dir /usr/local/openresty/ list )
5. maxmind db is /tmp/GeoLite2-Country.mmdb ( cat /usr/local/openresty/nginx/conf/nginx.conf | grep mmdb )
6. libmaxminddb-dev:amd64 and libmaxminddb0:amd64 and mmdb-bin version 1.0.4-2.1 , libgeoip1:amd64 version 1.6.9-1,geoipupdate version 2.2.1-1 , geoip-database version 20160408-1 ( dpkg -l | grep geo )
7. GeoLite2-Country_20200929.tar.gz means GeoLite2-Country db version is 20200929 (this repo's GeoLite2-Country.mmdb file)

Wrk Result

vagrant@openresty:/tmp$ ./wrk -t50 -c200 -d120s -s ./test.lua --latency http://127.0.0.1
Running 2m test @ http://127.0.0.1
50 threads and 200 connections
Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    25.73ms   24.23ms 987.43ms   98.34%
    Req/Sec   166.19     28.21     1.32k    83.94%
Latency Distribution
    50%   22.96ms
    75%   27.39ms
    90%   33.08ms
    99%   60.74ms
989959 requests in 2.00m, 705.81MB read
Requests/sec:   8243.20
Transfer/sec:      5.88MB

ref https://github.com/anjia0532/lua-resty-maxminddb-vagrant/blob/master/README.md

Use my Repo and make sure the problem can be reproduced