MMDB_free_entry_data_list (entry_data_list=0x23) at maxminddb.c:1860

[atomyuk]

still segfaulting, even in version 0.6

[anjia0532]

maybe u can delete maxm.MMDB_free_entry_data_list(entry_data_list[0]) from lua-resty-maxminddb.lua to try again.

my env

$ cat /etc/os-release
## output 
NAME="Ubuntu"
VERSION="16.04.4 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.4 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial

Prerequisites

1. install maxmind/libmaxminddb
2. download GeoLite2-City.tar.gz && save maxminddb file .
3. install openresty
4. opm --install-dir /path/to/your/lualib_dir/ get anjia0532/lua-resty-maxminddb download latest version

vhost config

# lua_package_path '/path/to/your/lualib_dir/?.lua;;';
server {
  listen 12345;
  server_name 127.0.0.1;
  location / {
    content_by_lua_block {
      local cjson = require 'cjson'
      local geo = require 'resty.maxminddb'
      geo.init("/path/to/GeoLite2-City.mmdb")
      local res,err = geo.lookup(ngx.var.arg_ip or ngx.var.remote_addr) --support ipv6 e.g. 2001:4860:0:1001::3004:ef68

      if not res then
         ngx.log(ngx.ERR,'failed to lookup by ip ,reason:',err)
      end

      ngx.say("full :",cjson.encode(res))
      if ngx.var.arg_node then
         ngx.say("node name:",ngx.var.arg_node," ,value:", cjson.encode(res[ngx.var.arg_node] or {}))
      end

      if ((res or {}).country or {}).iso_code then
          ngx.say(res['country']['iso_code'])
          ngx.exit(ngx.HTTP_OK)
      end
      ngx.say('FAIL')
    }
  }
}

run it

$ curl http://127.0.0.1:12345/?ip=95.134.195.14
## output
full :{"subdivisions":[{"geoname_id":687699,"names":{"en":"Zaporizhia","ru":"Запорожская область","de":"Saporischschja","fr":"Oblast de Zaporijia"},"iso_code":"23"}],"city":{"geoname_id":687700,"names":{"en":"Zaporizhia","ru":"Запорожье","de":"Saporischschja","fr":"Zaporijia"}},"registered_country":{"geoname_id":690791,"names":{"en":"Ukraine","ru":"Украина","fr":"Ukraine","pt-BR":"Ucrânia","zh-CN":"乌克兰","es":"Ucrania","de":"Ukraine","ja":"ウクライナ共和国"},"iso_code":"UA"},"country":{"geoname_id":690791,"names":{"en":"Ukraine","ru":"Украина","fr":"Ukraine","pt-BR":"Ucrânia","zh-CN":"乌克兰","es":"Ucrania","de":"Ukraine","ja":"ウクライナ共和国"},"iso_code":"UA"},"continent":{"geoname_id":6255148,"names":{"en":"Europe","ru":"Европа","fr":"Europe","pt-BR":"Europa","zh-CN":"欧洲","es":"Europa","de":"Europa","ja":"ヨーロッパ"},"code":"EU"},"location":{"time_zone":"Europe\/Zaporozhye","longitude":35.2833,"accuracy_radius":50,"latitude":47.85},"postal":{"code":"70459"}}
UA

[atomyuk]

when i deleted maxm.MMDB_free_entry_data_list(entry_data_list[0]) its not segfaulting!!! what is the purpose of maxm.MMDB_free_entry_data_list(entry_data_list[0])?

[anjia0532]

The MMDB_get_entry_data_list() and MMDB_get_metadata_as_entry_data_list() functions will allocate the linked list structure from the heap. Call this function to free the MMDB_entry_data_list_s structure.

ref https://github.com/maxmind/libmaxminddb/blob/master/doc/libmaxminddb.md#mmdb_free_entry_data_list

maybe u can change entry_data_list[0] to entry_data_list ?

[atomyuk]

after i changed maxm.MMDB_free_entry_data_list(entry_data_list[0]) to maxm.MMDB_free_entry_data_list(entry_data_list)

its failing with

2018/07/12 08:35:13 [error] 16499#16499: *1 lua entry thread aborted: runtime error: /usr/local/openresty/site/lualib/resty/maxminddb.lua:331: bad argument #1 to 'MMDB_free_entry_data_list' (cannot convert 'struct MMDB_entry_data_list_s *const' to 'struct MMDB_entry_data_list_s const') stack traceback: coroutine 0: [C]: in function 'MMDB_free_entry_data_list' /usr/local/openresty/site/lualib/resty/maxminddb.lua:331: in function 'lookup' /usr/local/openresty/nginx/conf/resty/lb/geoip.lua:15: in function 'lookup' content_by_lua(nginx.conf:52):5: in function <content_by_lua(nginx.conf:52):1>, client: 127.0.0.1, server: localhost, request: "GET /geoip HTTP/1.1", host: "localhost"

[anjia0532]

emmm. it's work for ubuntu(mean's maxm.MMDB_free_entry_data_list(entry_data_list[0])). I dont have Red Hat Enterprise Linux 7.6.1 os. sorry.

can u try command line for maxmind/libmaxminddb?

$ mmdblookup --file /path/to/GeoLite2-City.mmdb --ip 95.134.195.14
## output
{
    "city": 
      {
        "geoname_id": 
          687700 <uint32>
        "names": 
          {
            "de": 
              "Saporischschja" <utf8_string>
            "en": 
              "Zaporizhia" <utf8_string>
            "fr": 
              "Zaporijia" <utf8_string>
            "ru": 
              "Запорожье" <utf8_string>
          }
      }
    "continent": 
      {
        "code": 
          "EU" <utf8_string>
        "geoname_id": 
          6255148 <uint32>
        "names": 
          {
            "de": 
              "Europa" <utf8_string>
            "en": 
              "Europe" <utf8_string>
            "es": 
              "Europa" <utf8_string>
            "fr": 
              "Europe" <utf8_string>
            "ja": 
              "ヨーロッパ" <utf8_string>
            "pt-BR": 
              "Europa" <utf8_string>
            "ru": 
              "Европа" <utf8_string>
            "zh-CN": 
              "欧洲" <utf8_string>
          }
      }
    "country": 
      {
        "geoname_id": 
          690791 <uint32>
        "iso_code": 
          "UA" <utf8_string>
        "names": 
          {
            "de": 
              "Ukraine" <utf8_string>
            "en": 
              "Ukraine" <utf8_string>
            "es": 
              "Ucrania" <utf8_string>
            "fr": 
              "Ukraine" <utf8_string>
            "ja": 
              "ウクライナ共和国" <utf8_string>
            "pt-BR": 
              "Ucrânia" <utf8_string>
            "ru": 
              "Украина" <utf8_string>
            "zh-CN": 
              "乌克兰" <utf8_string>
          }
      }
    "location": 
      {
        "accuracy_radius": 
          50 <uint16>
        "latitude": 
          47.850000 <double>
        "longitude": 
          35.283300 <double>
        "time_zone": 
          "Europe/Zaporozhye" <utf8_string>
      }
    "postal": 
      {
        "code": 
          "70459" <utf8_string>
      }
    "registered_country": 
      {
        "geoname_id": 
          690791 <uint32>
        "iso_code": 
          "UA" <utf8_string>
        "names": 
          {
            "de": 
              "Ukraine" <utf8_string>
            "en": 
              "Ukraine" <utf8_string>
            "es": 
              "Ucrania" <utf8_string>
            "fr": 
              "Ukraine" <utf8_string>
            "ja": 
              "ウクライナ共和国" <utf8_string>
            "pt-BR": 
              "Ucrânia" <utf8_string>
            "ru": 
              "Украина" <utf8_string>
            "zh-CN": 
              "乌克兰" <utf8_string>
          }
      }
    "subdivisions": 
      [
        {
          "geoname_id": 
            687699 <uint32>
          "iso_code": 
            "23" <utf8_string>
          "names": 
            {
              "de": 
                "Saporischschja" <utf8_string>
              "en": 
                "Zaporizhia" <utf8_string>
              "fr": 
                "Oblast de Zaporijia" <utf8_string>
              "ru": 
                "Запорожская область" <utf8_string>
            }
        }
      ]
  }

[atomyuk]

  {
    "city": 
      {
        "geoname_id": 
          687700 <uint32>
        "names": 
          {
            "de": 
              "Saporischschja" <utf8_string>
            "en": 
              "Zaporizhia" <utf8_string>
            "fr": 
              "Zaporijia" <utf8_string>
            "ru": 
              "Запорожье" <utf8_string>
          }
      }
    "continent": 
      {
        "code": 
          "EU" <utf8_string>
        "geoname_id": 
          6255148 <uint32>
        "names": 
          {
            "de": 
              "Europa" <utf8_string>
            "en": 
              "Europe" <utf8_string>
            "es": 
              "Europa" <utf8_string>
            "fr": 
              "Europe" <utf8_string>
            "ja": 
              "ヨーロッパ" <utf8_string>
            "pt-BR": 
              "Europa" <utf8_string>
            "ru": 
              "Европа" <utf8_string>
            "zh-CN": 
              "欧洲" <utf8_string>
          }
      }
    "country": 
      {
        "geoname_id": 
          690791 <uint32>
        "iso_code": 
          "UA" <utf8_string>
        "names": 
          {
            "de": 
              "Ukraine" <utf8_string>
            "en": 
              "Ukraine" <utf8_string>
            "es": 
              "Ucrania" <utf8_string>
            "fr": 
              "Ukraine" <utf8_string>
            "ja": 
              "ウクライナ共和国" <utf8_string>
            "pt-BR": 
              "Ucrânia" <utf8_string>
            "ru": 
              "Украина" <utf8_string>
            "zh-CN": 
              "乌克兰" <utf8_string>
          }
      }
    "location": 
      {
        "accuracy_radius": 
          50 <uint16>
        "latitude": 
          47.850000 <double>
        "longitude": 
          35.283300 <double>
        "time_zone": 
          "Europe/Zaporozhye" <utf8_string>
      }
    "postal": 
      {
        "code": 
          "70459" <utf8_string>
      }
    "registered_country": 
      {
        "geoname_id": 
          690791 <uint32>
        "iso_code": 
          "UA" <utf8_string>
        "names": 
          {
            "de": 
              "Ukraine" <utf8_string>
            "en": 
              "Ukraine" <utf8_string>
            "es": 
              "Ucrania" <utf8_string>
            "fr": 
              "Ukraine" <utf8_string>
            "ja": 
              "ウクライナ共和国" <utf8_string>
            "pt-BR": 
              "Ucrânia" <utf8_string>
            "ru": 
              "Украина" <utf8_string>
            "zh-CN": 
              "乌克兰" <utf8_string>
          }
      }
    "subdivisions": 
      [
        {
          "geoname_id": 
            687699 <uint32>
          "iso_code": 
            "23" <utf8_string>
          "names": 
            {
              "de": 
                "Saporischschja" <utf8_string>
              "en": 
                "Zaporizhia" <utf8_string>
              "fr": 
                "Oblast de Zaporijia" <utf8_string>
              "ru": 
                "Запорожская область" <utf8_string>
            }
        }
      ]
  }

[anjia0532]

u can use ab/wrk/jmeter to test this lib when u delete maxm.MMDB_free_entry_data_list(entry_data_list[0])

use htop or top to monitoring memory.

ref https://github.com/anjia0532/lua-resty-maxminddb/issues/6#issuecomment-402084494

[anjia0532]

@atomyuk

cat server.conf
server {
    listen 80;
    server_name 127.0.0.1;
    location / {
        content_by_lua_block{
            local cjson = require 'cjson'
            local geo = require 'resty.maxminddb'
            if not geo.initted() then
                geo.init("/path/to/GeoLite2-City.mmdb")
            end
            local res,err = geo.lookup(ngx.var.arg_ip or ngx.var.remote_addr) --support ipv6 e.g. 2001:4860:0:1001::3004:ef68

            ngx.say("full :",cjson.encode(res))
            if ngx.var.arg_node then
               ngx.say("node name:",ngx.var.arg_node," ,value:", cjson.encode(res[ngx.var.arg_node] or {}))
            end
        }
    }
}

cat test.lua
wrk.method = "GET";
wrk.body = "";

logfile = io.open("wrk.log", "w");

request = function()
ip = tostring(math.random(1, 255)).."."..tostring(math.random(1, 255)).."."..tostring(math.random(1, 255)).."."..tostring(math.random(1, 255))
path = "/?ip=" .. ip
return wrk.format(nil, path)
end

response = function(status,header,body)
logfile:write("\nbody:" .. body .. "\n-----------------");
end

./wrk -t50 -c200 -d120s -s ./test.lua --latency http://127.0.0.1

delete maxm.MMDB_free_entry_data_list(entry_data_list[0]))

Running 2m test @ http://127.0.0.1
  50 threads and 200 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   136.61ms  218.78ms   2.00s    90.66%
    Req/Sec    62.49     39.18   690.00     65.25%
  Latency Distribution
     50%   55.15ms
     75%  126.28ms
     90%  331.55ms
     99%    1.16s 
  306033 requests in 2.04m, 302.93MB read
  Socket errors: connect 0, read 0, write 0, timeout 142
Requests/sec:   2495.67
Transfer/sec:      2.47MB

keep maxm.MMDB_free_entry_data_list(entry_data_list[0]))

Running 2m test @ http://127.0.0.1
  50 threads and 200 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    27.56ms   54.77ms   1.99s    97.31%
    Req/Sec   181.39     64.33     3.34k    70.94%
  Latency Distribution
     50%   18.64ms
     75%   26.42ms
     90%   46.20ms
     99%  121.75ms
  1059526 requests in 2.00m, 1.02GB read
  Socket errors: connect 0, read 12, write 0, timeout 48
Requests/sec:   8821.83
Transfer/sec:      8.67MB

[jtammen]

@anjia0532 Hey, I recently started using your library (thanks for your work on that, by the way!) and I am also hitting this problem in my environment.

I can confirm that the segfault does go away when I comment out the line maxm.MMDB_free_entry_data_list(entry_data_list[0])

In https://github.com/maxmind/libmaxminddb/issues/174, @oschwald suggests that this could happen while trying to free an uninitialized pointer.

[anjia0532]

@jtammen paste your code plz. like https://github.com/anjia0532/lua-resty-maxminddb/issues/9#issuecomment-404419219

[anjia0532]

$ dpkg -l | grep maxmind
ii  geoipupdate                          2.5.0-0+maxmind1~xenial                    amd64        Tool for updating GeoIP and GeoLite databases
ii  libmaxminddb-dev:amd64               1.3.2-0+maxmind1~xenial                    amd64        IP geolocation database library (development headers)
ii  libmaxminddb0:amd64                  1.3.2-0+maxmind1~xenial                    amd64        IP geolocation database library
ii  mmdb-bin                             1.3.2-0+maxmind1~xenial                    amd64        IP geolocation lookup command-line tool

$ geoipupdate
// upgrade to latest(Geo-City.mmdb)

$ cat /etc/os-release
NAME="Ubuntu"
VERSION="16.04.4 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.4 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial

$ uname -a
Linux openresty-server 4.4.0-127-generic #153-Ubuntu SMP Sat May 19 10:58:46 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

$ opm get anjia0532/lua-resty-maxminddb (install)

$ opm upgrade anjia0532/lua-resty-maxminddb (upgrade to latest version)

$ cat /path/to/resty/maxminddb.lua | grep _VERSION
_M._VERSION = '1.0.0'

nginx's config

  local cjson = require 'cjson'
  local geo = require 'resty.maxminddb'
  if not geo.initted() then
      geo.init("/path/to/GeoLite2-City.mmdb")
  end
  local res,err = geo.lookup(ngx.var.arg_ip or ngx.var.remote_addr) --support ipv6 e.g. 2001:4860:0:1001::3004:ef68

  if not res then
      ngx.log(ngx.ERR,'failed to lookup by ip ,reason:',err)
  end

  ngx.say("full :",cjson.encode(res))
  if ngx.var.arg_node then
     ngx.say("node name:",ngx.var.arg_node," ,value:", cjson.encode(res[ngx.var.arg_node] or {}))
  end

#ipv4
  $ curl localhost/ip=114.114.114.114&node=city
  #ipv6
  #$ curl localhost/ip=2001:4860:0:1001::3004:ef68&node=country
  full :{"city":{"geoname_id":1799962,"names":{"en":"Nanjing","ru":"Нанкин","fr":"Nankin","pt-BR":"Nanquim","zh-CN":"南京","es":"Nankín","de":"Nanjing","ja":"南京市"}},"subdivisions":[{"geoname_id":1806260,"names":{"en":"Jiangsu","fr":"Province de Jiangsu","zh-CN":"江苏省"},"iso_code":"32"}],"country":{"geoname_id":1814991,"names":{"en":"China","ru":"Китай","fr":"Chine","pt-BR":"China","zh-CN":"中国","es":"China","de":"China","ja":"中国"},"iso_code":"CN"},"registered_country":{"geoname_id":1814991,"names":{"en":"China","ru":"Китай","fr":"Chine","pt-BR":"China","zh-CN":"中国","es":"China","de":"China","ja":"中国"},"iso_code":"CN"},"location":{"time_zone":"Asia\/Shanghai","longitude":118.7778,"accuracy_radius":50,"latitude":32.0617},"continent":{"geoname_id":6255147,"names":{"en":"Asia","ru":"Азия","fr":"Asie","pt-BR":"Ásia","zh-CN":"亚洲","es":"Asia","de":"Asien","ja":"アジア"},"code":"AS"}}
  node name:city ,value:{"geoname_id":1799962,"names":{"en":"Nanjing","ru":"Нанкин","fr":"Nankin","pt-BR":"Nanquim","zh-CN":"南京","es":"Nankín","de":"Nanjing","ja":"南京市"}}

[jtammen]

@anjia0532

Server setup

$ dpkg -l | grep maxmind
ii  geoipupdate                      3.1.1-0+maxmind1~xenial                    amd64        Tool for updating GeoIP and GeoLite databases
ii  libmaxminddb-dev:amd64           1.3.2-0+maxmind1~xenial                    amd64        IP geolocation database library (development headers)
ii  libmaxminddb0:amd64              1.3.2-0+maxmind1~xenial                    amd64        IP geolocation database library
ii  mmdb-bin                         1.3.2-0+maxmind1~xenial                    amd64        IP geolocation lookup command-line tool

$ geoipupdate

$ ls -la /usr/share/GeoIP/
drwxr-xr-x   2 root root     4096 Dec  5 11:09 .
drwxr-xr-x 124 root root     4096 Dec  4 14:33 ..
-rw-------   1 root root        0 Dec  4 13:44 .geoipupdate.lock
-rw-r--r--   1 root root  1160739 Apr  9  2016 GeoIP.dat
-rw-r--r--   1 root root  4391541 Apr  9  2016 GeoIPv6.dat
-rw-r--r--   1 root root 60907419 Dec  5 11:09 GeoLite2-City.mmdb

$ cat /etc/os-release
NAME="Ubuntu"
VERSION="16.04.5 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.5 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial

$ uname -a
Linux XYZ 4.4.0-140-generic #166-Ubuntu SMP Wed Nov 14 20:09:47 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

$ opm get anjia0532/lua-resty-maxminddb
Package lua-resty-maxminddb-1.0.0 already installed.

$ opm upgrade anjia0532/lua-resty-maxminddb
* Fetching anjia0532/lua-resty-maxminddb > 1.0.0
Package anjia0532/lua-resty-maxminddb 1.0.0 is already the latest version.

$ ls -la /usr/local/openresty/site/lualib/resty/
drwxr-xr-x 2 root root  4096 Dec  4 16:47 .
drwxr-xr-x 3 root root  4096 Dec  4 13:44 ..
-rw-r--r-- 1 root root 10597 Dec  4 16:12 maxminddb.lua
-rw-r--r-- 1 root root  9232 Dec  4 13:44 redis.lua

$ cat /usr/local/openresty/site/lualib/resty/maxminddb.lua | grep _VERSION
_M._VERSION = '1.0.0'

nginx config

server {
  listen 80;
  server_name localhost;

  location / {
    content_by_lua_block {
      local cjson = require 'cjson'
      local geo = require 'resty.maxminddb'

      if not geo.initted() then
      geo.init("/usr/share/GeoIP/GeoLite2-City.mmdb")
      end

      local res,err = geo.lookup(ngx.var.arg_ip or ngx.var.remote_addr) --support ipv6 e.g. 2001:4860:0:1001::3004:ef68

      if not res then
        ngx.log(ngx.ERR,'failed to lookup by ip ,reason:',err)
      end

      ngx.say("full :",cjson.encode(res))
      if ngx.var.arg_node then
        ngx.say("node name:",ngx.var.arg_node," ,value:", cjson.encode(res[ngx.var.arg_node] or {}))
      end
    }
  }
}

$ curl localhost?ip=114.114.114.114\&node=city
full :{"country":{"geoname_id":1814991,"names":{"en":"China","ru":"Китай","fr":"Chine","pt-BR":"China","zh-CN":"中国","es":"China","de":"China","ja":"中国"},"iso_code":"CN"},"registered_country":{"geoname_id":1814991,"names":{"en":"China","ru":"Китай","fr":"Chine","pt-BR":"China","zh-CN":"中国","es":"China","de":"China","ja":"中国"},"iso_code":"CN"},"continent":{"geoname_id":6255147,"names":{"en":"Asia","ru":"Азия","fr":"Asie","pt-BR":"Ásia","zh-CN":"亚洲","es":"Asia","de":"Asien","ja":"アジア"},"code":"AS"},"location":{"longitude":113.7266,"accuracy_radius":50,"latitude":34.7725}}
node name:city ,value:{}

$ localhost?ip=217.229.23.250\&ode=city
full :{"subdivisions":[{"geoname_id":2911297,"names":{"en":"Hamburg","fr":"Hambourg","de":"Hamburg","es":"Hamburgo"},"iso_code":"HH"}],"city":{"geoname_id":2911298,"names":{"en":"Hamburg","ru":"Гамбург","fr":"Hambourg","pt-BR":"Hamburgo","zh-CN":"汉堡市","es":"Hamburgo","de":"Hamburg","ja":"ハンブルク"}},"registered_country":{"geoname_id":2921044,"names":{"en":"Germany","ru":"Германия","fr":"Allemagne","pt-BR":"Alemanha","zh-CN":"德国","es":"Alemania","de":"Deutschland","ja":"ドイツ連邦共和国"},"iso_code":"DE","is_in_european_union":false},"country":{"geoname_id":2921044,"names":{"en":"Germany","ru":"Германия","fr":"Allemagne","pt-BR":"Alemanha","zh-CN":"德国","es":"Alemania","de":"Deutschland","ja":"ドイツ連邦共和国"},"iso_code":"DE","is_in_european_union":false},"continent":{"geoname_id":6255148,"names":{"en":"Europe","ru":"Европа","fr":"Europe","pt-BR":"Europa","zh-CN":"欧洲","es":"Europa","de":"Europa","ja":"ヨーロッパ"},"code":"EU"},"location":{"time_zone":"Europe\/Berlin","longitude":10.1669,"accuracy_radius":5,"latitude":53.6063},"postal":{"code":"22143"}}
node name:city ,value:{"geoname_id":2911298,"names":{"en":"Hamburg","ru":"Гамбург","fr":"Hambourg","pt-BR":"Hamburgo","zh-CN":"汉堡市","es":"Hamburgo","de":"Hamburg","ja":"ハンブルク"}}

So in this sample configuration it actually works as expected.

My real nginx configuration is a little different, though:

map $geoip_continent_code $is_eu {
  default false;
  "EU" true;
}

server {
  listen 80;
  server_name localhost;

  set $geoip_continent_code '';
  set_by_lua_block $geoip_continent_code {
    local geo = require "resty.maxminddb"

    if not geo.initted() then
      geo.init("/usr/share/GeoIP/GeoLite2-City.mmdb")
    end
    local res, err = geo.lookup(ngx.var.arg_ip or ngx.var.remote_addr)

    if not res then
      ngx.log(ngx.NOTICE, "Failed to lookup by IP, reason: ", err)
      return ""
    end

    local continent = res.continent
    if not continent then
      ngx.log(ngx.NOTICE, "Result does not contain continent node")
      return ""
    end

    return res.continent.code
  }
  return 200 $geoip_continent_code;
}

$ curl localhost?ip=141.141.141.141\&node=city
NA

$ curl localhost?ip=217.229.23.250\&node=city
curl: (52) Empty reply from server

In nginx error.log: 2018/12/05 11:43:28 [alert] 31146#31146: worker process 31856 exited on signal 11 (core dumped)

In /var/log/syslog: Dec 5 11:43:28 XYZ kernel: [80124.746363] nginx[31856]: segfault at 5b ip 00007fcdaf66ce15 sp 00007fffc0146308 error 4 in libmaxminddb.so.0.0.7[7fcdaf66a000+5000]

After commenting out the line maxm.MMDB_free_entry_data_list(entry_data_list[0]) in maxminddb.lua:

$ curl localhost?ip=141.141.141.141\&node=city
NA

$ curl localhost?ip=217.229.23.250\&node=city
EU

So it might be related to the structure of the returned data from the database actually?!

[anjia0532]

$ find / -name maxminddb.lua
/etc/openresty/site/lualib/resty/maxminddb.lua
// make sure only one maxminddb.lua lib in your server.

$ cat /etc/openresty/site/lualib/resty/maxminddb.lua | grep MMDB_free_entry_data_list
void MMDB_free_entry_data_list(MMDB_entry_data_list_s *const entry_data_list);
  maxm.MMDB_free_entry_data_list(entry_data_list[0])
// not comment

$ curl localhost?ip=217.229.23.250\&node=city
full :{"subdivisions":[{"geoname_id":2911297,"names":{"en":"Hamburg","fr":"Hambourg","de":"Hamburg","es":"Hamburgo"},"iso_code":"HH"}],"city":{"geoname_id":2911298,"names":{"en":"Hamburg","ru":"Гамбург","fr":"Hambourg","pt-BR":"Hamburgo","zh-CN":"汉堡市","es":"Hamburgo","de":"Hamburg","ja":"ハンブルク"}},"registered_country":{"geoname_id":2921044,"names":{"en":"Germany","ru":"Германия","fr":"Allemagne","pt-BR":"Alemanha","zh-CN":"德国","es":"Alemania","de":"Deutschland","ja":"ドイツ連邦共和国"},"iso_code":"DE","is_in_european_union":false},"country":{"geoname_id":2921044,"names":{"en":"Germany","ru":"Германия","fr":"Allemagne","pt-BR":"Alemanha","zh-CN":"德国","es":"Alemania","de":"Deutschland","ja":"ドイツ連邦共和国"},"iso_code":"DE","is_in_european_union":false},"continent":{"geoname_id":6255148,"names":{"en":"Europe","ru":"Европа","fr":"Europe","pt-BR":"Europa","zh-CN":"欧洲","es":"Europa","de":"Europa","ja":"ヨーロッパ"},"code":"EU"},"location":{"time_zone":"Europe\/Berlin","longitude":10.1669,"accuracy_radius":5,"latitude":53.6063},"postal":{"code":"22143"}}
node name:city ,value:{"geoname_id":2911298,"names":{"en":"Hamburg","ru":"Гамбург","fr":"Hambourg","pt-BR":"Hamburgo","zh-CN":"汉堡市","es":"Hamburgo","de":"Hamburg","ja":"ハンブルク"}}

it's work.

[jtammen]

@anjia0532 Yes, it works with the sample nginx config (first one in my comment) but not my "real" config, i.e. using set_by_lua_block … maybe the problem is related to that. Did you try with my second nginx config example?

[anjia0532]

yeap,works too.

[oschwald]

I am not an expert with Lua, but this code appears to be ignoring the value in mmdb_error:

https://github.com/anjia0532/lua-resty-maxminddb/blob/702629d3fe43897de0bb0989dcbdeb298f70944b/lib/resty/maxminddb.lua#L328-L331

If there is an error, you should not try it iterate over the entry_data_list or call MMDB_free_entry_data_list on it. I suspect this is why some people are seeing the segfault.

[anjia0532]

@oschwald thanks for your reply,I am not an expert with c++ too 😅. make changes according to your suggestion (https://github.com/anjia0532/lua-resty-maxminddb/commit/5e4295726343d2c6bc8876de1c7948705ae110bd)

@jtammen running opm upgrade anjia0532/lua-resty-maxminddb and try again.

[jtammen]

@anjia0532 @oschwald Thanks for your effort … I did the upgrade, but unfortunately, that did not change anything for me, still segfaulting :-/

It still works for me if I change this line in my Lua script:

local res, err = geo.lookup(ngx.var.remote_addr)

to this using the sample IP:

local res, err = geo.lookup("141.141.141.141")

Or commenting out the line

maxm.MMDB_free_entry_data_list(entry_data_list[0])

So I still think it has to do with the actual data for the specific IP address.

As I am neither an expert in Lua nor C++, I fear there is nothing more that I could contribute. So I am looking at some other solutions for my requirements now.

Thanks again for your effort!

Edit: I just noticed sth (even more) strange: When I get a segfault and then do a completely unrelated changed in the nginx.conf, restart nginx and then reload the page, it works! If I then do another change (again, unrelated to that specific server) and restart, I get the segfault again… :-/

[anjia0532]

1. download maxminddb.lua(beta) and overwrite /usr/local/openresty/site/lualib/resty/maxminddb.lua.

2. service openresty reload

3. try again

4. u should be use wrk(wrk test) or apache ab to benchmarking

ps:

1. declare MMDB_close function
2. modify init function
3. reopen mmaxdb everytime
4. close before return

[jtammen]

@anjia0532 Thanks, this seems to work, at least the segfaults are gone now! Didn't have the time yet to do some benchmarking yet, so I don't know if this could be used in production environment :)

[anjia0532]

use your nginx config, 1.0.2 vs 1.0.1 benchmarking

v1.0.2(beta)

v1.0.1

31% qps

[anjia0532]

@jtammen fixed,upgrade to v1.2.0

thanks for @spacewander (PR https://github.com/anjia0532/lua-resty-maxminddb/pull/14)