pcall problem when calling require

[RocFang]

Hello!

Tipically, for versions compatibility consideration, we could use pcall to catch exception of require.

But I have found a strange luajit problem when using pcall and require, especially when ffi involves.

Minimum reproduction configuation:

1. conf/nginx.conf

events {}

http{
    lua_package_path  ';;$prefix/lua/?.lua';
    init_by_lua_block {
        local has_a, a
        has_a, a= pcall(require, "a")
        if not has_a then
            ngx.log(ngx.ERR, "failed to load a: ", a)
        end

        has_a, a = pcall(require, "a")
        if not has_a then
            ngx.log(ngx.ERR, "failed to load a again: ", a)
        else
            ngx.log(ngx.ERR, "type(a)=", type(a), " a=", a)
        end
    }
}

2. lua/a.lua

local ffi = require("ffi")

local _M = {}

ffi.cdef[[
int xxxx();
]]

local function get_caches_array()
    return tonumber(ffi.C.xxxx())
end

local a = get_caches_array()

_M.a = a
_M.get_caches_array = get_caches_array

return _M

Expectation:

Since a.lua contains a non defined xxxx ffi function, I expect everytime we call pcall(require, "a"), a false and an error message retuns.

behavior of diffirent version

openresty-1.13.6.2 meets my expectation, output:

[root@localhost pcall_error]# /usr/local/openresty-1.13.6.2/nginx/sbin/nginx  -p .
nginx: [error] [lua] init_by_lua:5: failed to load a: .//lua/a.lua:10: /usr/local/openresty-1.13.6.2/luajit/lib/libluajit-5.1.so.2: undefined symbol: xxxx
nginx: [error] [lua] init_by_lua:10: failed to load a again: loop or previous error loading module 'a'

From openresty-1.15.8.1, things becom strange, and have diffirent result in diffirent version.

openresty-1.15.8.1:

nginx: [error] [lua] init_by_lua:5: failed to load a: .//lua/a.lua:10: /usr/local/openresty-1.15.8.1/luajit/lib/libluajit-5.1.so.2: undefined symbol: xxxx
nginx: [error] [lua] init_by_lua:12: type(a)=userdata a=null

openresty-1.15.8.3:

nginx: [error] [lua] init_by_lua:5: failed to load a: .//lua/a.lua:10: /usr/local/openresty-1.15.8.3/luajit/lib/libluajit-5.1.so.2: undefined symbol: xxxx
nginx: [error] [lua] init_by_lua:12: type(a)=userdata a=null

openresty-1.19.9.1:

nginx: [error] [lua] init_by_lua:5: failed to load a: .//lua/a.lua:10: /usr/local/openresty/luajit/lib/libluajit-5.1.so.2: undefined symbol: xxxx
nginx: [error] [lua] init_by_lua:12: type(a)=number a=-5.6817549271743e-322

I'm not familiar with luajit internals, so have this feedback.

Thanks.

[zhuizhuhaomeng]

try with the 1.21.4.1rc2，

nginx: [error] [lua] init_by_lua:5: failed to load a: error loading module 'a' from file '/usr/local/openresty/nginx//lua/a.lua':
    /usr/local/openresty/nginx//lua/a.lua:1: '=' expected near 'ffi'
nginx: [error] [lua] init_by_lua:10: failed to load a again: error loading module 'a' from file '/usr/local/openresty/nginx//lua/a.lua':
    /usr/local/openresty/nginx//lua/a.lua:1: '=' expected near 'ffi'

[RocFang]

@zhuizhuhaomeng more interesting...

[zhuizhuhaomeng]

I think it is resolved, but I don't will PR resolve this issue.

[RocFang]

@zhuizhuhaomeng ha.. I have tried 1.21.4.1rc2 righ now, but the log if diffirent with yours:)

[root@localhost pcall_error]# /usr/local/openresty-1.21.4.1rc2/nginx/sbin/nginx  -v
nginx version: openresty/1.21.4.1rc2

[root@localhost pcall_error]# /usr/local/openresty-1.21.4.1rc2/nginx/sbin/nginx  -p .
nginx: [error] [lua] init_by_lua:5: failed to load a: .//lua/a.lua:10: /usr/local/openresty-1.21.4.1rc2/luajit/lib/libluajit-5.1.so.2: undefined symbol: xxxx
nginx: [error] [lua] init_by_lua:12: type(a)=number a=-5.6817549271743e-322

Do you mean it's resolved in master?

[zhuizhuhaomeng]

sorry, When I copy the Lua code and paste it into the file, "local" becomes "cal". The bug still exists.

[RocFang]

got it:)

[zhuizhuhaomeng]

run with luajit directly, the error is below. This may be the right error.

[root@localhost openresty]# ./luajit/bin/luajit t.lua 
failed to load a:   ./a.lua:10: ./luajit/bin/luajit: undefined symbol: xxxx
failed to load a again:     loop or previous error loading module 'a'

replace the ngx.print with print

[zhuizhuhaomeng]

@XmiliaH would you please have a look at this?

[zhuizhuhaomeng]

I don't why the second pcall(require, "a") do not enter into the function lj_cf_package_require.

[zhuizhuhaomeng]

I am sure it is a bug of the openresty. but I don't know how to debug.

[XmiliaH]

Yes. It is an openresty problem as it replaces require with a version that does not check for KEY_SENTINEL.

[zhuizhuhaomeng]

Thank you. @XmiliaH Would you please share the debug process?

[XmiliaH]

First I looked at the lj_cf_package_require and found KEY_SENTINEL. I the compared this value to the reported return value of -5.6817549271743e-322 which was the case. I then checked if there is a way that lj_cf_package_require might return that value but found none. I then reduced the test to:

package.loaded.abc = -5.6817549271743e-322
local res = require("abc")
print(res)

which showed that there is no problem with pcall. Instead it seems that the require function is replaced. To find the location of the replacement function I then used:

print(debug.getinfo(require, 'S').source)

to find that the require function is replaced in resty/core/base.lua,