Feature Request: allow linking to dynamic lua libs

[Trey2k]

Is your feature request related to a problem? Please describe. Currently bind_libraries only allows linking the lua state to the built in libraries.

Describe the solution you'd like bind_libraries, should be changed to work with both built in libraries and external dynamic libraries

Describe alternatives you've considered Alternatively we could use 2 separate methods.

Additional context This will be useful for using lua tooling such as luarocks.

[gilzoide]

Maybe you could just patch package.path/package.cpath/package.searchers if necessary and instruct people to call require from Lua instead of building a custom method in LuaAPI for supporting external libraries.

[Trey2k]

Maybe you could just patch package.path/package.cpath/package.searchers if necessary and instruct people to call require from Lua instead of building a custom method in LuaAPI for supporting external libraries.

This sounds more ideal then the proposal above. I have not used luarocks really at all so I would need to do some research.

One thing to note is sine we are using the Lua C api the require method is not present. People of course can implement it them selfs. I'm not sure if that would work for loading dynamic libs without a specific method. But as mentioned this is a topic I've not looked into much.

[gilzoide]

So, one could already tweak package.path/package.cpath and require stuff right now with godot_luaAPI. Something in the lines of (disclaimer, I haven't tested this in godot_luaAPI, but have used Lua a lot, so I guess it would work):

# `root_path` points to project path in editor or executable path in builds
var root_path = OS.get_executable_path().get_base_dir()
    if OS.has_feature("standalone")
    else ProjectSettings.globalize_path("res://")
root_path = root_path.trim_suffix("/")

var lua = LuaAPI.new()
# require and package table come from the package library
lua.bind_libraries(["package"])
lua.push_variant("root_path", root_path)
lua.do_string("""
-- Loading modules written in Lua use these path templates
package.path = root_path .. '/?.lua;' .. root_path .. '/?/init.lua'
-- Loading native modules use these path templates
package.cpath = root_path .. '/?.so;' .. root_path .. '/loadall.so'
-- myLuaLibrary may be a module written in either Lua or a DLL
myLuaLibrary = require('myLuaLibrary')
""")

In this case I'm patching the paths to point to a path relative to the project or build, but one could leave the default values and use paths that search for modules in the system in the default locations, or change it to anything else.

One thing to note is since we are using the Lua C api the require method is not present.

People would call require from Lua code, so I think that won't be necessary.

I'm not sure if that would work for loading dynamic libs without a specific method.

I'm not sure what you mean by that. But when loading DLL modules with require, Lua will search for a function to be exported with the symbol luaopen_myLuaLibrary in all the package.cpath templates, substituting ? by the library name. You can check out the documentation on package.searchers for all these rules.

My point is that "load dynamic Lua libs" is a built-in feature from Lua, so I think you don't have to worry too much about it, just let people use Lua's default mechanism for that.

[Trey2k]

One thing to note is since we are using the Lua C api the require method is not present.

People would call require from Lua code, so I think that won't be necessary.

What I meant is AFAIK the require method is not present within lua its self by default what executing it via the lua C api. Allowing anyone who is using lua to load a dll at will bypasses the sandbox so that's something that needs to be left up to the user. As it stands the user could define their own require method, but they only have a way to load lua code, but not the dynamic libs.

[Trey2k]

One thing to note is since we are using the Lua C api the require method is not present.

People would call require from Lua code, so I think that won't be necessary.

What I meant is AFAIK the require method is not present within lua its self by default what executing it via the lua C api. Allowing anyone who is using lua to load a dll at will bypasses the sandbox so that's something that needs to be left up to the user. As it stands the user could define their own require method, but they only have a way to load lua code, but not the dynamic libs.

I think I'm actually incorrect here now. Require may be part of the base lib.

[gilzoide]

the require method is not present within lua its self by default what executing it via the lua C api.

That's right, require is defined when you open the package library with luaopen_package or luaL_openlibs. Edit: reference docs

Allowing anyone who is using lua to load a dll at will bypasses the sandbox so that's something that needs to be left up to the user.

Agreed. The user could bind the package library and require stuff if they think they should. Ultimately, users build their own sandbox and they most certainly can do that from Lua itself. So the game's code can use require for whatever reason, but when loading external untrusted code from end users, they do so within a custom sandbox using load/loadfile environment parameter that may not have the default require of have a modified version of it.

[Trey2k]

the require method is not present within lua its self by default what executing it via the lua C api.

That's right, require is defined when you open the package library with luaopen_package or luaL_openlibs.

Allowing anyone who is using lua to load a dll at will bypasses the sandbox so that's something that needs to be left up to the user.

Agreed. The user could bind the package library and require stuff if they think they should. Ultimately, users build their own sandbox and they most certainly can do that from Lua itself. So the game's code can use require for whatever reason, but when loading external untrusted code from end users, they do so within a custom sandbox using load/loadfile environment parameter that may not have the default require of have a modified version of it, absolutely.

Yeah this makes sense, I apriciate you taking your time here. Working on this module has been my only experiance with Lua beyond being a end user. So I still have a lot of stuff to learn. But yeah it does seems this is uneeded in that case then so I will close the issue for now. Thanks!

[gilzoide]

No problem, glad I could help ^^

So I still have a lot of stuff to learn

Don't worry, we all have. As far as Lua is concerned, though, I'd say that the Lua/C API and this package stuff are 2 of the most complex subjects. That and sandboxing. Maybe coroutines as well. (oh my, that was 4 things already... =P)

Either way, you're making a very good job with godot_luaAPI, it's quite simple and very effective! Keep it up 🙌