Cannot load DLLs in v0.19

[AgentRev]

I use MSYS2 UCRT64 on Win 10, and I rely on MoonFLTK as part of a package I am working with. MoonFLTK gets compiled into a DLL, and is loaded via fl = require("moonfltk") in my script. With rlua v0.18.0, and system-lua54, everything works fine. This is what I have in my Cargo.toml:

[dependencies]
rlua = { version = "0.18.0", default-features = false, features = ["system-lua54"] }

However, I changed the version to v0.19.4, and it doesn't work anymore:

[dependencies]
rlua = { version = "0.19.4", default-features = false, features = ["system-lua54"] }

I now get the following error when I try to execute the script via rlua:

Lua error loading file: runtime error: [string "?"]:9:
module 'moonfltk' not found:
        no field package.preload['moonfltk']
        no file 'C:\msys64\home\AgentRev\.cargo\bin\lua\moonfltk.lua'
        no file 'C:\msys64\home\AgentRev\.cargo\bin\lua\moonfltk\init.lua'
        no file 'C:\msys64\home\AgentRev\.cargo\bin\moonfltk.lua'
        no file 'C:\msys64\home\AgentRev\.cargo\bin\moonfltk\init.lua'
        no file 'C:\msys64\home\AgentRev\.cargo\bin\..\share\lua\5.4\moonfltk.lua'
        no file 'C:\msys64\home\AgentRev\.cargo\bin\..\share\lua\5.4\moonfltk\init.lua'
        no file '.\moonfltk.lua'
        no file '.\moonfltk\init.lua'
    stack traceback:
        [C]: in ?
        [C]: in function 'require'
        [string "?"]:9: in main chunk

Looking at the paths, it's clear that v0.19 somehow altered how Lua resolves library files. This happens whether I use system-lua54 or builtin-lua54. With rlua v0.18.0, if I change the require to something random like moonfltkzzz, I get this message instead :

Lua error loading file: runtime error: [string "?"]:9:
module 'moonfltkzzz' not found:
        no field package.preload['moonfltkzzz']
        no file 'C:\msys64\home\AgentRev\.cargo\bin\..\share\lua\5.4\moonfltkzzz.lua'
        no file 'C:\msys64\home\AgentRev\.cargo\bin\..\share\lua\5.4\moonfltkzzz\init.lua'
        no file 'C:\msys64\home\AgentRev\.cargo\bin\..\lib\lua\5.4\moonfltkzzz.lua'
        no file 'C:\msys64\home\AgentRev\.cargo\bin\..\lib\lua\5.4\moonfltkzzz\init.lua'
        no file 'C:\msys64\home\AgentRev\.cargo\bin\..\share\lua\5.4\\moonfltkzzz.lua'
        no file 'C:\msys64\home\AgentRev\.cargo\bin\..\share\lua\5.4\\moonfltkzzz\init.lua'
        no file '.\moonfltkzzz.lua'
        no file '.\moonfltkzzz\init.lua'
        no file 'C:\msys64\home\AgentRev\.cargo\bin\..\lib\lua\5.4\moonfltkzzz.dll'
        no file 'C:\msys64\home\AgentRev\.cargo\bin\..\lib\lua\5.4\..\lib\lua\5.4\\moonfltkzzz.dll'
        no file 'C:\msys64\home\AgentRev\.cargo\bin\..\lib\lua\5.4\loadall.dll'
        no file '.\moonfltkzzz.dll'
    stack traceback:
        [C]: in ?
        [C]: in function 'require'
        [string "?"]:9: in main chunk

The problem is present in all 0.19.x versions. The system interpreter was left unchanged during all tests.

If I try v0.18.0 with builtin-lua54, I get a segfault when running the script, which is why I originally picked the system interpreter.

[jugglerchris]

Hi, This is a feature, but I think it needs to be more obvious in the README/documentation, sorry for that!

In 0.19.x, by default, loading DLLs is disabled as it's a safety issue in the Rust sense. There's an unsafe constructor which lets you specify that loading DLLs is ok. Instead of:

Lua::new()  // initialize with relatively safe defaults

use:

Lua::unsafe_new_with_flags(
    StdLib::ALL_NO_DEBUG,  // Load all Lua standard libraries apart from debug
   InitFlags::DEFAULT - InitFlags::REMOVE_LOADLIB,  // Allow loading binary libraries
)

Other default flags are PCALL_WRAPPERS (prevent Lua scripts from catching Rust panics) and LOAD_WRAPPERS (prevent loading Lua bytecode).

I will leave this issue open until the documentation has been improved.

As to the segfault, that's concerning. When you say "running the script", does that mean something that uses moonfltk? Perhaps the builtin Lua is compiled with some flags different from the system one which make it not binary compatible with it somehow.

[AgentRev]

I replaced all instances of Lua::new with Lua::unsafe_new_with_flags as you suggested, and I can now successfully use v0.19.4. Thanks. With builtin-lua54, everything works, however if I uninstall my system Lua, I get this error:

Lua error loading file: runtime error: error loading mo
dule 'moonfltk' from file '.\moonfltk.dll':
        The specified module could not be found.

    stack traceback:
        [C]: in ?
        [C]: in ?
        [C]: in function 'require'
        [string "?"]:4: in main chunk

Everything works fine again if I reinstall Lua. So, it seems moonfltk doesn't want to use the builtin Lua for some reason. Not that big of a deal, tho.

The segfault only happened with builtin v0.18 and below, I have not seen it so far with v0.19.

[jugglerchris]

Oh. That sounds to me like moonfltk.dll is linked to a lua.dll - so even if you're using the builtin Lua it'll still link to that. The builtin one should be statically linked with the executable using rlua, so it could be that in this case you end up with two Lua interpreters linked in, which sounds like it could be related to your segfaults on 0.18.

In principle I presume it'd be possible for the builtin Lua to be turned into a DLL instead of a static library, which might work better in this case.

[AgentRev]

The simplest option is probably for me to stick with system-lua54 then, and I will provide an offline package for Lua 5.4 to my very few users, who use MSYS2 solely for our app. MSYS2's package manager only provides the latest version, so this will ensure the app won't stop working when Lua 5.5 is eventually released.

[jugglerchris]

I think I've since found out a way to use the builtin Lua - the executable needs to export symbols. See https://github.com/amethyst/rlua/issues/273#issuecomment-1595731130 .

[jugglerchris]

As the changes needed are in applications, I think this issue can be closed.