FreeBSD runtime error

[hurzl]

I was able to build and install on FreeBSD (14.1), but get following error:

$ RUST_BACKTRACE=full sile
thread 'main' panicked at src/lib.rs:63:42:
called `Result::unwrap()` on an `Err` value: RuntimeError("error loading module 'lua-utf8' from file '/usr/local/lib/lua/5.4/lua-utf8.so':\n\t/usr/local/lib/lua/5.4/lua-utf8.so: Undefined symbol \"luaL_checkversion_\"\nstack traceback:\n\t[C]: in ?\n\t[C]: in ?\n\t[C]: in function 'require'\n\t./core/globals.lua:16: in main chunk\n\t[C]: in function 'require'\n\t./core/sile.lua:8: in main chunk\n\t[C]: in function 'require'")
stack backtrace:
   0:     0x34c4ba1b2d27 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hab54ed43dc5ba0ec
   1:     0x34c4ba1569bb - core::fmt::write::h5042a955cd11bd50
   2:     0x34c4ba1a6e5e - std::io::Write::write_fmt::h68c8fa8626ff4749
   3:     0x34c4ba1b2aa7 - std::sys_common::backtrace::print::h54c85461e44ea8d9
   4:     0x34c4ba1adf4b - std::panicking::default_hook::{{closure}}::hda2726f2fd9afb24
   5:     0x34c4ba1aeb88 - std::panicking::rust_panic_with_hook::hd230ff4672e5c1d1
   6:     0x34c4ba1b3262 - std::panicking::begin_panic_handler::{{closure}}::h31b4a6d94cd5ac8d
   7:     0x34c4ba1b31b9 - std::sys_common::backtrace::__rust_end_short_backtrace::h71cd43b91e216dee
   8:     0x34c4ba1ae0c6 - rust_begin_unwind
   9:     0x34c4ba15b992 - core::panicking::panic_fmt::h045fb8eaf2a66196
  10:     0x34c4ba15d0c5 - core::result::unwrap_failed::h91d65015008223f8
  11:     0x34c4ba180012 - sile::load_sile::h9176f3b929dd59b3
  12:     0x34c4ba101058 - sile::main::hcc11c96f31167b71
  13:     0x34c4ba1944a3 - std::sys_common::backtrace::__rust_begin_short_backtrace::h923b5b5e9f7e42ce
  14:     0x34c4ba105d23 - main
  15:     0x34ccde7dbc8a - __libc_start1
                               at /usr/src/lib/libc/csu/libc_start1.c:157:7
  16:     0x34c4ba10017d - _start

Any ideas? /usr/local/lib/lua/5.4/lua-utf8.so is of luautf8 0.1.5-2 If I comment out lua-utf8, it crashes on the next require.

I built with CFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib/ LUA_LIB=-llua-5.4 LUA=lua54 ./configure --with-system-luarocks --without-luajit

[alerque]

That error looks like the Lua module somehow got compiled against a different version of Lua than your SILE binary did.

The best way I know of on BSD right now is the routine found in the .cirrus.yml file we use to test Free BSD in CI. It takes the approach of installing as many relevant system dependencies as possible, then supplementing with a few more luarocks (including lua-utf8), then compiling with --with-system-lua-sources. But it is using LuaJIT, so all the lua stuff is the 5.1 variety.

In your case I'm guessing somehow the right version of Lua isn't being detected and passed to the Rust end of things with the Cargo feature flag for --features lua54. and it's still building an against vendored sources even with your LDFLAGS being set.

Try adding --with-system-lua-sources.

Also can I ask (to try to understand if we're doing something wrong) why you're trying to avoid LuaJIT?

[hurzl]

I thought of some confusion with lua / lua54, too ...

Now --with-system-lua-sources ends with

pkg-config exited with status code 1
  > PKG_CONFIG_ALLOW_SYSTEM_LIBS=1 PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=1 pkg-config --libs --cflags lua5.4

  The system library `lua5.4` required by crate `mlua-sys` was not found.

and this is because the package name here is lua-5.4

Without --without-luajit I get

checking for lua54 version... 5.4
checking for lua54 jit version... lua54: (command line):2: bad argument #1 to 'find' (string expected, got nil)
stack traceback:
    [C]: in function 'string.find'
    (command line):2: in main chunk
    [C]: in ?

configure: error: invalid Lua jit version number

$ luajit
LuaJIT 2.0.5 -- Copyright (C) 2005-2017 Mike Pall. http://luajit.org/

[alerque]

Where did you come by the Lua 5.4 installation? Are you using Free BSD packages for it? I'm confused why it is in /usr/local/... in the first place.

It does look like we also have a bug in that it shouldn't be trying to check for LuaJIT if configuring --without-luajit. I'll look into that.

Meanwhile I'm still curious why you aren't just using LuaJIT in the first place given that it's over twice as fast and you won't loose any features relevant to SILE for it being forked from 5.1 era PUC Lua.

[alerque]

and this is because the package name here is lua-5.4

The package name doesn't matter, the issue here is we're using pkg-config to find the binary and it isn't returning anything for the expected binary name. Is the binary named lua-5.4? Does pkg-config --libs --cflags lua-5.4 return anything useful?

[hurzl]

Yes, /usr/local/ is usual for packages in FreeBSD.

The rest I don't understand. With luajit I get a configure error.

The binary name is lua54

$ pkg-config --libs --cflags lua-5.4
-I/usr/local/include/lua54 -L/usr/local/lib -llua-5.4 -lm

[alerque]

In addition to other issues here it looks like you have some problems with cached values in autoconf. You may be running ./configure with new flags but they are getting appended to previous ones you've used (Running ./config.status --config will probably reveal this). You can clear all that out and start over with make distclean.

[alerque]

I would suggest starting fresh with clean sources (or at least make distclean) then not tinker with the Lua detection at all and use roughly what is in the CI builds:

pkg install -y autoconf automake fontconfig GentiumPlus gmake harfbuzz jq libtool pkgconf png rust
pkg install -y luajit lua51-luaexpat lua51-lpeg lua51-luafilesystem lua51-luarocks lua51-luasec lua51-luasocket lua51-lzlib
luarocks51 install cassowary
luarocks51 install cldr
luarocks51 install compat53
luarocks51 install fluent
luarocks51 install linenoise
luarocks51 install loadkit
luarocks51 install lua_cliargs
luarocks51 install luaepnf
luarocks51 install luarepl
luarocks51 install luautf8
luarocks51 install penlight
luarocks51 install vstruct

# cd to clean sile source dir, then if Git use ./bootstrap.sh

./configure MAKE=gmake --disable-font-variations --with-system-lua-sources --with-system-luarocks --without-manual
gmake all
gmake install

[hurzl]

If I set LUA=/usr/local/bin/luajit, I only get sile-lua, which is only for "compatibility and demonstration purposes"`?

[alerque]

You shouldn't need to set LUA=... anything at all. I would make sure your sources are clean and build with the arguments above. At least on Free BSD 14.0 it does figure it out from there. As soon as you go manually overriding paths to anything you have to get a whole bunch of them right (the binary, the package path, the headers, etc.). If you just have the right dependencies installed at the outset and don't try to override any Lua settings at all it should work fine.

[alerque]

Also if you want to simplify system packages and LuaRocks managed stuff you can let SILE vendor them, in which case the install process should look more like this:

# system setup
pkg install -y autoconf automake fontconfig GentiumPlus gmake harfbuzz jq libtool pkgconf png rust luajit

# cd to clean sile source dir, then if Git use ./bootstrap.sh

./configure MAKE=gmake --disable-font-variations --without-system-lua-sources --without-system-luarocks --without-manual
gmake all
gmake install

[hurzl]

Ok I made distclean and installed luarocks51 (the default on my system is 54), installed all the packages above plus lua-zlib luaexpat luasec and configured with

CFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib/ ./configure MAKE=gmake --with-system-lua-sources --with-system-luarocks --without-manual

(Why --disable-font-variations?)

Now it works. Thanks.

[alerque]

The --disable-font-variations flag is because last I checked the HarfBuzz version on Free BSD was too old to support it. That may not be true any more; I have not tried it on 14.1 yet. You can certainly try it with --enable-font-variations. It should then complain during the configure if the HarfBuzz subsetter module isn't available.

I'm still a little confused why you used the CFLAGS and LDFLAGS overrides as those should have been figured out automatically if your system reports the right values from pkg-cofig.

[hurzl]

Yes --without-system-lua-sources --without-system-luarocks seems much less complicated, but that would be the default anyway?

Also gmake, so ./configure CFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib/ should be enough ...

I got no error regarding harfbuzz. Version is 9.0.0. Without CFLAGS I got an error finding "png.h" somewhere.

The confusion came from the lua versions, I only got the default luarocks54, and got no error compiling with lua54.