search

edubart / otclient

An alternative tibia client for otserv written in C++11 and Lua, made with a modular system that uses lua scripts for ingame interface and functionality, making otclient flexible and easy to customize
Other
646 stars 399 forks source link

Cannot load incompatible bytecode (compiling bytecode with luajit) #1205

Closed Source61 closed 1 year ago

Source61 commented 1 year ago

Steps to reproduce (include any configuration/script required to reproduce)

  1. Compile otclient
  2. Compile a lua module such as /modules/corelib/math.lua to bytecode using LATEST LuaJIT version
  3. Run otclient and check otclient.log

Expected behaviour

Should run without an issue

Actual behaviour

Complains about incompatible bytecode and exits.

Environment

Windows 10 with Debian 12 WSL. Modified OTClient version from 2-3 years ago. Never touched src/framework/luaengine/*. The issue seems to have started happening after I upgraded all the software on the WSL one day about 6-9 months ago and likely LuaJIT was upgraded as well, never had this issue before and been using LuaJIT compilation for years. I've also tried upgrading to the most recent LuaJIT (and all other) vcpkg libraries to no avail.

This issue incidentally happened around the same time (9 months ago) https://github.com/neovim/neovim/issues/18169 https://stackoverflow.com/questions/72875661/i-got-this-error-after-upgrading-neovim-to-version-0-7-2-2

diath commented 1 year ago

The issue always stems from incompatible LuaJIT versions between the one you used to compile the Lua files to bytecode and the one you linked the client against, please keep in mind that LuaJIT doesn't guarantee bytecode compatibility even for MINOR releases.

sonoro1234 commented 1 year ago

https://github.com/LuaJIT/LuaJIT/issues/967#issuecomment-1448580644

This comment should be here, not in LuaJIT. Just to discover: https://github.com/edubart/otclient/commit/e3caaacc55b68d27176645fbc2dc059398c9458f

sonoro1234 commented 1 year ago

Also, if you close the issue it is very difficult to trace it. Perhaps you should reopen it and reread diath's comment.

Source61 commented 1 year ago

The issue always stems from incompatible LuaJIT versions between the one you used to compile the Lua files to bytecode and the one you linked the client against, please keep in mind that LuaJIT doesn't guarantee bytecode compatibility even for MINOR releases.

Indeed. But the problem is, as I commented here today https://github.com/LuaJIT/LuaJIT/issues/967#issuecomment-1448580644, that updating otclient-sdk with the latest LuaJIT version didn't help as the application crashed (application cannot start or be opened or something like that), and I saw a warning while compiling otclient with one of the luaengine files about redeclaring macros, I don't know, I just think it might be an issue with otclient at this point since updating LuaJIT in vcpkg, in otclient-sdk, and the LuaJIT binary, did not help - in fact otclient crashes with the latest LuaJIT version.

I now use an old LuaJIT binary which works for me, but underlying the issue there seem to be an issue with otclient/outdated LuaJIT code as far as I can tell.

Also, if you close the issue it is very difficult to trace it. Perhaps you should reopen it and reread diath's comment.

I didn't close any of the issues, they were closed by the maintainers. Thanks for commenting here. I don't feel it's appropriate to re-create a new issue when the first one was closed though. I feel like if the maintainer want to close the issue, he wants to close the issue, he should re-open it if he understands the issue, if he doesn't understand or accept the issue then there's nothing I can do, spamming new issues won't help, that's how I feel.

LuaJIT/LuaJIT#967 (comment)

This comment should be here, not in LuaJIT. Just to discover: e3caaac

I will try this, thanks. Update: Tried it, but it appears I already have this commit.

diath commented 1 year ago

that updating otclient-sdk with the latest LuaJIT version didn't help as the application crashed

The client crashes because it cannot load incompatible bytecode from corelib, and an error in corelib is considered a fatal error because it's essential for the client to function.

but underlying the issue there seem to be an issue with otclient/outdated LuaJIT code as far as I can tell.

Always compare the output of luajit -v in the terminal and print(jit.version) in the game client, if they don't match, that's an issue with using incompatible versions, not with otclient/LuaJIT, vcpkg doesn't always immediately update their packages to the newest versions, so if you download the latest LuaJIT from their website, and update vcpkg LuaJIT to latest, that doesn't automatically imply they're both the same version.

Source61 commented 1 year ago

that updating otclient-sdk with the latest LuaJIT version didn't help as the application crashed

The client crashes because it cannot load incompatible bytecode from corelib, and an error in corelib is considered a fatal error because it's essential for the client to function.

No, it gives me an "incompatible bytecode" error in the logs when there's a version mismatch, this crashes the client before it even gets to open the log, before any Lua code is loaded, basically somewhere in the start of or pre-main (since I open/print to logfile quite early in C++ main instead of the old init.lua). Actually pre-main most likely yes, because my first lines of main is:

int main(int argc, const char* argv[])
{
    std::vector<std::string> args(argv, argv + argc);

    if (!g_custom.isRelease())
        g_logger.setLogFile(g_resources.getWorkDir() + "otclient.log");

    g_logger.info(stdext::date_time_string("Application started at %d %b %Y %T"));

And the application started at timestamp is not updated.

If you want I can try to run the client in a debugger and see what line is causing the crash, if you're willing to look this through. Otherwise I'm just gonna continue using an old LuaJIT version. (And btw the versions doesn't have to match perfectly, you can use beta3 for beta1, just not the most newest version(s)).

but underlying the issue there seem to be an issue with otclient/outdated LuaJIT code as far as I can tell.

Always compare the output of luajit -v in the terminal and print(jit.version) in the game client, if they don't match, that's an issue with using incompatible versions, not with otclient/LuaJIT, vcpkg doesn't always immediately update their packages to the newest versions, so if you download the latest LuaJIT from their website, and update vcpkg LuaJIT to latest, that doesn't automatically imply they're both the same version.

Hard to print(jit.version) in a client that won't allow you to load the Lua code necessary to run a print statement. Not even that, but the client crashes way before Lua code is even run, so I can't just temporarily avoid loading bytecode as a temporary bypass.

That said, how would I find out about the vcpkg LuaJIT version? Through print(jit.version)? Or does that print the otclient-sdk version? Update: It prints LuaJIT2.1.0-beta1, same as otclient-sdk version, I doubt vcpkg version is this old. Update2: Found the vcpkg version in includes/luajit.h:

#define LUAJIT_VERSION      "LuaJIT 2.1.0-beta3"
#define LUAJIT_VERSION_NUM  20100  /* Version 2.1.0 = 02.01.00. */
diath commented 1 year ago

That's unrelated to your issue with incompatible bytecode. I can build and run the client just fine using a fresh Win10 VM and latest LuaJIT from vcpkg so it's unlikely that the issue is related to either otclient or LuaJIT. Considering that your client crashes before it even enters the main routine, it's probably related to your compiler setup/libs/DLL files.

Source61 commented 1 year ago

That's unrelated to your issue with incompatible bytecode. I can build and run the client just fine using a fresh Win10 VM and latest LuaJIT from vcpkg so it's unlikely that the issue is related to either otclient or LuaJIT. Considering that your client crashes before it even enters the main routine, it's probably related to your compiler setup/libs/DLL files.

Well the question is what LuaJIT cli version you're using to compile the bytecode. It worked fine for me as well using Debian 11, then I upgraded to Debian 12 (Testing) and it still was fine, but then about 9-11 months ago they updated the version, so when I upgraded my LuaJIT cli version it broke compatibility with the client (otclient SDK). Just for the record I'm using Windows 10 + Debian 12 WSL on this laptop that I use for OT dev.

It doesn't seem that the issue is vcpkg, but with otclient-SDK like I said. In fact print(jit.version) does NOT print my vcpkg version, it prints my otclient-SDK version, which is LuaJIT 2.1.0-beta1. Removing otclient-SDK doesn't work as it will complain about missing LuaJIT, at least I'm pretty sure, as I've tried to remove other otclient-SDK libraries and have it complain to me about this. UPGRADING otclient-SDK results in otclient crashing before entering main.

Update: Actually, only removing otclient-sdk LuaJIT from the project worked and my version is now LuaJIT 2.1.0-beta3... However this version STILL IS NOT recent enough to use the latest Debian 12 LuaJIT release. luajit -v reports "LuaJIT 2.1.0-beta3" however it's not the same version as the tags 2.1.0-beta3, it's more recent. Debian 11-12 has many different versions of LuaJIT 2.1.0-beta3. The latest version of Debian 12's LuaJIT 2.1.0-beta3 is called "2.1.0~beta3+git20220320+dfsg-4.1"

You can find all the release versions here: http://snapshot.debian.org/package/luajit/ Anyway, I guess the simplest solution for now is just to use tags 2.1.0 beta3 cli version and work with that.