libqodot fails to find libmap on Unix systems

[w84death]

Can't open dynamic library: /home/kj/Nextcloud/Gamedev/godot/the-complex-project/addons/qodot/bin/x11/libqodot.so. Error: libmap.so: cannot open shared object file: No such file or directory

To reproduce:

• create new project
• put addons/
• run Godot

I did this on old projects, new one. I downloaded release zip, compiled myself (using LLVM). Still the same error.

[kkga]

having the same issue on macOS:

 Can't open dynamic library: /Applications/Godot.app/Contents/MacOS/../Frameworks/libqodot.dylib, error: dlopen(/Applications/Godot.app/Contents/MacOS/../Frameworks/libqodot.dylib, 2): image not found.
 modules/gdnative/gdnative.cpp:483 - No valid library handle, can't get symbol from GDNative object
 modules/gdnative/nativescript/nativescript.cpp:1506 - No nativescript_init in "res://addons/qodot/bin/osx/libqodot.dylib" found
 res://addons/qodot/src/nodes/qodot_map.gd:89 - Attempt to call function 'new' in base 'NativeScript' on a null instance

[Shfty]

From the look of it, @kkga's issue is being caused by a typo I made when packaging the zip - addons/qodot/bin/osx was accidentally named addons/qodot/bin/mac instead.

I suspect the linux shared library issue will still apply after fixing that, but this explains why the posted errors differ. I'll update the release, then dig in with a linux VM and see if I can figure out what's going on.

[Shfty]

I've fixed this under a Linux VM and released Qodot 1.6.1 with up-to-date binaries, as well as getting someone to test on their own Linux box.

I'll leave this issue open for now, since it would be good to confirm whether it works under OSX before closing it.

[varkatope]

I had this error on Linux and then sort of muddled through building the .so libraries (Are these instructions anywhere? Are we meant to have to run a scons build or is there some automatic thing I missed?) but I get a hard crash when trying to build the example map. I'll try to poke around more later, but here's the backtrace in case that means anything to you:

Godot Engine v3.2.stable.official - https://godotengine.org OpenGL ES 3.0 Renderer: AMD RAVEN (DRM 3.36.0, 5.5.2-1-MANJARO, LLVM 9.0.1)

Building /home/varkatope/Game Development/qodot-example/maps/metal-arch.map

handle_crash: Program crashed with signal 11 Dumping the backtrace. Please include this when reporting the bug on https://github.com/godotengine/godot/issues [1] /usr/lib/libc.so.6(+0x3bfb0) [0x7f12b0bebfb0] (??:0) [2] /home/varkatope/Game Development/qodot-example/addons/qodot/bin/x11/libmap.so(map_parser_load+0x60) [0x7f12ac013e00] (??:0) [3] /home/varkatope/Game Development/qodot-example/addons/qodot/bin/x11/libqodot.so(qodot_load_map+0x58) [0x7f12af3d32a8] (??:0) [4] /home/varkatope/Applications/Godot_v3.2-stable_x11.64() [0x25bda57] (:?) [5] /home/varkatope/Applications/Godot_v3.2-stable_x11.64() [0xbe221f] (??:?) [6] /home/varkatope/Applications/Godot_v3.2-stable_x11.64() [0xb83d9d] (??:?) [7] /home/varkatope/Applications/Godot_v3.2-stable_x11.64() [0x29a4dbb] (:?) [8] /home/varkatope/Applications/Godot_v3.2-stable_x11.64() [0x29c2bd8] (:?) [9] /home/varkatope/Applications/Godot_v3.2-stable_x11.64() [0xbe221f] (??:?) [10] /home/varkatope/Applications/Godot_v3.2-stable_x11.64() [0xbfd725] (??:?) [11] /home/varkatope/Applications/Godot_v3.2-stable_x11.64() [0xbfd928] (:?) [12] /home/varkatope/Applications/Godot_v3.2-stable_x11.64() [0xc1940e] (??:?) [13] /home/varkatope/Applications/Godot_v3.2-stable_x11.64() [0xbe22e3] (??:?) [14] /home/varkatope/Applications/Godot_v3.2-stable_x11.64() [0xb83d9d] (??:?) [15] /home/varkatope/Applications/Godot_v3.2-stable_x11.64() [0x29a4dbb] (:?) [16] /home/varkatope/Applications/Godot_v3.2-stable_x11.64() [0x29c2bd8] (:?) [17] /home/varkatope/Applications/Godot_v3.2-stable_x11.64() [0xbe221f] (??:?) [18] /home/varkatope/Applications/Godot_v3.2-stable_x11.64() [0xb83d9d] (??:?) [19] /home/varkatope/Applications/Godot_v3.2-stable_x11.64() [0x29a4dbb] (:?) [20] /home/varkatope/Applications/Godot_v3.2-stable_x11.64() [0x29bb680] (:?) [21] /home/varkatope/Applications/Godot_v3.2-stable_x11.64() [0x29bc011] (:?) [22] /home/varkatope/Applications/Godot_v3.2-stable_x11.64() [0x2882bef] (:?) [23] /home/varkatope/Applications/Godot_v3.2-stable_x11.64() [0xbe22e3] (??:?) [24] /home/varkatope/Applications/Godot_v3.2-stable_x11.64() [0xc88666] (??:?) [25] /home/varkatope/Applications/Godot_v3.2-stable_x11.64() [0xc89660] (??:?) [26] /home/varkatope/Applications/Godot_v3.2-stable_x11.64() [0x187b410] (??:?) [27] /home/varkatope/Applications/Godot_v3.2-stable_x11.64() [0x292e850] (??:?) [28] /home/varkatope/Applications/Godot_v3.2-stable_x11.64() [0x84877d] (??:?) [29] /usr/lib/libc.so.6(__libc_start_main+0xf3) [0x7f12b0bd7153] (??:0) [30] /home/varkatope/Applications/Godot_v3.2-stable_x11.64() [0x855b0e] (??:?) -- END OF BACKTRACE --

[kkga]

Tested with the latest 1.6.1 release (downloaded a zip). Still getting the following errors on macOS:

 Can't open dynamic library: /Users/kkga/repos/qodot-example/addons/qodot/bin/osx/libqodot.dylib, error: dlopen(/Users/kkga/repos/qodot-example/addons/qodot/bin/osx/libqodot.dylib, 2): Library not loaded: libqodot/libmap/build/libmap.dylib
  Referenced from: /Users/kkga/repos/qodot-example/addons/qodot/bin/osx/libqodot.dylib
  Reason: image not found.
 modules/gdnative/gdnative.cpp:483 - No valid library handle, can't get symbol from GDNative object
 modules/gdnative/nativescript/nativescript.cpp:1506 - No nativescript_init in "res://addons/qodot/bin/osx/libqodot.dylib" found
 res://addons/qodot/src/nodes/qodot_map.gd:89 - Attempt to call function 'new' in base 'NativeScript' on a null instance.

[w84death]

Works fine for me on Linux. I needed to remove all .import and re-click here and there. But in the end it works builds map super fast now.

[Shfty]

Reopening this as there are still users having issues on Unix systems.

@varkatope Try the latest zip from the releases page - that has prebuilt Linux binaries that are confirmed working on a couple of different Linux distros. Re. the build process, you're meant to invoke scons platform=x11 bits=64 target=release from the qodot-plugin folder, then copy the appropriate libraries from the nested libqodot/build and libqodot/libmap/build folders into a platform-appropriate subfolder of the addons/qodot/bin directory.

@kkga That's progress - the new error gives me an idea of where it's looking for libmap. I'll have a further poke at it later and see what I can turn up.

[Shfty]

@kkga I'm looking into getting access to a machine to test on. It might take a bit, so in the meantime here are some recompiled OSX libs with tweaked compile params for you to try: qodot-osx-test-libs-1.zip

Hopefully they work- let me know.

[kkga]

@Shfty thanks for investigating this 🙇 Tried the new libs with the latest 1.6.1 release in the qodot-example and a fresh new project... here are the errors I get now:

Wondering if there might be any chance that something's wrong on my end?

[Shfty]

@kkga I doubt it's an issue your end, as it's having the same issue as Linux did with a slightly different flavor to it.

libqodot tries to load, but is looking for libmap in libqodot/libmap/build, which is the directory structure of the build environment. Based on this, it seems that the clang compiler on OSX has a different way of baking in search paths than gcc on Linux.

I've done some more digging, and it looks like the $ORIGIN specifier I used to make that work under gcc on Linux is replaced by @loader_path in clang on OSX, so I've put together two more sets of binaries to try: qodot-test-libs-2.zip qodot-test-libs-3.zip

A note for myself: If one of these ends up working, the first set uses '-Wl,-rpath', '-Wl,@loader_path', and the second uses '-Wl,-rpath,@loader_path'.

[kkga]

@Shfty Tested with both versions: same error :/

[Shfty]

@kkga Hmm, looks like I'm going to have to get this OSX machine sorted to fix it then.

I can think of three possible intermediary solutions:

• Put libmap.dylib in one of your system library paths (such as /usr/lib)
  • Newer versions of OSX lock this by default, so may be difficult
• Create [godot project folder]/addons/qodot/bin/osx/libqodot/libmap/build and put libmap.dylib in there
• Create [godot project folder]/libqodot/libmap/build and put libmap.dylib in there

Which of the latter two will work depends on the root path that's getting picked - a.k.a. whether OSX is looking relative to the plugin folder or project folder.

[varkatope]

@Shfty It's entirely possible I'm doing something wrong, but I can't get it to build the level from .map without completely crashing Godot, whether I use the 1.6.1 release .zip or I invoke scons to generate new libraries. I used the qodot-example map you made and the release zip and got the below.

Building /home/varkatope/Game Development/qodot-example/maps/metal-arch.map

build_map
remove_children
Done in 0.006208 sec

load_map
handle_crash: Program crashed with signal 11

[kkga]

@Shfty putting it into [godot project folder]/libqodot/libmap/build did work for now, thanks for the suggestion 👍

[grenappels]

^ seconding, that also worked for me on mac 🙏

[Shfty]

Good to hear it's a usable workaround- I should be able to test this on OSX soon, so with any luck it won't be needed for long!

@varkatope What distro are you using? That's the first I've seen of a crash during load_map. If possible, it'd be helpful if you could attach a debugger to the Godot editor process and show me the details of where/how it crashes during build.

[varkatope]

@Shfty Sorry, I included my distro further up in the thread, but that gets lost pretty easily. I'm on Manjaro. Here's a trace from GDB. Not sure how useful this is going to be. First time using an external debugger, honestly.

https://pastebin.com/ndnChJdg

[Shfty]

@varkatope That's surprising - the linux build was tested by another user on Manjaro, with no crashes to speak of.

Looking at the stack trace, the part that's crashing is the parsing code that turns the raw map file text into a list of entities, brushes and face planes prior to processing. Unfortunately I can't tell much more in terms of specifics, as there are no debug symbols present in the trace.

If you build the plugin with scons platform=x11 bits=64 target=debug it should give you debugger-friendly versions of libqodot.so and libmap.so, as well as libqodot.pdb and libmap.pdb in the respective build folders. If you put all of those in addons/qodot/bin/x11 and rerun the crashing build with GDB attached, it should output a trace with proper function names and line numbers that I can use to figure out what's going on.

[varkatope]

@Shfty Those .pdb files don't seem to generate, but they wouldn't, though, would they? Aren't they a Windows thing? At any rate I generated target=debug libraries but didn't seem to have gotten a trace that was more useful than the last when running the editor or running the test scene from the editor.

Next I exported the project with debug flag set and ran it from gdb and tried to reload the map. I assume it uses the same code to do so. I got the following, which I'm hoping might be more useful? There's at least a line number...

Starting program: /home/varkatope/Desktop/qodot_debug_test/qodot.x86_64 qodot.x86_64
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
[New Thread 0x7ffff7fcd700 (LWP 9956)]
Godot Engine v3.2.stable.official - https://godotengine.org
[Detaching after fork from child process 9957]
[Detaching after fork from child process 9974]
[New Thread 0x7fffe9709700 (LWP 9991)]
[New Thread 0x7fffe8f08700 (LWP 9992)]
[New Thread 0x7fffe3fff700 (LWP 9993)]
[New Thread 0x7fffe37fe700 (LWP 9994)]
[New Thread 0x7fffe2ffd700 (LWP 9995)]
[New Thread 0x7fffe27fc700 (LWP 9996)]
[New Thread 0x7fffe1ffb700 (LWP 9997)]
[New Thread 0x7fffe17fa700 (LWP 9998)]
[New Thread 0x7fffe0ff9700 (LWP 9999)]
[New Thread 0x7fffbffff700 (LWP 10000)]
[New Thread 0x7fffbf7fe700 (LWP 10001)]
[New Thread 0x7fffbeffd700 (LWP 10002)]
[New Thread 0x7fffbe7fc700 (LWP 10003)]
[New Thread 0x7fffbdffb700 (LWP 10004)]
[New Thread 0x7fffbd7fa700 (LWP 10005)]
[New Thread 0x7fffbcff9700 (LWP 10006)]
[New Thread 0x7ffff6623700 (LWP 10007)]
OpenGL ES 3.0 Renderer: AMD RAVEN (DRM 3.36.0, 5.5.2-1-MANJARO, LLVM 9.0.1)
[New Thread 0x7fffe8066700 (LWP 10008)]
[New Thread 0x7fffe0577700 (LWP 10009)]

Building /home/varkatope/GameDevelopment/qodot-example/maps/metal-arch.map

build_map
remove_children
Done in 0.000311 sec

load_map

Thread 1 "qodot.x86_64" received signal SIGSEGV, Segmentation fault.
0x00007ffff4023951 in map_parser_load (map_file=0x411cb30 "/home/varkatope/GameDevelopment/qodot-example/maps/metal-arch.map") at libqodot/libmap/src/c/map_parser.c:115
115     comment = false;

[kkga]

I can confirm that the latest 1.6.2 release fixes the issue on macOS 👍

[Shfty]

@varkatope Ah, I see - I'm mainly familiar with Windows development, so the linux side of debug symbols is still a bit of a mystery to me.

The log you posted looks useful though- it's segfaulting before even beginning to read the map file, at which point the only thing that's happened is an initialization pass over the parser's dynamic memory to make sure it has valid values.

That narrows things down a lot, so I'll do some poking around and see if there's some invalid behaviour happening in that initialization pass.

Edit: No luck, I'm afraid. Stepping through line-by-line, no dynamic memory allocation / freeing even happens by the time execution reaches that line inside map_parser_load.

I'm going to close this issue and make a new one for your specific crash, since this thread is specific to the issue of misconfigured search paths.

New thread: https://github.com/Shfty/qodot-plugin/issues/86