Cannot load native module with MinGW (GC error)

[grmoth]

I'm trying to load native modules, but I get the same error everytime (and with every module I tried)

Janet 1.32.1-local mingw/x64/gcc - '(doc)' for help
repl:1:> (use spork/cmath)
janet interpreter runtime error at line 525 in file src/core/gc.c: please initialize janet before use

As noted above, every thing is compiled with MinGW, and the modules are installed with jpm, apparently without any warning or error.

Everything else (including loading source module, and compiling to binary .exe) seem to work without an issue.

[bakpakin]

This is most likely related to an issue with how linking is done here - presubmably, spork/cmath was compiled with a different C runtime than your version of Janet was, resulting in this issue.

[grmoth]

Ok, I think it's beyond my knowledge, and I apologize for the probably stupid question.

Both Janet and spork/cmath are compiled with the same version of MinGW's gcc, and are linked to the (presumably) same KERNEL32.dll and msvcrt.dll. Shouldn't it be share the same C runtime ?

[bakpakin]

How did you build Janet with mingw? Did you use meson or make?

I will try to repro but some more precise details on how you built everything would help

[grmoth]

I used make, with the default Makefile, except for the "ln ..." lines in the install part, replaced by cp. jpm is also installed with make.

The config.janet is the default, except for cc/cc-link ("gcc") and importlibext (".dll", without it jpm throw a file not found error at the copying step).

MinGW is installed from Scoop.

[bakpakin]

Ok, I was able to get mingw working but I needed to make some changes as well to the jpm config. I set importlibext to nil instead of .dll - importlibs are not needed when you are using a compiler that is not MSVC (it's not a windows thing, its a compiler thing). I did need to make sure also the my janet binary was availble as janet.exe and not just janet, but after that it worked.

You can use ldd on cmath.dll to check what it's linked to

[grmoth]

Thank you for taking the time to check. With the same change as you, I still get the same error. I assume that the error does not come from janet itseft, but from my setup.

ldd shows that janet.exe and cmath.dll are linked to the same .dll, except that cmath.dll is also linked to libjanet.so (I guess that it is expected), and that the memory location for MinGW's libwinpthread-1.dll is different between the two, despite the file path being the same (but I have no idea what does it mean).

[grmoth]

I tried to replicate on another computer, and run exactly into the same symptoms. I must be doing something wrong, but I have no idea what.

Mingw is the one from scoop, corresponding to GCC 13.2.0 with POSIX threads and UCRT. I tried other versions, with the same results.

Only changes to the Makefile are the replacement of ln lines with a copy, and the executable renamed to janet.exe.

Build is run with the command : make.exe install install-jpm-git -j8 CC=gcc PREFIX=C:/Users/grmoht/local UNAME=MINGW

config.janet is the one generated by make-config.janet, with the only change to cc, cc-link and importlibext exposed before :

# Autogenerated by generate-config in jpm/make-config.janet
(def config @{:ar "ar"
  :auto-shebang true
  :binpath "C:/Users/grmoht/local/bin"
  :c++ "c++"
  :c++-link "c++"
  :cc "gcc"
  :cc-link "gcc"
  :cflags @["-std=c99"]
  :cflags-verbose @[]
  :cppflags @["-std=c++11"]
  :curlpath "curl"
  :dynamic-cflags @["-fPIC"]
  :dynamic-lflags @["-shared"]
  :gitpath "git"
  :headerpath "C:/Users/grmoht/local/include/janet"
  :importlibext nil
  :is-msvc false
  :janet "janet"
  :janet-cflags @[]
  :janet-importlib "C:/Users/grmoht/local/lib/janet.lib"
  :janet-lflags @["-lws2_32" "-lwsock32" "-lpsapi"]
  :ldflags @[]
  :lflags @[]
  :libpath "C:/Users/grmoht/local/lib"
  :manpath "C:/Users/grmoht/local/share/man/man1/"
  :modext ".dll"
  :nocolor false
  :pkglist "https://github.com/janet-lang/pkgs.git"
  :silent false
  :statext ".static.lib"
  :tarpath "tar"
  :test false
  :use-batch-shell false
  :verbose false})

ldd shows a few differences (reodered to ease comparison) :

$ ldd janet.exe
    KERNEL32.DLL => /c/WINDOWS/System32/KERNEL32.DLL (0x7ffb79cc0000)
    KERNELBASE.dll => /c/WINDOWS/System32/KERNELBASE.dll (0x7ffb79990000)
    libwinpthread-1.dll => /c/Users/grmoht/Scoop/apps/mingw/current/bin/libwinpthread-1.dll (0x7ffb6b6d0000)
    msvcrt.dll => /c/WINDOWS/System32/msvcrt.dll (0x7ffb7bb30000)
    MSWSOCK.DLL => /c/WINDOWS/SYSTEM32/MSWSOCK.DLL (0x7ffb789e0000)
    ntdll.dll => /c/WINDOWS/SYSTEM32/ntdll.dll (0x7ffb7bc30000)
    PSAPI.DLL => /c/WINDOWS/System32/PSAPI.DLL (0x7ffb7b4d0000)
    RPCRT4.dll => /c/WINDOWS/System32/RPCRT4.dll (0x7ffb7a340000)
    ucrtbase.dll => /c/WINDOWS/System32/ucrtbase.dll (0x7ffb79740000)
    WS2_32.dll => /c/WINDOWS/System32/WS2_32.dll (0x7ffb79e30000)
    WSOCK32.dll => /c/WINDOWS/SYSTEM32/WSOCK32.dll (0x7ffb64cb0000)

$ ldd cmath.dll
    KERNEL32.DLL => /c/WINDOWS/System32/KERNEL32.DLL (0x7ffb79cc0000)
    KERNELBASE.dll => /c/WINDOWS/System32/KERNELBASE.dll (0x7ffb79990000)
    libgcc_s_seh-1.dll => /c/Users/grmoht/Scoop/apps/mingw/current/bin/libgcc_s_seh-1.dll (0x7ffb5d190000)
    libjanet.so => /c/Users/grmoht/Developpement/lib/libjanet.so (0x7ffb08a10000)
        libwinpthread-1.dll => /c/Users/grmoht/Scoop/apps/mingw/current/bin/libwinpthread-1.dll (0x7ffb6a7a0000)
    msvcrt.dll => /c/WINDOWS/System32/msvcrt.dll (0x7ffb7bb30000)
    MSWSOCK.DLL => /c/WINDOWS/SYSTEM32/MSWSOCK.DLL (0x7ffb789e0000)
    ntdll.dll => /c/WINDOWS/SYSTEM32/ntdll.dll (0x7ffb7bc30000)
    PSAPI.DLL => /c/WINDOWS/System32/PSAPI.DLL (0x7ffb7b4d0000)
    RPCRT4.dll => /c/WINDOWS/System32/RPCRT4.dll (0x7ffb7a340000)
    ucrtbase.dll => /c/WINDOWS/System32/ucrtbase.dll (0x7ffb79740000)
    WS2_32.dll => /c/WINDOWS/System32/WS2_32.dll (0x7ffb79e30000)
    WSOCK32.dll => /c/WINDOWS/SYSTEM32/WSOCK32.dll (0x7ffb64cb0000)

cmath.dll is linked to the same dll than janet.exe, plus libjanet.so and libgcc_s_seh-1.dll. (libjanet.so is also linked to the later.) One notable change : the memory location of libwinpthread-1.dll is different between janet.exe/libjanet.so and cmath.dll, while for every other library the location is the same. But I don't know if this is important or not.

I also tried to static-link the MinGW libraries, the results are the same (except that libwinpthread-1.dll and libgcc_s_seh-1.dll disappear from the ldd output, obviously).

[grmoth]

I've understood the source of the issue, when a compilation of janet suddenly worked with no apparent difference. But I could see that the "good" .dlls was linked to janet.exe while the "bad" is linked to libjanet.so.

Long story short : the janet.lib is generated twice during the compilation : once for janet.exe and once for libjanet.so. If janet.exe is compiled first, janet.lib is overridden by the libjanet.so version, and I cannot build working .dlls. When janet.exe is compiled last, the problem disappears.

So deleting the generation of the import lib of libjanet.so in the makefile suppress the issue.

[bakpakin]

Thanks for the deep dive, @grmoth! This explains a lot, and explains the variability. It should be easy to modify the Makefile to remove this issue