Loading native module from built binary crashes on windows

[saikyun]

I've learnt that on Linux you can use -rdynamic to load native modules from built binaries. But when looking around this flag doesn't seem to be needed on windows. However, I get crashes when trying to load native modules this way on windows.

Here is an example, create this file called runthing.janet

(spit "project.janet"
``
(declare-project
  :name "runthing"
  :dependencies ["https://github.com/janet-lang/circlet"])

(declare-executable
  :name "runthing"
  :entry "native-thing.janet")
``)

(spit "native-thing.janet"

(defn main [& args] (spit "runthis.janet"

# change this to where you have circlet installed
(import /C:/Users/saiky/AppData/Local/Apps/Janet/Library/circlet)
(print "lul")
```)

  (dofile "runthis.janet"))
````)

Then run this in a native tools command prompt:

janet runthing.janet && jpm deps && jpm build && build\runthing.exe

It should print lul, and it does if I do janet native-thing.janet, but it does not when running the .exe.

[sogaiu]

Once I change:

(import /C:/Users/saiky/AppData/Local/Apps/Janet/Library/circlet)

in the code above for runthing.janet appropriately for my Windows 10 environment, I was able to reproduce. That is, I didn't see lul output.

[sogaiu]

The exit code after was -1073741819.

Got this at the command prompt via the command:

echo %errorlevel%

[bakpakin]

I am interested in the verbose output of jpm, I suspect something has gone wrong with the linking. I also don't recall if jpm willl only statically link libraries that are in the designated syspath.

[bakpakin]

I also wonder if everything works if you try to just import circlet rather than the full path

[saikyun]

By changing to absolute import, like this:

(spit "project.janet"
``
(declare-project
  :name "runthing"
  :dependencies ["https://github.com/janet-lang/circlet"])

(declare-executable
  :name "runthing"
  :entry "native-thing.janet")
``)

(spit "native-thing.janet"

(defn main [& args] (spit "runthis.janet"

# change this to where you have circlet installed
(import circlet)
(print "lul")
```)

  (dofile "runthis.janet"))
````)

Then running using: janet runthing.janet && jpm -v deps && jpm -v build && build\runthing.exe

I get the following:

C:\Users\saiky\dev\native-test\test2>janet runthing.janet && jpm -v deps && jpm -v build && build\runthing.exe
git -C C:\Users\saiky\AppData\Local\Apps\Janet\Library\/.cache/git__https___github.com_janet-lang_circlet fetch --tags origin
git -C C:\Users\saiky\AppData\Local\Apps\Janet\Library\/.cache/git__https___github.com_janet-lang_circlet fetch origin HEAD
From https://github.com/janet-lang/circlet
 * branch            HEAD       -> FETCH_HEAD
git -C C:\Users\saiky\AppData\Local\Apps\Janet\Library\/.cache/git__https___github.com_janet-lang_circlet reset --hard FETCH_HEAD
HEAD is now at 2e84f54 Merge pull request #15 from saikyun/master
git -C C:\Users\saiky\AppData\Local\Apps\Janet\Library\/.cache/git__https___github.com_janet-lang_circlet submodule update --init --recursive
removing C:\Users\saiky\AppData\Local\Apps\Janet\Library\/circlet.dll
removing C:\Users\saiky\AppData\Local\Apps\Janet\Library\/circlet.meta.janet
removing C:\Users\saiky\AppData\Local\Apps\Janet\Library\/circlet.static.lib
removing manifest C:\Users\saiky\AppData\Local\Apps\Janet\Library\/.manifests/circlet.jdn
Uninstalled.
generating C:\Users\saiky\AppData\Local\Apps\Janet\Library\/.manifests/circlet.jdn...
git remote get-url origin
git rev-parse HEAD
Installed as 'circlet'.
copying build/circlet.dll to C:\Users\saiky\AppData\Local\Apps\Janet\Library\...
C:\Windows\System32\xcopy.exe build\circlet.dll C:\Users\saiky\AppData\Local\Apps\Janet\Library\ /y /s /e /i
build\circlet.dll
1 File(s) copied
copying build/circlet.meta.janet to C:\Users\saiky\AppData\Local\Apps\Janet\Library\...
C:\Windows\System32\xcopy.exe build\circlet.meta.janet C:\Users\saiky\AppData\Local\Apps\Janet\Library\ /y /s /e /i
build\circlet.meta.janet
1 File(s) copied
copying build/circlet.static.lib to C:\Users\saiky\AppData\Local\Apps\Janet\Library\...
C:\Windows\System32\xcopy.exe build\circlet.static.lib C:\Users\saiky\AppData\Local\Apps\Janet\Library\ /y /s /e /i
build\circlet.static.lib
1 File(s) copied
generating executable c source build/runthing.exe.c from native-thing.janet...
compiling build/runthing.exe.c to build/build___runthing.exe.o...
cl.exe /c /nologo /MD -IC:\Users\saiky\AppData\Local\Apps\Janet\C\ -IC:\Users\saiky\AppData\Local\Apps\Janet\Library\ -O2 /Fobuild/build___runthing.exe.o build/runthing.exe.c
runthing.exe.c
linking build/runthing.exe...
link.exe /OUT:build/runthing.exe build/build___runthing.exe.o C:\Users\saiky\AppData\Local\Apps\Janet\C\/libjanet.lib /nologo
   Creating library build\runthing.lib and object build\runthing.exp
error: could not find module circlet:
    /circlet.jimage
    /circlet.janet
    /circlet/init.janet
    /circlet.dll
  in require-1 [boot.janet] on line 2732, column 20
  in import* [boot.janet] on line 2771, column 15
  in _thunk [runthis.janet] (tailcall) on line 5, column 1
  in dofile [boot.janet] (tailcall) on line 2709, column 7

[saikyun]

Could anyone give me some tips on how to debug this? I'd like to work more on this (specifically with cgen), but I don't really know where to start. :)

[saikyun]

Maybe not much to go on, but running the runthing.exe using windbg shows me an error that isn't showing when just running the .exe. No clue if it's helpful or just because of thing happening differently when using windbg:

error: could not open file runthis.janet with mode wb
  in spit [boot.janet] on line 1708, column 13
  in main [native-thing.janet] on line 7, column 3

[JohnDoneth]

I am having this issue too when trying to run eval-string to load a native module. Minimal example to reproduce:

(eval-string ``
    (import spork/json) 
    (pp spork/json/decode)
  ``)

This gives me the same error code mentioned above of -1073741819 which translates to an ACCESS_VIOLATION error.

[bakpakin]

I strongly suspect that this is due to incorrect setup of jpm - I am able to compile, load and run spork.json with the latest janet and jpm on my Windows 10 machine. Perhaps jpm and janet were compiled with different visual studios/architectures?