runghc can load .so but ghc-compiled executables can't.

[YuMingLiao]

[root@nixos:~/runghc-or-ghc]# tutd -d data
Project:M36 TutorialD Interpreter 0.9.5
Type ":help" for more information.
A full tutorial is available at:
https://github.com/agentm/project-m36/blob/master/docs/tutd_tutorial.markdown
TutorialD (master/main): loadatomfunctions "MyServerFunctions" "haskellFunctions" "MyServerFunctions.so"
TutorialD (master/main): :commit
TutorialD (master/main): :quit
Goodbye.

[root@nixos:~/runghc-or-ghc]# runghc hair.hs 
┌──────────┬──────────┐
│hair::Hair│name::Text│
├──────────┼──────────┤
│Blond     │"Colin"   │
└──────────┴──────────┘

[root@nixos:~/runghc-or-ghc]# ghc hair.hs
[1 of 1] Compiling Main             ( hair.hs, hair.o )
Linking hair ...

[root@nixos:~/runghc-or-ghc]# ./hair
hair: Failed to load MyServerFunctions.so
CallStack (from HasCallStack):
  error, called at ./src/lib/ProjectM36/Transaction/Persist.hs:169:19 in project-m36-0.9.5-3VsotkcQy7E7glkiX9DhMg:ProjectM36.Transaction.Persist

[YuMingLiao]

It happens even when not using those type trickey with extra pacakges but only someFunctions the old way.

[YuMingLiao]

Here is an example using .o

[root@nixos:~/runghc-or-ghc]# tutd -d data
Warning: Haskell scripting disabled: ScriptSessionLoadError <command line>: cannot satisfy -trust project-m36
    (use -v for more information)
Project:M36 TutorialD Interpreter 0.9.5
Type ":help" for more information.
A full tutorial is available at:
https://github.com/agentm/project-m36/blob/master/docs/tutd_tutorial.markdown
TutorialD (master/main): loadatomfunctions "OldWayAtomFunctions" "someFunctions" "OldWayAtomFunctions.o"
TutorialD (master/main): :commit
TutorialD (master/main): :quit

[root@nixos:~/runghc-or-ghc]# runghc hair.hs
Segmentation fault (core dumped)

[root@nixos:~/runghc-or-ghc]# ghc hair.hs 
[1 of 1] Compiling Main             ( hair.hs, hair.o )
Linking hair ...

[root@nixos:~/runghc-or-ghc]# ./hair
Warning: Haskell scripting disabled: ScriptSessionLoadError <command line>: cannot satisfy -trust project-m36
    (use -v for more information)
hair: data/5da7b22b-cdfa-4a59-af93-5703efe76361/../compiled_modules/OldWayAtomFunctions.o: unknown symbol `base_GHCziBase_const_closure'
hair: Could not load Object Code data/5da7b22b-cdfa-4a59-af93-5703efe76361/../compiled_modules/OldWayAtomFunctions.o.

Segmentation fault (core dumped)

[YuMingLiao]

< FunctionBuiltInBody (\(x:_) -> pure (BoolAtom (const True x)))}
> FunctionBuiltInBody (\(x:_) -> pure (BoolAtom True))}

[root@nixos:~/runghc-or-ghc]# tutd -d data
Warning: Haskell scripting disabled: ScriptSessionLoadError <command line>: cannot satisfy -trust project-m36
    (use -v for more information)
Project:M36 TutorialD Interpreter 0.9.5
Type ":help" for more information.
A full tutorial is available at:
https://github.com/agentm/project-m36/blob/master/docs/tutd_tutorial.markdown
TutorialD (master/main): loadatomfunctions "OldWayAtomFunctions" "someFunctions" "OldWayAtomFunctions.o"
TutorialD (master/main): :commit
TutorialD (master/main): :quit
Goodbye.

[root@nixos:~/runghc-or-ghc]# runghc hair.hs
Segmentation fault (core dumped)

[root@nixos:~/runghc-or-ghc]# ghc hair.hs

[root@nixos:~/runghc-or-ghc]# ./hair
Warning: Haskell scripting disabled: ScriptSessionLoadError <command line>: cannot satisfy -trust project-m36
    (use -v for more information)
hair: data/3a715e4c-1047-4bbb-84cf-551df5794812/../compiled_modules/OldWayAtomFunctions.o: unknown symbol `projectzmm36zm0zi9zi5zm3VsotkcQy7E7glkiX9DhMg_ProjectM36ziBase_BoolAtom_con_info'
hair: Could not load Object Code data/3a715e4c-1047-4bbb-84cf-551df5794812/../compiled_modules/OldWayAtomFunctions.o.

Segmentation fault (core dumped)

[agentm]

Hm, I'll admit that we've never tested with runghc. Maybe gdb will explain the crash- I'll take a look.

[YuMingLiao]

.so:

[root@nixos:~/runghc-or-ghc/shared_object]# runghc hair.hs
loadFunctionFromDirectory
loadFunction
LoadAutoObjectFile OldWayAtomFunctions someFunctions data/faaf9602-a4e5-4192-b464-24f63e9e6b57/../compiled_modules/OldWayAtomFunctions.so
loadFunction
LoadDLLFile OldWayAtomFunctions someFunctions data/faaf9602-a4e5-4192-b464-24f63e9e6b57/../compiled_modules/OldWayAtomFunctions.so
loadDLL
loadFuncForSymobl
OldWayAtomFunctions_someFunctions_closure
┌──────────┬──────────┐
│hair::Hair│name::Text│
├──────────┼──────────┤
│Blond     │"Colin"   │
└──────────┴──────────┘

[root@nixos:~/runghc-or-ghc/shared_object]# ./hair
loadFunctionFromDirectory
loadFunction
LoadAutoObjectFile OldWayAtomFunctions someFunctions data/faaf9602-a4e5-4192-b464-24f63e9e6b57/../compiled_modules/OldWayAtomFunctions.so
loadFunction
LoadDLLFile OldWayAtomFunctions someFunctions data/faaf9602-a4e5-4192-b464-24f63e9e6b57/../compiled_modules/OldWayAtomFunctions.so
loadDLL
hair: Failed to load OldWayAtomFunctions.so
CallStack (from HasCallStack):
  error, called at ./src/lib/ProjectM36/Transaction/Persist.hs:169:21 in project-m36-0.9.5-3VsotkcQy7E7glkiX9DhMg:ProjectM36.Transaction.Persist

[root@nixos:~/runghc-or-ghc/shared_object]# nm hair | grep OldWayAtomFunctions_someFunctions_closure
(no result)

So it seems loadDLL works for runghc but not ghc-compiled executables. Not sure why.

[YuMingLiao]

loadDLL.hs

import GHCi.ObjLink

main = do
  initObjLinker RetainCAFs
  loadDLL "/root/runghc-or-ghc/shared_object/data/compiled_modules/OldWayAtomFunctions.so"
  _ <- resolveObjs
  lookupSymbol "OldWayAtomFunctions_someFunctions_closure"

[root@nixos:~/runghc-or-ghc/shared_object]# runghc loadDLL.hs 
(successfully finished)

[root@nixos:~/runghc-or-ghc/shared_object]# ./loadDLL
loadDLL: ^^ Could not load 'OldWayAtomFunctions_someFunctions_closure', dependency unresolved. See top entry above.

[YuMingLiao]

Okay, I've made it work.

[root@nixos:~/runghc-or-ghc/shared_object]# ghc loadDLL.hs -no-keep-hi-file -no-keep-o-file -dynamic
[1 of 1] Compiling Main             ( loadDLL.hs, loadDLL.o )
Linking loadDLL ...

[root@nixos:~/runghc-or-ghc/shared_object]# ./loadDLL
(successfully finished)

[root@nixos:~/runghc-or-ghc/shared_object]# ghc hair.hs -no-keep-hi-file -no-keep-o-file -dynamic
[1 of 1] Compiling Main             ( hair.hs, hair.o )
Linking hair ...

[root@nixos:~/runghc-or-ghc/shared_object]# ./hair
loadFunctionFromDirectory
loadFunction
LoadAutoObjectFile OldWayAtomFunctions someFunctions data/faaf9602-a4e5-4192-b464-24f63e9e6b57/../compiled_modules/OldWayAtomFunctions.so
loadFunction
LoadDLLFile OldWayAtomFunctions someFunctions data/faaf9602-a4e5-4192-b464-24f63e9e6b57/../compiled_modules/OldWayAtomFunctions.so
loadDLL
loadFuncForSymobl
OldWayAtomFunctions_someFunctions_closure
┌──────────┬──────────┐
│hair::Hair│name::Text│
├──────────┼──────────┤
│Blond     │"Colin"   │
└──────────┴──────────┘

It seems that if a database loads .so file, the Haskell client code needs to be compiled by ghc with a -dynamic flag.

So I guess if I have dynamically-load atom functions in my database, I can't deploy my program as a fully static executable.

[agentm]

Indeed, library loading can be very tricky, so I am willing to improve it however possible.

Could you tell me more about the use-case for using runghc?

[agentm]

So I guess if I have dynamically-load atom functions in my database, I can't deploy my program as a fully static executable.

Well, at the very least, Project:M36 needs to load Project:M36 as a library before it can load the symbols in the object, which haskell compiles by default as a shared library to load at runtime via GHC (library). That's why the cabal file compiles everything with -rdynamic.

I think that you are correct then- any use of the ScriptSession will require dynamic loading of some kind. If you want a statically-linked executable (which, in general, is difficult but not impossible with GHC), then your best option is to compile your functions directly into Project:M36 and dump ScriptSession.

[agentm]

After further reflection, you might be able to link the object file with your client, but you will still need to be able to load Project:M36 as a library.

[YuMingLiao]

Could you tell me more about the use-case for using runghc?

I think runghc is fine running code that load .so. I use runghc to develop code because I don't need to compile it every time. I was trying to create fully static executable for minimal deployment. After I start service, the "Failed to load ..." message shows up. Hence the issue.