Open mpickering opened 5 years ago
Unfortunately it opens another can of worms. Dynamic linking in Haskell doesn't work as of 2019, there are a bunch of issues. It may work if we also build a static version, doubling build times. Cabal-install, Stack and GHC are all broken in one way or another:
https://github.com/haskell/cabal/issues/1720 https://github.com/commercialhaskell/stack/issues/3409 https://gitlab.haskell.org/ghc/ghc/issues/16526
and so on :) Basically it seems to work, but corner cases (such as dynamic linking of Setup.hs
) don't work. Also Windows has had dynamic linking completely broken since GHC 7.8, and while it seems to be fixed now, neither Stack nor Haskell.org have binaries with dynamic base
, which prevents -dynamic
from working on Windows in general even if it's able now on Windows GHC developers' boxes.
I'm a big proponent of dynamic builds, they are better in every aspect than "traditional" static builds. So dynamic linking of HIE is a pretty nice feature to have.
But it's blocked by upstream bugs for now.
hie
is the only binary which needs to be dynamically linked. It doesn't have a custom Setup.hs
or anything complicated like the tickets you are suggesting. Have you tried compiling HIE with the executable-dynamic: True
option set? Does it not work on your system?
I didn't try it. But it can't work on Windows because I checked the last binaries and .dlls aren't there. So any build script should at very least change the setting if on Windows.
Also, the dynamic linking may seem to work, but bugs affect corner cases we may or may not encounter in the future. For example, the static Setup.hs linking bug affects dependencies too, but only if we disable "vanilla" (static) libraries with library-vanilla: false
as well. And with vanilla libraries build time is twice as long.
So yeah, we may try to do it, but the idea of my comments is merely that we may regret :) as we need to support a wide range of platforms and scenarios.
Seems working on the first look on Win 10. Just to be sure i did it right:
I just checked that hie launches ok for stack project and handles basic diagnostic like misspelled name of type in scope.
Is hie dynamically linked indeed? can you look at the .exe with http://dependencywalker.com/ ? I don't think stack
uses cabal.project
at all so I suspect you got a statically linked .exe
@nponeccop i've loaded the EXE into dep walker, how exactly i determine if it's dynamically linked?
It should have some 90+ libHS*.dll. If it has the same dlls as stack.exe then it's static.
Welp, it didn't work then. All I see is winapi related libs and no traces of anything libHS*-like, module(?) list is slightly longer than stack.exe, with addition of DBGHELP.DLL, PSAPI.DLL, SHLWAPI.DLL and WINMM.DLL.
with addition of DBGHELP.DLL, PSAPI.DLL, SHLWAPI.DLL and WINMM.DLL
these are all windows dlls. So it's definitely not dynamic.
@mpickering how do you build hie so it's dynamic? with cabal new-build
?
I can have a look at this. What is actually required for this?
@power-fungus We want to build hie and all its dependencies with -dynamic
, which means generating .so
and linking dynamically. Note that this means that the snapshots in ~/.stack are built with -dynamic
too.
I know one way to do it:
ghc-options:
"$everything": -dynamic
I think it goes to ~/.stack/config
or something
@nponeccop so this is really more of a thing the user has to do? Because the build process does not touch these files.
We need to answer some questions:
hie
executable with Stack?So we need someone to do this research. I only found the ghc-options: -dynamic
solution but it seems to be a hack: e.g. cabal-install has a high-level settings of executable-dynamic: true
and corresponding configure options instead. Neither I tested that it actually worked
@nponeccop Please can you reopen this issue as it is not resolved yet.
You can read more about the investigations I did into this issue on the GHC issue tracker
In summary
--enable-library-for-ghci
and --enable-split-sections
. There has recently been a patch which disables --enable-library-for-ghci
but it will take a while before all users have moved onto a commit which contains this fix.It seems that your argument boils down to the fact that dynamic linking might not work on all platforms but as it improves the experience of users on platforms where it does work then we should try to enable it or at least provide an option to enable it.
Further to this, we should certainly make sure that hie-nix
dynamically links hie
as it is nix users who are most going to be affected.
Oh sorry I closed it by mistake
In the interim, can we make this platform-dependent? So it is enabled for platforms where we know it works.
@alanz I am currently experimenting with dynamic building on linux. But I need assistence in testing if the build script I write does indeed build everything dynamically.
Can someone try to check if cabal new-install hie --enable-executable-dynamic -w ...
does the correct thing? Or should a different command be used?
I need assistance in testing if the build script I write does indeed build everything dynamically.
The goal is just to build the hie executable dynamically. So you can check it with ldd {full path to hie}
. If you see many libHS*
files it's dynamic. If you see about 10 references then it's static.
Are there any updates on this? I can add cabal-dynamic-*
targets (or something similar) and adding the flag --enable-executable-dynamic
. However, I am not an expert on the caveats that need to be considered.
Well, -dynamic
is faster but apparently not a good option by default. So perhaps it's worth adding a target as an experiment, most people probably won't notice the extra few seconds it takes..
For the record, --enable-executable-dynamic
does nothing on windows in my tests (inspecting exes with the dependency walker)
Moreove it seems that ghc doesn't support it yet: https://gitlab.haskell.org/ghc/ghc/wikis/windows-dynamic-linking
So no benefit but no harm either.
See https://gitlab.haskell.org/ghc/ghc/issues/15524
https://github.com/haskell/haskell-ide-engine/pull/1126/commits/c356c389c1b9a017470667892587b380c8ebaa85
Testing this change on my branch it makes a huge difference.
Any volunteers to integrate this into
stack
and the./install.hs
script?@power-fungus perhaps?