Open jamesthompson opened 4 years ago
@jamesthompson Thank you for the detailed report and the reproduction test-case. I'm able to reproduce the issue on Linux with nixpkgs.
There are two issues at play here: 1) Paths of external source files are different in the execroot than in the repository root. 2) GHCi (generally speaking) can't load static libraries.
I've added some verbosity for debugging and a hacky workaround for 1) here.
Passing -v
to GHCi shows that it tries to load libpgcommon.a
from external/nixpkgs_postgres/lib
. That is the path of the "source file" (i.e. not generated file) libpgcommon.a
in the execroot. These source file paths are correct for sources from the main workspace, but incorrect for source files from external repositories. That's an issue in rules_haskell that should be fixed. One possible approach is to prefix $RULES_HASKELL_EXEC_ROOT
for such external sources. This is a bit tricky since we don't have access to the execroot from within Starlark, and we'd like these locations to be correct for the hie-bios
output group as well.
In https://github.com/tweag/rules_haskell/commit/28f99d2fe31fe5572ac1994c50021917654873c8 I've hacked around that limitation using a genrule
that copies the library files to turn them into generated files rather than source files. This way they will be located under bazel-out/...
and GHCi can find them in the repository root as well. (Note, this is done for debugging purposes. I wouldn't consider that a solution to this issue.)
When GHCi is able to find libpgcommon.a
then it fails to load it with the following error:
<command line>: User-specified static library could not be loaded (bazel-out/k8-fastbuild/bin/external/nixpkgs_postgres/gen/lib/libpgcommon.a)
Loading static libraries is not supported in this configuration.
Try using a dynamic library instead.
Generally speaking GHCi cannot load static libraries when built with the dynamic RTS as is the default on Linux and MacOS. One way around this is to build GHC to use the static RTS, see https://github.com/tweag/rules_haskell/pull/970#issuecomment-506714877 for details. Unfortunately, there are no static RTS bindists available for Linux and MacOS, so that requires compiling GHC.
@aherrmann Many thanks indeed for the thorough and rapid response! I'm wondering how it is that stack or a regular cabal-install GHC would deal with these static libraries. I'm sure in the past I must have used these libraries in GHCi.
@aherrmann As a workaround attempt I thought to elide the static library from your genrule and try that, low and behold: it works.
lib_files = glob(["lib/**/*.so*", "lib/**/*.dylib"], allow_empty = True)
Perhaps a short term solution would be to ensure that the .a
-suffixed libraries are always excluded from being passed to the GHCi wrapper?
With this change I'm able to continue development. Thanks a lot for your help.
@jamesthompson Thanks for the follow-up, that's good to know. If libpgcommon.a
is not required then you can avoid the issue without using a genrule
to copy files. Instead, you can just use a filegroup
that only collects the shared libraries. Based on the rules_nixpkgs
template something like the following should do:
package(default_visibility = ["//visibility:public"])
filegroup(
name = "lib",
srcs = glob(["lib/**/*.so*", "lib/**/*.dylib"]),
)
- Paths of external source files are different in the execroot than in the repository root.
This aspect has been fixed in the meantime. The genrule trick is no longer necessary.
- GHCi (generally speaking) can't load static libraries.
There is not much rules_haskell can do about this. postgres is a bit peculiar in that pgcommon only seems to be offered in static form, not in dynamic form. Otherwise, it should pick the dynamic version.
Perhaps a short term solution would be to ensure that the .a-suffixed libraries are always excluded from being passed to the GHCi wrapper?
Yes, that seems like a viable solution. rules_haskell does support a GHC with a static runtime, in those cases we'd still want to permit .a
extensions. But, otherwise we could skip them.
Describe the bug
Inability to load statically linked library in regular dynamically linked
haskell_library
rule-depending@repl
orhaskell_repl
.Specifically, in my issue case
postgresql
compiles at least one library as a statically-linked output among other dynamically linked.so
/ (macOS.dylib
) outputs. The first issue encountered is for thepgcommon
library - which is only available as a statically-linked library (libpgcommon.a
) (I think regardless of the standard nix derivation from nixpkgs).Building a
haskell_library
dependent transitively upon the postgresqllibpq
library works without issue - e.g. with the hackage packagepostgresql-libpq
.However attempting to load a GHCi session with either the
haskell_library
-@repl
orhaskell_repl
rules results in failure to load the required static libraries - e.g.pgcommon
which has the namelibpgcommon.a
. It is attempting to resolve the expected dynamically named equivalentlibpgcommon.dylib
(on macOS) which does not exist.See below for an exposition of the error given by GHCi:
A
haskell_binary
target depending on ahaskell_library
target which itself transitively depends onlibpq
also seemingly works fine.To Reproduce
I've built a minimal reproducing case here in a branch: https://github.com/jamesthompson/rules_haskell/commit/9bbc8b2c4067a04f75ea2ee643f3cc31f1c8f9a7
Expected behavior
Ideally, using
@repl
or bazel run "haskell_repl
" would load the statically linked libraries where no dynamically linked equivalents are available.Environment
Additional context I'm unclear if this is related to issues given here for
hrepl