Closed lunaris closed 5 years ago
If so, do we want to undertake this work? If not, what is the alternative (e.g. should we just use Haskell packages from nixpkgs and only offer things like *_custom_package_github for non-Hackage-package cases)?
Fair question! I think there's no clear answer. There are two broad strategies available currently:
Pros:
Cons:
Pros:
Cons:
For people with lots of system dependencies, Nixpkgs could well be the better option, because in such a use case you want to carefully manage the versions of system dependencies and Nix is a good way to do that anyways. For those who don't, having everything in Bazel and not dealing with Nix at all is quite nice. In the latter use case, it doesn't matter much that Hackage packages that depend on external libraries require more work that is already done in Nixpkgs. And if you need the portability, it's probably your only option.
I've been playing with this for a week or so now and it's tricky. Some packages are fairly straightforward to sort out (postgresql-simple
, libpq
), others less so (libssh2
due to its use of c2hs
and company). I'm going to see how I fare using Nix for a select few base packages (essentially anything that steps outside "pure Haskell" on Hackage) -- in theory these dependencies shouldn't change that often. It does seem somewhat untenable to put in the same amount of work that has already been expended in nixpkgs again in Hazel/rules_haskell
.
@lunaris how exactly did you solve this in the end? I'm currently trying to build a project which uses cryptonite and running into this issue
In the end we moved to use the Nixpkgs support exposes by rules_haskell
for Hackage packages (since that arrived at about the time I
raised/potentially partly in response to this issue). Disadvantages include
having to manage two caches and bits of build process, advantages are
leveraging the packaging efforts of the Nixpkgs community. So far it seems
to work very well.
On Wed, 13 Feb 2019, 16:00 David Smith <notifications@github.com wrote:
@lunaris https://github.com/lunaris how exactly did you solve this in the end? I'm currently trying to build a project which uses cryptonite and running into this issue
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/FormationAI/hazel/issues/41#issuecomment-463255584, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJ_0bo2xYLAIjGHgfsdXgsn7q-j7fp4ks5vNDa2gaJpZM4WBYQ2 .
what do you mean, you import cryptonite from nixpkgs? were you able to use this with hazel?
Yes, sorry. In short, we don't use Hazel -- all Hackage packages come from
Nixpkgs (using rules_haskell
which in turn uses rules_nixpkgs
). Nix
thus takes care of building those packages/special rules and foreign
interface trickery etc.; we only manage Bazel rules (like
haskell_library
) for code we write.
On Wed, 13 Feb 2019, 16:20 David Smith <notifications@github.com wrote:
what do you mean, you import cryptonite from nixpkgs? were you able to use this with hazel?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/FormationAI/hazel/issues/41#issuecomment-463263737, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJ_0dzbGILgy6vVta8T5o9IyZiPBVBKks5vNDtpgaJpZM4WBYQ2 .
ok thanks. I found there was way too much boiler plate using nixpkgs haskell packages, especially with a multi-module project. I had to repeat similar code in 3 or more places.
I don't suppose you have some idea of how it would be possible to patch it so that hazel could use it? As of this moment I have hacked things by getting nixpkgs to manage this one package, not very nice at all.
Some packages (I think) require custom build instructions to compile properly in the environment that Hazel,
rules_haskell
and (assuming this is the default mode of usage forrules_haskell
) Nix/nixpkgs. As an example,cryptonite-0.25
contains anextra-lib: pthread
declaration that states it must be linked with thepthread
library on POSIX. To get this to work, the rough steps are something like:glibc
from nixpkgs usingnixpkgs_package
with a customBUILD
that usespatchelf
(also pulled in fromnixpkgs
) to patchlibpthread
and expose it.libpthread
toextra_libs
when configuring Hazel.@haskell_cryptonite
libpthread.so
is in fact anld
script that combines the dynamic and static portions of the library.In short a custom build process for a working copy of
libpthread
needs to be devised in Bazel and then supplied usingextra_libs
to Hazel. Having already encountered this issue forlibstdc++
(and not really considering that it might bite in other forms), https://github.com/tweag/rules_haskell/pull/387 lays out what I thought a better interface torules_haskell
might be to enable users ofrules_haskell
and Hazel to navigate this situation a bit better. However, on encountering this and also examining bits of Hazel such as the customBUILD.*
s forwai
,conduit
, etc., I'm wondering -- are we destined to implement a similar effort to the one already being undertaken by nixpkgs, whereby there is a large amount of custom packaging for finicky cases like this? If so, do we want to undertake this work? If not, what is the alternative (e.g. should we just use Haskell packages from nixpkgs and only offer things like*_custom_package_github
for non-Hackage-package cases)?CC @mboes since perhaps this affects
rules_haskell
/there has been discussion around this already.