Haskell deps that use backpack get mangled

[andrewthad]

Issue description

I have a repository on github that illustrates this issue. Using nix to build a haskell project with a dependency that uses backpack internally does not work correctly.

Steps to reproduce

The steps to reproduce are provided in the repository linked above, but a copy is provided here:

Run the following commands, which results in the error shown afterwards:

bash> git clone https://github.com/andrewthad/nix-backpack-problem
bash> nix-build --pure release.nix
... elided for brevity ...
Preprocessing executable 'example' for nix-backpack-problem-1.0.0..
Building executable 'example' for nix-backpack-problem-1.0.0..
[1 of 1] Compiling Main             ( Main.hs, dist/build/example/example-tmp/Main.o )
/nix/store/ah79awgrjc75c68nn6gp32v4a9qli7mn-primitive-containers-0.1.0.0/lib/ghc-8.4.1/primitive-containers-0.1.0.0/Data/Map/Unboxed/Lifted.hi
Declaration for primitive-containers-0.1.0.0-Dh702QUZ6mrBrQ6cfYPU3V:Data.Map.Unboxed.Lifted.$fIsListMap2{v r8x}:
  Bad interface file: /nix/store/ah79awgrjc75c68nn6gp32v4a9qli7mn-primitive-containers-0.1.0.0/lib/ghc-8.4.1/primitive-containers-0.1.0.0/Map.hi
      Something is amiss; requested module  primitive-containers-0.1.0.0-JNYz44HuDdtIDtASnNcM9F-map-indef+2bbwYj3qpR1AhwaWbfmWe2:Map differs from name found in the interface file primitive-containers-0.1.0.0-JNYz44HuDdtIDtASnNcM9F-map-indef+K2XRbKjbYZpLEn7VFxOfUn:Map (if these names look the same, try again with -dppr-debug)

<no location info>: error:
    Cannot continue after interface file error
builder for '/nix/store/0bkrbljm2kyph09waf7lw9ksmqfkrckf-nix-backpack-problem-1.0.0.drv' failed with exit code 1
error: build of '/nix/store/0bkrbljm2kyph09waf7lw9ksmqfkrckf-nix-backpack-problem-1.0.0.drv' failed

Make sure you do not have a dist or dist-newstyle directory in here since that confuses nix-build and can make it claim that it has successfully built the project. I am able to successfully build this project, and projects like it, with cabal new-build when I am not using nix. I suspect that the problem is that the different instantiations of the same indefinite module are somehow clobbering one another.

cc @chessai @taktoa

[andrewthad]

cc @ezyang

[ezyang]

Yes, the path /nix/store/ah79awgrjc75c68nn6gp32v4a9qli7mn-primitive-containers-0.1.0.0/lib/ghc-8.4.1/primitive-containers-0.1.0.0/Map.hi definitely looks like clobbering; it really needs to have the Backpack ID (which Cabal should have generated in the folder structure) here.

I am not too familiar with Nix. Does anyone here know / know who would know how Nix supports convenience libraries (without Backpack) and, if it does, how it manages installing them into the nix store?

[cocreature]

@ezyang Nix uses Setup.hs directly, see https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/haskell-modules/generic-builder.nix#L333. I’m not sure it does anything special for convenience libs.

[ezyang]

The registration code looks fine, because it is using ‘id’ to determine what name it should register as, and id does contain the instantiation ID (that’s the hash that is after the plus sign in the ID). Also, registration code being a problem is inconsistent with the symptoms, which is that the build products themselves are clobbering each other.

So, what I expect, is that Nix is feeding Cabal information about where it should “install” the files, and that’s the crux of the matter. In particular, these lines don’t look too good: https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/haskell-modules/generic-builder.nix#L141

Libsubdir is using $pkgid only, which looks bad to me. The default setting in Cabal is $abi/$libname; note the use of $libname here: $libname will record instantiation information; $pkgid does not. But I imagine that if we change this, there are knock on effects in the Nix file. "--ghc-option=-optl=-Wl,-rpath=$out/lib/${ghc.name}/${pname}-${version}" also looks questionable.

[chessai]

@peti this seems more like a bug than a request for an enhancement, if i'm not mistaken

[quasicomputational]

Packages that only use mixins for hiding/renaming modules are also broken, even if they don't use backpack itself (i.e., no signatures).

[cartazio]

Theres a reasonably decent probability that (should all the details work out) the next minor / major releases of vector will use backpack to simplify the internal of vector while providing the same apis with improved extensibility. (To be clear, I’m still futzing around with some prototypes to understand if there’s any blockers I’ve not anticipated)

I’m hoping to have some clarity on how effective this design direction is over the coming month or so and be able to do a release for everyone sometime in the next 6 months if this all works out as I hope.

Since I use cabal I’m not impacted by this and other build systems having buggy support for ghc Haskell. But id like to still be friends with them after the next release of vector. So I’d really appreciate it if this bug could be taken seriously.

Also it’s a bummer that a really cool modularity feature isn’t supported.

But seriously: please fix this. Vector is widely used and I want to make sure it keeps on getting better. Not supporting backpack hurts the entire ecosystem and will suck for everyone.

[expipiplus1]

The testcase @andrewthad posted builds successfully now. Can this issue be closed and backpack marked as supported?

CC @shlevy, I think you are using backpack with nix, right?

[chessai]

Unfortunately this is not the only place where backpack is still an issue. @andrewthad has another one that we ran into at work awhile back. Perhaps he can provide a minimal example, as I no longer work there.

[andrewthad]

There isn't any nix file in the repo for building it, but https://github.com/andrewthad/vex-instantiate is another super minimal example of a library that cabal build (3.0) can build, but as of November 15, 2019, I don't think nix was able to.

[stale[bot]]

Hello, I'm a bot and I thank you in the name of the community for opening this issue.

To help our human contributors focus on the most-relevant reports, I check up on old issues to see if they're still relevant. This issue has had no activity for 180 days, and so I marked it as stale, but you can rest assured it will never be closed by a non-human.

The community would appreciate your effort in checking if the issue is still valid. If it isn't, please close it.

If the issue persists, and you'd like to remove the stale label, you simply need to leave a comment. Your comment can be as simple as "still important to me". If you'd like it to get more attention, you can ask for help by searching for maintainers and people that previously touched related code and @ mention them in a comment. You can use Git blame or GitHub's web interface on the relevant files to find them.

Lastly, you can always ask for help at our Discourse Forum or at #nixos' IRC channel.

[expipiplus1]

still important to me

[stale[bot]]

I marked this as stale due to inactivity. → More info

[ezyang]

bump

[stale[bot]]

I marked this as stale due to inactivity. → More info

[expipiplus1]

Still important

On Sat, 21 Aug 2021, 10:55 am stale[bot], @.***> wrote:

I marked this as stale due to inactivity. → More info https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/NixOS/nixpkgs/issues/40128#issuecomment-903043402, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGRJXALAPCO4OGR2SGZEKLT54IQ7ANCNFSM4E6URGSA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

[stale[bot]]

I marked this as stale due to inactivity. → More info

[expipiplus1]

Still important

[raehik]

I've just encountered this trying to build lzo which depends on digest-sig.

[s-and-witch]

Hello there.

I want to investigate this issue, but I'm not familiar with haskell4nix infra and don't know much about Cabal, so, as the first step, I just came up with self-contained reproducer that works with modern nixpkgs: https://github.com/s-and-witch/backpack-example

The error in my reproducer differs from the one mention in the topic start, and I don't actually know if this is the same issue, or I just found the other one.

Anyway, Cabal library want usecases-0.1.0.0-5NHg...+HmuJ... package to be produced from usecases-0.1.0.0-5NHg... (UseCases as UseCasesProd) requires (Sig as Prod) statement. May anyone guide me on how I may fix it (or at least point to the links to the related sources, so I would be able to came up with a fix prototype?).