unix-2.8.6.0 "multiple definition" link errors

simonmichael commented 1 day ago

Your environment

Steps to reproduce

I observed this while building dependencies for hledger, specifically doctest, on github's x64 linux worker with ubuntu-latest image.

Actual behaviour

A package depending on unix, doctest here, shows a number of "multiple definition" errors when linking with libHSunix:

doctest                          > /usr/bin/ld.gold: error: /home/runner/.stack/snapshots/x86_64-linux/9b0f7252b2210020b5ab1b80c846576eb4fa8328011f810b4adf4eadf8b41a2e/9.10.1/lib/x86_64-linux-ghc-9.10.1-69c3/unix-2.8.6.0-8raI90cdflw2fHQVjiSRIl/libHSunix-2.8.6.0-8raI90cdflw2fHQVjiSRIl.a(HsUnix.o): multiple definition of '__hsunix_get_environ'
doctest                          > /usr/bin/ld.gold: /usr/local/.ghcup/ghc/9.10.1/lib/ghc-9.10.1/lib/../lib/x86_64-linux-ghc-9.10.1/unix-2.8.5.1-e9c3/libHSunix-2.8.5.1-e9c3.a(HsUnix.o): previous definition here
doctest                          > /usr/bin/ld.gold: error: /home/runner/.stack/snapshots/x86_64-linux/9b0f7252b2210020b5ab1b80c846576eb4fa8328011f810b4adf4eadf8b41a2e/9.10.1/lib/x86_64-linux-ghc-9.10.1-69c3/unix-2.8.6.0-8raI90cdflw2fHQVjiSRIl/libHSunix-2.8.6.0-8raI90cdflw2fHQVjiSRIl.a(HsUnix.o): multiple definition of '__hsunix_rtldNext'
doctest                          > /usr/bin/ld.gold: /usr/local/.ghcup/ghc/9.10.1/lib/ghc-9.10.1/lib/../lib/x86_64-linux-ghc-9.10.1/unix-2.8.5.1-e9c3/libHSunix-2.8.5.1-e9c3.a(HsUnix.o): previous definition here
doctest                          > /usr/bin/ld.gold: error: /home/runner/.stack/snapshots/x86_64-linux/9b0f7252b2210020b5ab1b80c846576eb4fa8328011f810b4adf4eadf8b41a2e/9.10.1/lib/x86_64-linux-ghc-9.10.1-69c3/unix-2.8.6.0-8raI90cdflw2fHQVjiSRIl/libHSunix-2.8.6.0-8raI90cdflw2fHQVjiSRIl.a(HsUnix.o): multiple definition of '__hsunix_rtldDefault'
doctest                          > /usr/bin/ld.gold: /usr/local/.ghcup/ghc/9.10.1/lib/ghc-9.10.1/lib/../lib/x86_64-linux-ghc-9.10.1/unix-2.8.5.1-e9c3/libHSunix-2.8.5.1-e9c3.a(HsUnix.o): previous definition here
doctest                          > /usr/bin/ld.gold: error: /home/runner/.stack/snapshots/x86_64-linux/9b0f7252b2210020b5ab1b80c846576eb4fa8328011f810b4adf4eadf8b41a2e/9.10.1/lib/x86_64-linux-ghc-9.10.1-69c3/unix-2.8.6.0-8raI90cdflw2fHQVjiSRIl/libHSunix-2.8.6.0-8raI90cdflw2fHQVjiSRIl.a(HsUnix.o): multiple definition of '__hsunix_push_module'
doctest                          > /usr/bin/ld.gold: /usr/local/.ghcup/ghc/9.10.1/lib/ghc-9.10.1/lib/../lib/x86_64-linux-ghc-9.10.1/unix-2.8.5.1-e9c3/libHSunix-2.8.5.1-e9c3.a(HsUnix.o): previous definition here
doctest                          > /usr/bin/ld.gold: error: /home/runner/.stack/snapshots/x86_64-linux/9b0f7252b2210020b5ab1b80c846576eb4fa8328011f810b4adf4eadf8b41a2e/9.10.1/lib/x86_64-linux-ghc-9.10.1-69c3/unix-2.8.6.0-8raI90cdflw2fHQVjiSRIl/libHSunix-2.8.6.0-8raI90cdflw2fHQVjiSRIl.a(HsUnix.o): multiple definition of '__hsunix_clocks_per_second'
doctest                          > /usr/bin/ld.gold: /usr/local/.ghcup/ghc/9.10.1/lib/ghc-9.10.1/lib/../lib/x86_64-linux-ghc-9.10.1/unix-2.8.5.1-e9c3/libHSunix-2.8.5.1-e9c3.a(HsUnix.o): previous definition here
doctest                          > /usr/bin/ld.gold: error: /home/runner/.stack/snapshots/x86_64-linux/9b0f7252b2210020b5ab1b80c846576eb4fa8328011f810b4adf4eadf8b41a2e/9.10.1/lib/x86_64-linux-ghc-9.10.1-69c3/unix-2.8.6.0-8raI90cdflw2fHQVjiSRIl/libHSunix-2.8.6.0-8raI90cdflw2fHQVjiSRIl.a(HsUnix.o): multiple definition of '__hscore_readdir'
doctest                          > /usr/bin/ld.gold: /usr/local/.ghcup/ghc/9.10.1/lib/ghc-9.10.1/lib/../lib/x86_64-linux-ghc-9.10.1/unix-2.8.5.1-e9c3/libHSunix-2.8.5.1-e9c3.a(HsUnix.o): previous definition here
doctest                          > /usr/bin/ld.gold: error: /home/runner/.stack/snapshots/x86_64-linux/9b0f7252b2210020b5ab1b80c846576eb4fa8328011f810b4adf4eadf8b41a2e/9.10.1/lib/x86_64-linux-ghc-9.10.1-69c3/unix-2.8.6.0-8raI90cdflw2fHQVjiSRIl/libHSunix-2.8.6.0-8raI90cdflw2fHQVjiSRIl.a(HsUnix.o): multiple definition of '__hscore_d_name'
doctest                          > /usr/bin/ld.gold: /usr/local/.ghcup/ghc/9.10.1/lib/ghc-9.10.1/lib/../lib/x86_64-linux-ghc-9.10.1/unix-2.8.5.1-e9c3/libHSunix-2.8.5.1-e9c3.a(HsUnix.o): previous definition here
doctest                          > /usr/bin/ld.gold: error: /home/runner/.stack/snapshots/x86_64-linux/9b0f7252b2210020b5ab1b80c846576eb4fa8328011f810b4adf4eadf8b41a2e/9.10.1/lib/x86_64-linux-ghc-9.10.1-69c3/unix-2.8.6.0-8raI90cdflw2fHQVjiSRIl/libHSunix-2.8.6.0-8raI90cdflw2fHQVjiSRIl.a(HsUnix.o): multiple definition of '__hscore_free_dirent'
doctest                          > /usr/bin/ld.gold: /usr/local/.ghcup/ghc/9.10.1/lib/ghc-9.10.1/lib/../lib/x86_64-linux-ghc-9.10.1/unix-2.8.5.1-e9c3/libHSunix-2.8.5.1-e9c3.a(HsUnix.o): previous definition here

Downgrading unix-2.8.6.0 to unix-2.8.5.1 in my build plan fixed the problem.

hasufell commented 1 day ago

Can you provide a minimal reproducer?

hasufell commented 1 day ago

How did you even manage to build a package with doctest, GHC 9.10.1 and unix 2.8.6.0?

That isn't possible:

$ cabal build -w ghc-9.10.1
Resolving dependencies...
Error: [Cabal-7107]
Could not resolve dependencies:
[__0] trying: unix-test-0.1.0.0 (user goal)
[__1] next goal: unix (dependency of unix-test)
[__1] rejecting: unix-2.8.5.1/installed-fee6 (conflict: unix-test => unix>=2.8.6.0)
[__1] trying: unix-2.8.6.0
[__2] trying: filepath-1.5.2.0/installed-fab1 (dependency of unix +/-os-string)
[__3] trying: doctest-0.23.0 (dependency of unix-test)
[__4] next goal: process (dependency of doctest)
[__4] rejecting: process-1.6.19.0/installed-2f6e (conflict: unix==2.8.6.0, process => unix==2.8.5.1/installed-fee6)
[__4] trying: process-1.6.25.0
[__5] next goal: ghc (dependency of doctest)
[__5] rejecting: ghc-9.10.1/installed-7767 (conflict: process==1.6.25.0, ghc => process==1.6.19.0/installed-2f6e)
[__5] rejecting: ghc; 9.10.1, 9.8.2, 9.8.1, 9.6.6, 9.6.5, 9.6.4, 9.6.3, 9.6.1, 9.4.8, 9.4.7, 9.4.6, 9.4.5, 9.4.4, 9.4.3, 9.4.2, 9.4.1, 9.2.8, 9.2.7, 9.2.6, 9.2.5, 9.2.4, 9.2.3, 9.2.2, 9.2.1, 9.0.2, 8.10.7, 8.10.2, 8.10.1, 8.8.3, 8.8.1, 8.6.5, 8.6.4, 8.6.1, 8.4.4, 8.4.3, 8.4.1, 8.2.2, 8.2.1, 9.2.3.20220620 (constraint from non-reinstallable package requires installed instance)
[__5] fail (backjumping, conflict set: doctest, ghc, process)

hasufell commented 1 day ago

This looks like a stack bug to me, @mpilgrem

simonmichael commented 1 day ago

I can't @hasufell, but here's the failing run if it provides any more clues: https://github.com/simonmichael/hledger/actions/runs/12136160544/job/33836702162

simonmichael commented 1 day ago

And some pinned versions: https://github.com/simonmichael/hledger/blob/master/stack.yaml

hasufell commented 1 day ago

Well, if you look at the log, it's trying to link two versions of unix.

mpilgrem commented 15 hours ago

I do not understand why Stack is building unix-2.8.6.0 at all, given that unix-2.8.5.1 is specified as an extra-dep. I'll see if I can reproduce it locally.

simonmichael commented 15 hours ago

The build log was from when unix-2.8.6.0 was specified in the stack.yaml. (I linked the stack.yaml version where it's replaced by unix-2.8.5.1.)

mpilgrem commented 14 hours ago

@simonmichael, I'll see if I can get an explanation of 'why', but my local 'fix' was to replace the package versions specified that are boot packages with the GHC 9.10.1 boot package versions. So, the stack.yaml becomes (extract):

extra-deps:
- base-compat-0.14.0
# for hledger-lib
- Cabal-3.12.0.0
- Cabal-syntax-3.12.0.0
- directory-1.3.8.3    # Changed!
- filepath-1.5.2.0     # Changed!
- ghc-boot-9.10.1@rev:1
- process-1.6.19.0     # Changed!
- time-1.12.2
- unix-2.8.5.1
#- unix-2.8.6.0  # https://github.com/haskell/unix/issues/329
# for windows
- Win32-2.14.0.0
# for hledger
- haskeline-0.8.2.1
# to silence a warning
- wizards-1.0.3@rev:3
# for dev builds when the debug flag is enabled:
# - ghc-debug-convention-0.6.0.0
# - ghc-debug-stub-0.6.0.0

flags:
  Win32:
    os-string: true
  directory:
    os-string: true
  unix:
    os-string: true~yaml

The explanation is that package doctest depends on package ghc and package ghc is not re-installable: you can't build it against a different version of process than the GHC boot package version. I ran into something similar here: https://discourse.haskell.org/t/why-does-ghc-9-2-4-require-process-1-6-13-2/4884

simonmichael commented 12 hours ago

Thanks for the explanation @mpilgrem. Pretty much the same workaround I found (I haven't had trouble with the other deps yet, just unix).

I'm sure we've all been bitten by this before, so possibly still something stack could prevent ?

But, clearly not a problem with the unix package, so I'll close this soon.

haskell / unix