[Ubuntu] [3.12.0.0-prerelease] Semaphore does not exist

jasagredo commented 3 weeks ago

Describe the bug End of log says it all:

Benchmark 2: cabal build -O1 --semaphore all
Resolving dependencies...
Build profile: -w ghc-9.8.2 -O1
In order, the following will be built (use -v for more details):
...
 - ouroboros-consensus-cardano-0.15.0.0 (test:cardano-test) (first run)
Created semaphore called cabal_semaphore_1a with 24 slots.
Configuring test suite 'doctest' for ouroboros-consensus-0.17.0.0...
Configuring library for strict-sop-core-0.1.0.0...
Preprocessing test suite 'doctest' for ouroboros-consensus-0.17.0.0...
Building test suite 'doctest' for ouroboros-consensus-0.17.0.0...
cabal_semaphore_1a: semOpen: does not exist (No such file or directory)
Preprocessing library for strict-sop-core-0.1.0.0...
Building library for strict-sop-core-0.1.0.0...
cabal_semaphore_1a: semOpen: does not exist (No such file or directory)
Error: [Cabal-7125]
Failed to build test:doctest from ouroboros-consensus-0.17.0.0.
Failed to build strict-sop-core-0.1.0.0 (which is required by test:mock-test from ouroboros-consensus-diffusion-0.15.0.0, test:infra-test from ouroboros-consensus-diffusion-0.15.0.0 and others).

To Reproduce

I have reproduced it two times with the following invocation:

hyperfine --prepare "cabal clean" "cabal build -O1 -j all" "cabal build -O1 --semaphore all" [--show-output]

On https://github.com/IntersectMBO/ouroboros-consensus/commit/98dbae087fb6cce193f96df97eacf08cf1ce4520

System information

Ubuntu 22.04 under WSL, Windows 11


❯ cabal --version
cabal-install version 3.11.0.0
compiled using version 3.12.0.0 of the Cabal library

❯ ghc --version The Glorious Glasgow Haskell Compilation System, version 9.8.2


Despite that message, it is the one installed by the command on the 3.12.0.0 release:

❯ ghcup list ... ✔✔ cabal 3.12.0.0-prerelease stray

jasagredo commented 3 weeks ago

I changed branches, did some changes, and the error seems to persist, regardless of whether I cabal clean or not:

❯ cabal build all --semaphore
...
 - ouroboros-consensus-cardano-0.15.0.0 (test:cardano-test) (first run)
Created semaphore called cabal_semaphore_10 with 24 slots.
Preprocessing test suite 'doctest' for ouroboros-consensus-0.17.0.0...
Building test suite 'doctest' for ouroboros-consensus-0.17.0.0...
cabal_semaphore_10: semOpen: does not exist (No such file or directory)
Error: [Cabal-7125]
Failed to build test:doctest from ouroboros-consensus-0.17.0.0.

mpickering commented 3 weeks ago

It seems the project doesn't built with 9.8.2 so I can't reproduce as-per the instructions:

[nix-shell:~/ouroboros-consensus]$  cabal build -O1 --semaphore all
Resolving dependencies...
Error: [Cabal-7107]
Could not resolve dependencies:
[__0] trying: ouroboros-consensus-0.17.0.0 (user goal)
[__1] trying: vector-0.13.1.0 (dependency of ouroboros-consensus)
[__2] next goal: cardano-crypto-class (dependency of ouroboros-consensus)
[__2] rejecting: cardano-crypto-class-2.1.4.0 (conflict: pkg-config package libblst-any, not found in the pkg-config database)
[__2] rejecting: cardano-crypto-class-2.1.3.0 (conflict: vector==0.13.1.0, cardano-crypto-class => vector<0.13)
[__2] skipping: cardano-crypto-class; 2.1.2.0, 2.1.1.0, 2.1.0.2, 2.1.0.1, 2.1.0.0 (has the same characteristics that caused the previous version to fail: excludes 'vector' version 0.13.1.0)
[__2] trying: cardano-crypto-class-2.0.0.1
[__3] next goal: base (dependency of ouroboros-consensus)
[__3] rejecting: base-4.19.1.0/installed-862d (conflict: cardano-crypto-class => base>=4.14 && <4.17)
[__3] skipping: base; 4.19.1.0, 4.19.0.0, 4.18.2.0, 4.18.1.0, 4.18.0.0, 4.17.2.1, 4.17.2.0, 4.17.1.0, 4.17.0.0 (has the same characteristics that caused the previous version to fail: excluded by constraint '>=4.14 && <4.17' from 'cardano-crypto-class')
[__3] rejecting: base; 4.16.4.0, 4.16.3.0, 4.16.2.0, 4.16.1.0, 4.16.0.0, 4.15.1.0, 4.15.0.0, 4.14.3.0, 4.14.2.0, 4.14.1.0, 4.14.0.0, 4.13.0.0, 4.12.0.0, 4.11.1.0, 4.11.0.0, 4.10.1.0, 4.10.0.0, 4.9.1.0, 4.9.0.0, 4.8.2.0, 4.8.1.0, 4.8.0.0, 4.7.0.2, 4.7.0.1, 4.7.0.0, 4.6.0.1, 4.6.0.0, 4.5.1.0, 4.5.0.0, 4.4.1.0, 4.4.0.0, 4.3.1.0, 4.3.0.0, 4.2.0.2, 4.2.0.1, 4.2.0.0, 4.1.0.0, 4.0.0.0, 3.0.3.2, 3.0.3.1 (constraint from non-reinstallable package requires installed instance)
[__3] fail (backjumping, conflict set: base, cardano-crypto-class, ouroboros-consensus)
After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: base, ouroboros-consensus, vector, cardano-crypto-class
Try running with --minimize-conflict-set to improve the error message.

mpickering commented 3 weeks ago

I can't reproduce this on NixOS.

To reproduce I entered the shell.nix using nix-shell shell.nix after setting up the binary cache.

Then cabal build -O1 -w $(which ghc-9.8.2) --semaphore all, and observed that the call to ghc was passed the semaphore.

Perhaps something to do with WSL?

mpickering commented 3 weeks ago

Results from the repo that @jasagredo tried the flag on:

Benchmark 1: cabal build -w /nix/store/kyh03hm4ni2d5rhydlsgcv88c1vdlylg-ghc-9.8.2/bin/ghc-9.8.2 -O1 -j all
  Time (mean ± σ):     186.922 s ±  5.796 s    [User: 468.499 s, System: 54.065 s]
  Range (min … max):   180.807 s … 197.243 s    10 runs

Benchmark 2: cabal build -O1 -w /nix/store/kyh03hm4ni2d5rhydlsgcv88c1vdlylg-ghc-9.8.2/bin/ghc-9.8.2 --semaphore all
  Time (mean ± σ):     161.768 s ±  7.623 s    [User: 564.039 s, System: 62.299 s]
  Range (min … max):   151.206 s … 173.469 s    10 runs

jasagredo commented 3 weeks ago

Confirming whether this is a WSL thing would be nice. I might try on a full Ubuntu I have around.

jasagredo commented 3 weeks ago

I just checked in my full Ubuntu, and I get the same error, cabal_semaphore_25: semOpen: does not exist (No such file or directory). See this pastebin (the output is a bit long) https://pastebin.com/y0XPsAMT

mpickering commented 3 weeks ago

@jasagredo Does this happen when building other projects as well? If I can reproduce in a docker container that would be good.

jasagredo commented 3 weeks ago

I just checked now with ouroboros-network which I had around and it also fails: https://pastebin.com/uruqaihZ

cloudyluna commented 3 weeks ago

Hello. I'm also hitting this error on my system when trying to build a simple project generated with cabal init and relude as an extra dependency.

System information

OS: GNU/Linux OpenSUSE 15.5
Arch: x86_64
WSL or native: native
Haskell tools installed and used with: ghcup

ghcup version:

$ ghcup --version
The GHCup Haskell installer, version 0.1.22.0

ghc version:

$ ghc --version
The Glorious Glasgow Haskell Compilation System, version 9.8.2

cabal version:

$ cabal --version
cabal-install version 3.11.0.0
compiled using version 3.12.0.0 of the Cabal library

libc version:

$ ldd --version
ldd (GNU libc) 2.31
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.

uname:

$ uname -a
Linux suse 5.14.21-150500.55.52-default #1 SMP PREEMPT_DYNAMIC Tue Mar 5 16:53:41 UTC 2024 (a62851f) x86_64 x86_64 x86_64 GNU/Linux

To reproduce

Run these commands in order.

git clone https://github.com/cloudyluna/simple-semaphore-build
cd simple-semaphore-build
cabal build all --semaphore -v

Build log (failed with error)

https://gist.github.com/cloudyluna/b13e7d7569ffa3ad1804ba7ec76813b7

haskell / cabal