haskell / cabal

Official upstream development repository for Cabal and cabal-install
https://haskell.org/cabal
Other
1.6k stars 690 forks source link

`pkgconf`-provided `pkg-config` does `--modversion` differently; dependency resolution fails mysteriously #8923

Closed jashank closed 9 months ago

jashank commented 1 year ago

Describe the bug

On a system where pkgconf provides pkg-config, Cabal incorrectly reports dependency resolution failures. It turns out (modulo a bug in pkgconf), pkgconf --modversion only reports information about one package at a time, whereas Cabal happily tries the lot: https://github.com/haskell/cabal/blob/3af1731c01c35614fd902ee5c1aec40f5587fde6/cabal-install-solver/src/Distribution/Solver/Types/PkgConfigDb.hs#L70-L72

(Whether that pkgconf behaviour is correct is another question, and one I'm certainly not qualified to answer.)

To Reproduce

(I use Pango as an example here, because it's where I started; however, this affects pretty much any package that has a pkg-config dependency.)

% pkg-config --about | head -n1
pkgconf 1.9.4
% pkg-config --modversion pango
1.50.14
% cabal v2-install --lib pango
Resolving dependencies...
Error: cabal: Could not resolve dependencies:
[__0] next goal: pango (user goal)
[__0] rejecting: pango-0.13.8.2, pango-0.13.8.1, pango-0.13.8.0,
pango-0.13.6.1, pango-0.13.6.0, pango-0.13.5.0, pango-0.13.4.0,
pango-0.13.3.1, pango-0.13.3.0, pango-0.13.2.0, pango-0.13.1.1,
pango-0.13.1.0, pango-0.13.0.5, pango-0.13.0.4, pango-0.13.0.3,
pango-0.13.0.2, pango-0.13.0.1, pango-0.13.0.0, pango-0.12.5.3,
pango-0.12.5.0, pango-0.12.4, pango-0.12.3, pango-0.12.2, pango-0.12.1,
pango-0.12.0, pango-0.11.2, pango-0.11.1, pango-0.11.0 (conflict: pkg-config
package pango>=1.0, not found in the pkg-config database)
[__0] fail (backjumping, conflict set: pango)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: pango
`cabal v2-install --verbose --lib pango`

```shell-session % cabal v2-install --verbose --lib pango Reading available packages of hackage.haskell.org... Using most recent state specified from most recent cabal update index-state(hackage.haskell.org) = 2023-04-27T03:20:47Z Running: /home/jashank/bin/local/ghc --numeric-version looking for tool ghc-pkg near compiler in /home/jashank/bin/local found ghc-pkg in /home/jashank/.ghcup/ghc/9.6.1/bin/ghc-pkg-9.6.1 Running: /home/jashank/.ghcup/ghc/9.6.1/bin/ghc-pkg-9.6.1 --version Running: /home/jashank/bin/local/ghc --supported-languages Running: /home/jashank/bin/local/ghc --info creating /tmp/jashank/cabal-install.-1289081/dist-newstyle creating /tmp/jashank/cabal-install.-1289081/dist-newstyle/cache Compiler settings changed, reconfiguring... Running: /home/jashank/bin/local/ghc --numeric-version looking for tool ghc-pkg near compiler in /home/jashank/bin/local found ghc-pkg in /home/jashank/.ghcup/ghc/9.6.1/bin/ghc-pkg-9.6.1 Running: /home/jashank/.ghcup/ghc/9.6.1/bin/ghc-pkg-9.6.1 --version Running: /home/jashank/bin/local/ghc --supported-languages Running: /home/jashank/bin/local/ghc --info Running: /home/jashank/bin/local/ghc --print-global-package-db Reading available packages of hackage.haskell.org... Using most recent state specified from most recent cabal update index-state(hackage.haskell.org) = 2023-04-27T03:20:47Z Running: /usr/bin/pkg-config --version Running: /usr/bin/pkg-config --variable pc_path pkg-config Running: /usr/bin/pkg-config --version Running: /usr/bin/pkg-config --list-all Running: /usr/bin/pkg-config --modversion guile-3.0 libtcmalloc_debug polyml xcb-renderutil 'flac++' bullet libtasn1 freetype2 libxslt harfbuzz-subset edje panelw libpci wayland-cursor cairo-tee pamc xcb-icccm gmpxx webkit2gtk-4.0 orc-0.4 eo-cxx pangoot libidn2 fftw3q ector wayland-client gobject-2.0 xcb-aux efreet-mime ecore-wl2 gmodule-2.0 kadm-client fftw3f libsharpyuv xcb-damage fontconfig gdk-x11-3.0 xtst yajl bdw-gc libunwind-coredump eio-cxx libnl-route-3.0 xpm libselinux kdb libevent_extra xcb-util readline pangofc fftw3 libjpeg harfbuzz ecore-sdl tic tk xkbcommon-x11 xcb-record libwebpdecoder libpcre2-16 cddlib evas-cxx openssl libout123 expat libnl-nf-3.0 vips-cpp cairo-fc liblzma libpcre efl-ui xcb-dri3 evas xcb-keysyms m17n-shell efl-canvas-wl atf-sh sysprof-capture-4 PyImath hogweed python-3.11 xcb-randr kadm-server xcb-render xcb-xkb xfixes gsl z3 libcrypto libbrotlienc libsasl2 harfbuzz-icu jansson libpcre2-posix libnl-genl-3.0 xcb-sync wayland-server gobject-introspection-no-export-1.0 libxcrypt libgsasl elementary libstartup-notification-1.0 m17n-core xcb-xrm atspi-2 xcursor x11 ldap pixman-1 fribidi 'gtk+-broadway-3.0' libexslt vorbisfile lber jemalloc gl libuv ecore-x libbsd-overlay libprofiler xcb-dri2 poppler-glib cairo-script xcb-xinerama libunwind-ptrace libpcreposix libattr libffi cairo-ft libpulse-mainloop-glib cairo-xcb cairo-xlib blkid gmodule-no-export-2.0 efl-cxx glesv2 matio cairo-xlib-xrender harfbuzz-cairo atf-c xmuu libgsf-1 epoxy krb5-gssapi xau libidn libpng mit-krb5 eldbus-cxx libunwind verilator xcb-atom libpulse libusb-1.0 menu libevent xaw3d 'gtk+-3.0' xft audit xdamage eina-cxx libnl-idiag-3.0 xaw7 cairo-gobject pango mit-krb5-gssapi libpcre32 uuid ecore-ipc gmp guile-2.2 sqlite3 wayland-scanner 'ncurses++w' avahi-libevent libgit2 gssrpc icu-uc libacl m17n-flt libcrypt libssl gtk4-wayland 'atf-c++' efreet xt gdk-pixbuf-2.0 gtk4-x11 cairo-pdf python zlib xcb-event sm ethumb-client ecore-cxx mpfr m17n-gui xrender python3-embed xscrnsaver tcl ecore-imf-evas krb5 gpg-error libtcmalloc atk-bridge-2.0 graphene-gobject-1.0 xcb-cursor alsa avahi-core libnl-cli-3.0 cairo-script-interpreter ecore-imf gdk-broadway-3.0 xkbfile libpcrecpp libxml-2.0 opus xcb-glx xapian-core elput ecore-evas luajit xcb-xfixes libmpg123 libseccomp libavif libsystemd 'gtk+-x11-3.0' ogg pangoft2 efreet-trash opengl eio glesv1_cm xcb-screensaver libglvnd xrandr ecore xcb-xv ice libbrotlicommon xcb-image libtirpc gmodule-export-2.0 cairo-png xcb-xinput libwebpdemux xext nettle libwebp wayland-egl libgcrypt libpcre16 cairo-ps xcb-shape cloudproviders gtk4-unix-print xcb-res libunwind-generic libpcre2-8 fwupd-efi libsoup-2.4 egl libpulse-simple gdk-wayland-3.0 yaml-0.1 libsyn123 libopenjp2 cairo gobject-introspection-1.0 xcb-present liblz4 libcap-ng ecore-input-evas xcb-dpms pangocairo eo ecore-fb elua ncursesw xcb-composite xcb-xvmc xinerama gdlib cgif graphite2 gnutls-dane libcurl xcb-xselinux atk libwebpmux OpenEXR webkit2gtk-web-extension-4.0 sndfile xi gdk-3.0 libsepol re2 ethumb emile libtiff-4 valgrind eolian libmd xcb-ewmh cairo-xcb-shm flac gnutls libevent_openssl libmariadb gio-unix-2.0 lmdb sdl2 edje-cxx python-3.11-embed libinput eet emotion atomic_ops libevent_core gtk4 ecore-drm2 ephysics libbsd dbus-1 imagequant libnl-3.0 com_err ruby eldbus libpsl pam libverto efl-core tinfo efl-net libtcmalloc_minimal_debug ecore-file gthread-2.0 ecore-con fftw3l gmodule libpng16 eeze librsvg-2.0 'gtk+-wayland-3.0' panel vorbisenc spng eina cairo-svg glib-2.0 libkeyutils python2 ecore-input menuw orc-test-0.4 xcb-xtest harfbuzz-gobject graphene-1.0 libelf xdmcp eolian-cxx python3 javascriptcoregtk-4.0 bzip2 vorbis x11-xcb pangoxft libperf libpq auparse 'gtk+-unix-print-3.0' libarchive pthread-stubs hunspell icu-i18n glib libevent_pthreads eet-cxx libotf icu-io gio-2.0 xcb-xf86dri gtk4-broadway vips embryo 'ncurses++' tree-sitter alsa-topology lcms2 ethumb_client gthread glx poppler xcomposite ncurses libsoup-gnome-2.4 p11-kit-1 mount libedit libpcre2-32 history xcb ecore-buffer libzstd ecore-avahi avahi-client ecore-audio xkbcommon gnome-todo libexif libthai python-2.7 wayland-egl-backend fuse xkbregistry elementary-cxx formw libtcmalloc_minimal Imath pam_misc libudev datrie-0.2 gail-3.0 efl form cfitsio xcb-shm libnl-xfrm-3.0 libbrotlidec xmu compositeproto dri2proto videoproto shared-mime-info fixesproto xcb-proto udev xcmiscproto xf86dgaproto emacs fontsproto chemical-mime-data dri3proto resourceproto dmxproto randrproto recordproto xbitmaps xproto damageproto kbproto dpmsproto xextproto bash-completion dracut xineramaproto inputproto xwaylandproto glproto xtrans bigreqsproto xf86bigfontproto renderproto scrnsaverproto xorg-macros presentproto xf86vidmodeproto bodr xf86driproto eigen3 systemd Resolving dependencies... CallStack (from HasCallStack): withMetadata, called at src/Distribution/Simple/Utils.hs:368:14 in Cabal-3.10.1.0-inplace:Distribution.Simple.Utils Error: cabal: Could not resolve dependencies: [__0] next goal: pango (user goal) [__0] rejecting: pango-0.13.8.2, pango-0.13.8.1, pango-0.13.8.0, pango-0.13.6.1, pango-0.13.6.0, pango-0.13.5.0, pango-0.13.4.0, pango-0.13.3.1, pango-0.13.3.0, pango-0.13.2.0, pango-0.13.1.1, pango-0.13.1.0, pango-0.13.0.5, pango-0.13.0.4, pango-0.13.0.3, pango-0.13.0.2, pango-0.13.0.1, pango-0.13.0.0, pango-0.12.5.3, pango-0.12.5.0, pango-0.12.4, pango-0.12.3, pango-0.12.2, pango-0.12.1, pango-0.12.0, pango-0.11.2, pango-0.11.1, pango-0.11.0 (conflict: pkg-config package pango>=1.0, not found in the pkg-config database) [__0] fail (backjumping, conflict set: pango) After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: pango (29) ```

Expected behavior

Packages with pkg-config dependencies, such as Pango, work.

System information

% lsb_release -d
Description:    Fedora release 39 (Rawhide)
% cabal --version
cabal-install version 3.10.1.0
compiled using version 3.10.1.0 of the Cabal library 
% ghc --version
The Glorious Glasgow Haskell Compilation System, version 9.6.1
% ghcup --version
The GHCup Haskell installer, version 0.1.19.2

Additional context

Dretch commented 1 year ago

I have worked around this issue for my use case by wrapping pkg-config with a script that fails when given more than one argument, so triggering the fallback code path in cabal that calls pkg-config once for each package:

#!/bin/bash
# workaround https://github.com/haskell/cabal/issues/8923 by making pkg-config fail
# when given more than one package, instead of silently using only the first package 
mkdir -p pkg-config-hack
cat <<'EOF' > pkg-config-hack/pkg-config
#!/bin/bash
if [ "$1" == "--modversion" ] && [ "$2" != "" ] && [ "$3" != "" ]; then
  exit 1
fi
exec /usr/bin/pkg-config $@
EOF
chmod +x pkg-config-hack/pkg-config
export PATH=$(pwd)/pkg-config-hack:$PATH
juhp commented 1 year ago

(I believe this was actually caused by a change of behavior of pkgconf-1.9 relative to 1.8 which still supported outputting many package versions (though admittedly that use-case feature made less sense.)

juhp commented 1 year ago

I think the safest would be just to run pkgconf individually on each pkg always? (though reading all pkgs makes less sense anyway)

juhp commented 10 months ago

Fortunately the behavior seems reverted in pkgconf-2.0, but many distros are still on 1.8 or 1.9...

juhp commented 10 months ago

As a first step I applied this simple patch to Fedora cabal-install-3.8.1 - if anyone else wants this.

vdukhovni commented 8 months ago

I have worked around this issue for my use case by wrapping pkg-config with a script that fails when given more than one argument, so triggering the fallback code path in cabal that calls pkg-config once for each package:

Thanks for that, though the fix for cabal 3.10 has been committed, it is not yet released, so I have to resort to the work-around. An improved version below (simpler and with correct quoting of $@).

#! /bin/bash
[[ $# < 3 || $1 != "--modversion" ]] && exec /usr/bin/pkg-config "$@"
juhp commented 1 month ago

(Just confirming that this fix is in cabal-install-solver-3.10.3.0 (released 21st March), thank you -- hopefully also in the next lts-22...;-)