Closed deepfire closed 6 years ago
deepfire
commented
6 years ago As an addition, the reproduction can be facilitated using the following Nixpkgs commit: https://github.com/NixOS/nixpkgs/pull/36781/commits/2c5d8e63a384bc7fcfa2aecd94ce355f10814180
hvr
commented
6 years ago @deepfire if it's easy for you, could you run cabal w/ the cabal +RTS -hT -RTS flags to generate a heap profile (works with the standard RTS; no need for the profiling RTS), and graph that via hp2ps or some other tool and attach a plot here?
deepfire
commented
6 years ago @hvr, both the profile and the rendered PS are at: https://mega.nz/#F!XJpzgKgZ!q9QXfRkPYgg9B-wAOcbRuw
hvr
commented
6 years ago @deepfire thx... so what's apparent to me is there's at the very least a serious space-leak building up; but that's possibly just an accidental additional issue exposed by this.
I suspect something some loop or infinite data (maybe stdout data?) accumulation thing to be going on.
deepfire
commented
6 years ago The unabridged strace of cabal configure, up to the point of its death at the hands of the OOM killer is also available there.
deepfire
commented
6 years ago Interestingly, the final file reads of the process were:
[pid 26705] read(16, "cabal-install/1.22.4.0/cabal-install.cabal0000006440000045230000000000000000021077001252221670500020677 0ustar0000ryantHackage000000000000000000Name: cabal-install\nVersion: 1.22.4.0\nSynopsis: The command-line interface for Cabal and Hackage.\nDescription:\n The \\'cabal\\' command-line program simplifies the process of managing\n Haskell software by automating the fetching, configuration, compilation\n and installation of Haskell libraries and programs.\nhomepage: http://www.haskell.org/cabal/\nbug-reports: https://github.com/haskell/cabal/issues\nLicense: BSD3\nLicense-File: LICEN"..., 8192) = 8192
[pid 26705] read(16, ": network-uri < 2.6, network < 2.6\n\n if os(windows)\n build-depends: Win32\n cpp-options: -DWIN32\n else\n build-depends: unix\n\n if arch(arm)\n cc-options: -DCABAL_NO_THREADED\n else\n ghc-options: -threaded\n default-language: Haskell2010\n\n-- Large, system tests that build packages.\ntest-suite package-tests\n type: exitcode-stdio-1.0\n hs-source-dirs: tests\n main-is: PackageTests.hs\n other-modules:\n PackageTests.Exec.Check\n PackageTests.Freeze.Check\n PackageTests.MultipleSource.Check\n PackageTests.PackageTester\n build-depends:\n Cabal,\n HUnit,\n QuickCheck >= 2.1.0.1 && < 2.8,\n base,\n bytestring,\n directory,\n extensible-exceptions,\n filepath,\n process,\n regex-posix,\n test-framework,\n test-framework-hunit,\n test-framework-quickcheck2 >= 0.2.12\n\n if os(windows)\n build-depends: Win32 >= 2 && < 3\n cpp-options: -DWIN32\n else\n build-depends: unix >= 2.0 && < 2.8\n\n if arch(arm)\n cc-options: -DCABAL_NO_THREADED\n else\n ghc-options: -"..., 8192) = 8192
[pid 26705] read(16, "cabal-install/1.20.1.0/cabal-install.cabal0000006440000045230000000000000000016526001252222102600020665 0ustar0000ryantHackage000000000000000000Name: cabal-install\nVersion: 1.20.1.0\nSynopsis: The command-line interface for Cabal and Hackage.\nDescription:\n The \\'cabal\\' command-line program simplifies the process of managing\n Haskell software by automating the fetching, configuration, compilation\n and installation of Haskell libraries and programs.\nhomepage: http://www.haskell.org/cabal/\nbug-reports: https://github.com/haskell/cabal/issues\nLicense: BSD3\nLicense-File: LICEN"..., 8192) = 8192cabal-install reading information about itself, then OOMing.. just correlation or something bigger?
deepfire
commented
6 years ago ..and the FD being read is:
[pid 26705] openat(AT_FDCWD, "/home/deepfire/.cabal/packages/hackage.haskell.org/01-index.tar", O_RDONLY|O_NOCTTY|O_NONBLOCK) = 16Fwiw, I did try to reproduce this locally w/ cabal get data-default-0.7.1.1 and
$ PATH=/opt/ghc/8.4.1/bin:$PATH /opt/cabal/2.2/bin/cabal configure -v -v -v -v +RTS -s
/opt/ghc/8.4.1/bin/ghc --numeric-version
looking for tool ghc-pkg near compiler in /opt/ghc/8.4.1/bin
found ghc-pkg in /opt/ghc/8.4.1/bin/ghc-pkg
/opt/ghc/8.4.1/bin/ghc-pkg --version
/opt/ghc/8.4.1/bin/ghc --supported-languages
/opt/ghc/8.4.1/bin/ghc --info
Reading available packages of hackage.haskell.org...
Using most recent state specified from most recent cabal update
index-state(hackage.haskell.org) = 2018-03-11T11:51:23Z
/usr/bin/pkg-config --version
/usr/bin/pkg-config --list-all
/usr/bin/pkg-config --modversion sqlite3 shared-mime-info blkid libfreenect pixman-1 iso-codes gmodule-export-2.0 libevent_pthreads xaw7 gstreamer-interfaces-0.10 cairo-svg libdrm_intel recordproto nettle gnutls libpcreposix lapack-netlib lqr-1 atk-bridge-2.0 m17n-gui libelf cairo-tee xcb-render dbus-1 e2p dri fftw3 xkbcommon dri3proto libzmq udisks systemd gnome-icon-theme-symbolic libbsd SDL_gfx wayland-cursor gstreamer-pbutils-0.10 libcurl pangoxft QtDBus xcursor cairo-png xft libidn adwaita-icon-theme libxslt glu 'gtk+-x11-3.0' gstreamer-rtp-0.10 yaml-0.1 libexif gdk-pixbuf-2.0 xf86vidmodeproto gtksourceview-3.0 xtrans mircookie QtScriptTools lapack caca gnome-system-tools damageproto QtDeclarative libpulse hidapi-libusb QtXml expat videoproto cairo-xlib mit-krb5-gssapi libavformat gstreamer-netbuffer-0.10 uuid valgrind banshee-lastfm-gui fontutil xcb-dri2 xcb-dri3 panel swipl gconf-2.0 kbproto randrproto pygoocanvas libxdot openal mobile-broadband-provider-info xrandr libgssglue 'Magick++' dleyna-server-1.0 wayland-egl bash-completion QtGui gdk-2.0 libraw1394 capi20 xcomposite sdl libtasn1 dri2proto QtScript OpenEXR gstreamer-riff-0.10 icu-i18n libgphoto2 datrie-0.2 QtDesigner sm libswresample-ffmpeg cairo-ps p11-kit-1 hexchat-plugin libpulse-mainloop-glib libcdt python gstreamer-check-0.10 gdk-mir-3.0 protobuf-lite python2 mirclient gnome-icon-theme gmodule-2.0 libavcodec ImageMagick gstreamer-dataprotocol-0.10 gdk-pixbuf-xlib-2.0 libdrm_radeon libdrm scrnsaverproto fixesproto SDL2_ttf libpulse-simple slang QtDesignerComponents sqlcipher pango banshee-mono-media m17n-flt geoclue-2.0 libgdiplus m17n-core flac libedit opencv pangoft2 libsodium atspi-2 gobject-introspection-no-export-1.0 gdk-3.0 xorg-sgml-doctools sane-backends compositeproto panelw libbsd-ctor SDL2_image libsystemd xvmc xineramaproto icu-uc libpcre banshee-hyena libpcre16 ginn cairo-xcb-shm libthai javascriptcoregtk-3.0 glproto cairo-pdf gstreamer-sdp-0.10 libevent_openssl pangocairo gstreamer-app-0.10 lcms2 pm-utils gstreamer-0.10 gnome-mime-data-2.0 x11-xcb libpq gthread-2.0 xcb-glx 'gtk+-broadway-3.0' gstreamer-base-0.10 vdpau libssl libusb-1.0 libpcrecpp cairo-xcb IL gl libcrypto gstreamer-plugins-base-0.10 menu QtXmlPatterns Qt3Support MagickWand xorg-wacom mit-krb5 egl intel-gen4asm libdrm_nouveau alsa libspreadsheet-1.12 QtTest cairo-gobject vpx gstreamer-controller-0.10 libxml-2.0 hidapi-hidraw gmodule-no-export-2.0 libdrm_amdgpu bdw-gc x11 libsepol xtst libgvpr gstreamer-fft-0.10 gobject-introspection-1.0 ddjvuapi libevent libpcre32 wayland-client libpathplan QtCLucene QtCore gsl 'gtk+-unix-print-2.0' gstreamer-cdda-0.10 gdk-broadway-3.0 hogweed xmu sndfile mysqlclient poppler-data libexslt wayland-server libsoup-2.4 ibus-table 'gtk+-mir-3.0' xi 'libcrypto++' dleyna-server-service-1.0 banshee-services udev libR ILUT xextproto xt cairo-fc xv glib-2.0 cairo xorg-macros yelp-xsl SDL2_gfx Wand libswscale-ffmpeg gdk-wayland-3.0 icu-io cairo-ft banshee-hyena-data-sqlite xcb-sync banshee-lastfm gio-unix-2.0 MagickCore vorbisfile libswresample libdc1394-2 QtSql glesv2 libavutil-ffmpeg pygtksourceview-2.0 'ncurses++w' xfixes vorbisenc presentproto libquvi-scripts xcb-xfixes xbitmaps libdw 'gtk+-unix-print-3.0' banshee-thickclient tic sdl2 gstreamer-tag-0.10 xinerama SDL2_mixer libbsd-overlay tinfo tomboy-addins epoxy formw libpng m17n-shell usbutils xpm 'gtk+-wayland-3.0' gstreamer-rtsp-0.10 gdk-x11-2.0 blas-netlib xml2po librsvg-2.0 xshmfence xcb-randr IlmBase banshee-musicbrainz freetype2 xext libjpeg libudev libgphoto2_port libtiff-4 xau harfbuzz vorbis ice xdamage cairo-script libselinux fftw3l xkbcomp openssl ILU 'gtk+-2.0' xcb-present ncursesw dleyna-connector-dbus-1.0 fftw3f QtHelp atk icu-le libffi wayland-scanner xxf86vm xkeyboard-config gnome-doc-utils xcb-shape python-2.7 fftw3q harfbuzz-icu libavcodec-ffmpeg pthread-stubs banshee-collection-indexer 'caca++' gdlib xrender gstreamer-net-0.10 banshee-core SDL_ttf icu-lx inputproto 'ImageMagick++' libpng12 banshee-webbrowser xcb-shm xscrnsaver xproto protobuf libmpg123 mircore dbus-glib-1 xcb liblzma libswscale gdk-x11-3.0 libavutil gstreamer-audio-0.10 libpcsclite ogg banshee-hyena-gui gstreamer-video-0.10 ncurses form fontconfig libxfconf-0 dbus-python ruby-2.3 QtUiTools webkitgtk-3.0 cairo-xlib-xrender 'gtk+-3.0' banshee-nowplaying ext2fs harfbuzz-gobject gstreamer-floatcast-0.10 lxappearance zlib xfce4-session-2.0 libnotify com_err 'ncurses++' libcgraph menuw gobject-2.0 m17n-db ruby libotf gio-2.0 QtSvg d3d libgvc xdmcp 'gtk+-x11-2.0' QtNetwork blas notify-python sox libavformat-ffmpeg renderproto libclc
Resolving dependencies...
Warning: solver failed to find a solution:
Could not resolve dependencies:
[__0] trying: data-default-0.7.1.1 (user goal)
[__1] unknown package: data-default-instances-old-locale (dependency of
data-default)
[__1] fail (backjumping, conflict set: data-default,
data-default-instances-old-locale)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: data-default (2),
data-default-instances-old-locale (1)
Trying configure anyway.
Using internal setup method with build-type Simple and args:
["configure","--verbose=2","--ghc","--prefix=/home/hvr/.cabal","--user","--extra-prog-path=/home/hvr/.cabal/bin"]
Using Parsec parser
Configuring data-default-0.7.1.1...
CallStack (from HasCallStack):
die', called at ./Distribution/Simple/Configure.hs:958:20 in Cabal-2.2.0.0-FGQ6k6zzd10HxtpUMDQKJe:Distribution.Simple.Configure
configureFinalizedPackage, called at ./Distribution/Simple/Configure.hs:462:12 in Cabal-2.2.0.0-FGQ6k6zzd10HxtpUMDQKJe:Distribution.Simple.Configure
configure, called at ./Distribution/Simple.hs:596:20 in Cabal-2.2.0.0-FGQ6k6zzd10HxtpUMDQKJe:Distribution.Simple
confHook, called at ./Distribution/Simple/UserHooks.hs:67:5 in Cabal-2.2.0.0-FGQ6k6zzd10HxtpUMDQKJe:Distribution.Simple.UserHooks
configureAction, called at ./Distribution/Simple.hs:178:19 in Cabal-2.2.0.0-FGQ6k6zzd10HxtpUMDQKJe:Distribution.Simple
defaultMainHelper, called at ./Distribution/Simple.hs:120:19 in Cabal-2.2.0.0-FGQ6k6zzd10HxtpUMDQKJe:Distribution.Simple
defaultMainArgs, called at ./Distribution/Client/SetupWrapper.hs:420:29 in main:Distribution.Client.SetupWrapper
cabal: Encountered missing dependencies:
data-default-class >=0.1.2.0,
data-default-instances-containers -any,
data-default-instances-dlist -any,
data-default-instances-old-locale -any
792,359,824 bytes allocated in the heap
378,358,152 bytes copied during GC
45,241,120 bytes maximum residency (16 sample(s))
1,635,616 bytes maximum slop
114 MB total memory in use (0 MB lost due to fragmentation)
Tot time (elapsed) Avg pause Max pause
Gen 0 740 colls, 0 par 0.417s 0.417s 0.0006s 0.0795s
Gen 1 16 colls, 0 par 0.005s 0.005s 0.0003s 0.0006s
TASKS: 4 (1 bound, 3 peak workers (3 total), using -N1)
SPARKS: 0 (0 converted, 0 overflowed, 0 dud, 0 GC'd, 0 fizzled)
INIT time 0.001s ( 0.001s elapsed)
MUT time 0.351s ( 0.796s elapsed)
GC time 0.423s ( 0.422s elapsed)
EXIT time 0.005s ( 0.012s elapsed)
Total time 0.780s ( 1.230s elapsed)
Alloc rate 2,256,035,837 bytes per MUT second
Productivity 45.7% of total user, 65.6% of total elapsed
gc_alloc_block_sync: 0
whitehole_spin: 0
gen[0].sync: 0
gen[1].sync: 0Nor did cabal-install from master exhibit the issue... (I don't use Nix (yet), so this was the quickest thing I could do).
gbaz
commented
6 years ago Just to confirm, is this cabal-install built against the released ghc 8.4.1?
hvr
commented
6 years ago @gbaz mine wasn't (it was compiled w/ GHC 8.2.2)
...but I just tried w/ cabal HEAD + hackage-security HEAD + ghc-8.4.1
and your suspicion was spot on! Needs more investigation... :-/
/cc @23Skidoo
PS: repro instructions for building cabal HEAD w/ GHC 8.4.1
cabal get -s cabal-install
cd cabal-install
cabal get -s hackage-security
echo "packages: Cabal hackage-security/hackage-security cabal-install" > cabal.project
cabal new-build -w ghc-8.4.1 exe:cabalPS2: I semi-ruled out libraries other than pretty/template-haskell/ghc-prim/base being at fault by pinnding all other libs down to the same versions, and compiling exe:cabal with GHC 8.2.2 and GHC 8.4.1, which resulted in the former one working.
PS3: So the abnormal heap allocation w/ GHC 8.4.1 occurs in the https://github.com/haskell/cabal/blob/master/cabal-install/Distribution/Client/IndexUtils.hs#L685-L714 loop; but it's not obvious to me where. @bgamari suggests to add cost center/SCCs to figure this one out, or to look at the generated STG.
PS4: I won't be able to continue debugging this for a while; so any help is appreciated if somebody wants to take over!
PS5: This seems to point to a bug in the code generation or optimiser, c.f. https://ghc.haskell.org/trac/ghc/ticket/13930
deepfire
commented
6 years ago @gbaz, confirming, everything, including Cabal/cabal-install built with GHC 8.4.1-release -- as per commit in the issue header.
hvr
commented
6 years ago @23Skidoo I think we should close this one; the code triggering this compiler bug in GHC 8.4.1 was refactored to avoid it (and fwiw, the compiler bug was fixed in GHC 8.4.2 anyway); nor do we need to backport anything afaics
Likely a duplicate of #4589, but submitting as new, because the latter was closed and this has been half a year and a compiler release since..
cabal-installrevision:728ad1a1e066da453ae13ee479629c00d8c2f32dGHC 8.4.1 revision:0a3e2f324dbd525d626ebd3d97e8ffa1cf2f0ffb(release)Notes
/nix/store/k79gs1h0y3mvqj5rd8xvw1z6rnd8clap-ghc-8.4.1-with-packages/bin/ghc --infoinside the same directory is fine, and yields Output [1]Output
strace -f -eexecve -s1024 cabal -v -v -v -v configure: