ploxiln commented 5 years ago

[x] are reporting a bug others will be able to reproduce and not asking a question. If you're not sure or want to ask a question do so on our Discourse: https://discourse.brew.sh
[ ] ran a brew command and reproduced the problem with multiple formulae? If it's a problem with a single, official formula (not cask) please file this issue at Homebrew/homebrew-core: https://github.com/Homebrew/homebrew-core/issues/new/choose. If it's a brew cask problem please file this issue at https://github.com/Homebrew/homebrew-cask/issues/new/choose. If it's a tap (e.g. Homebrew/homebrew-php) problem please file this issue at the tap.
[x] ran brew update and can still reproduce the problem?
[x] ran brew doctor, fixed all issues and can still reproduce the problem?
[x] ran brew config and brew doctor and included their output with your issue?

I only reproduced with libpng, but this seems to me like an issue with the core framework, or something that should be detected in the core framework.

What you were trying to do (and why)

upgrade libpng ... from source ... I know you guys seem to not like that anymore, but I started with gentoo about 15 years ago ...

What happened (include command output)

libpng upgrade worked, then homebrew hit an apparently infinite stack recursion while checking dependents

==> make install
...
/usr/local/Cellar/libpng/1.6.36: 27 files, 1.2MB, built in 1 minute 33 seconds
Error: stack level too deep
/usr/local/Homebrew/Library/Homebrew/options.rb:26:in `hash'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/set.rb:313:in `add'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/set.rb:388:in `block in merge'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/set.rb:96:in `each'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/set.rb:96:in `each_entry'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/set.rb:96:in `do_with_enum'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/set.rb:388:in `merge'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/set.rb:90:in `initialize'
/usr/local/Homebrew/Library/Homebrew/options.rb:64:in `new'
/usr/local/Homebrew/Library/Homebrew/options.rb:64:in `initialize'
/usr/local/Homebrew/Library/Homebrew/options.rb:60:in `new'
/usr/local/Homebrew/Library/Homebrew/options.rb:60:in `create'
/usr/local/Homebrew/Library/Homebrew/tab.rb:256:in `used_options'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:355:in `from_keg'
/usr/local/Homebrew/Library/Homebrew/keg.rb:416:in `to_formula'
/usr/local/Homebrew/Library/Homebrew/cmd/upgrade.rb:200:in `map'
/usr/local/Homebrew/Library/Homebrew/cmd/upgrade.rb:200:in `block in upgradable_dependents'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/set.rb:306:in `each_key'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/set.rb:306:in `each'
/usr/local/Homebrew/Library/Homebrew/cmd/upgrade.rb:190:in `upgradable_dependents'
/usr/local/Homebrew/Library/Homebrew/cmd/upgrade.rb:217:in `block in upgradable_dependents'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/set.rb:306:in `each_key'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/set.rb:306:in `each'
/usr/local/Homebrew/Library/Homebrew/cmd/upgrade.rb:190:in `upgradable_dependents'
/usr/local/Homebrew/Library/Homebrew/cmd/upgrade.rb:217:in `block in upgradable_dependents'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/set.rb:306:in `each_key'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/set.rb:306:in `each'
/usr/local/Homebrew/Library/Homebrew/cmd/upgrade.rb:190:in `upgradable_dependents'
... repeats ...

What you expected to happen

Maybe nothing after the successful install, I expect libpng-1.6.36 to be fully ABI compatible with libpng-1.6.35, or maybe homebrew automatically re-builds all the formula which depend on libpng:

$ brew uses --installed --include-optional libpng | cat
cairo
freetype
gd
graphviz
gst-plugins-good
qrencode
webp

$ brew uses --installed libpng | cat
cairo
fontconfig
freetype
gd
gobject-introspection
graphviz
gst-libav
gst-plugins-base
gst-plugins-good
gst-plugins-ugly
gst-rtsp-server
harfbuzz
libass
libbluray
libsoup
mpv
pango
qrencode
vala
webp

interesting how --include-optional just avoids including transitive dependents in this case

Step-by-step reproduction instructions (by running `brew` commands)

I did this a couple times to check it wasn't just a fluke. I still have libpng-1.6.35 installed, so first:

$ brew uninstall --ignore-dependencies libpng
Uninstalling /usr/local/Cellar/libpng/1.6.36... (28 files, 1.2MB)
libpng 1.6.35 is still installed.
Remove all versions with `brew uninstall --force libpng`.

Then:

$ brew upgrade libpng
==> Upgrading 1 outdated package:
libpng 1.6.36
==> Upgrading libpng 
...

$ brew doctor
Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. Thanks!

Warning: You have HOMEBREW_BUILD_FROM_SOURCE set. This environment variable is
intended for use by Homebrew developers. If you are encountering errors,
please try unsetting this. Please do not file issues if you encounter
errors when using this environment variable.

Warning: Some installed formulae are missing dependencies.
You should `brew install` the missing dependencies:
  brew install p11-kit

Run `brew missing` for more details.

$ brew config
HOMEBREW_VERSION: 1.8.6-14-ga2b4d4a
ORIGIN: https://github.com/Homebrew/brew.git
HEAD: a2b4d4aac294ec0872cd84d44f65bd5bd507e166
Last commit: 3 days ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: a4af237fa6836920fdec9d9c8a3c14260123ac11
Core tap last commit: 70 minutes ago
HOMEBREW_PREFIX: /usr/local
HOMEBREW_BUILD_FROM_SOURCE: 1
HOMEBREW_DEV_CMD_RUN: 1
HOMEBREW_NO_ANALYTICS: 1
HOMEBREW_NO_AUTO_UPDATE: 1
HOMEBREW_NO_EMOJI: 1
CPU: quad-core 64-bit haswell
Homebrew Ruby: 2.3.7 => /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/bin/ruby
Clang: 10.0 build 1000
Git: 2.20.1 => /usr/local/bin/git
Curl: 7.54.0 => /usr/bin/curl
Java: 1.8.0_192
macOS: 10.13.6-x86_64
CLT: 10.1.0.0.1.1539992718
Xcode: N/A
XQuartz: 2.7.11 => /opt/X11

I was in the process of upgrading stuff:

$ brew outdated
awscli (1.16.60) < 1.16.70
bison (3.2.2) < 3.2.3
cmake (3.13.1) < 3.13.2
fdk-aac (0.1.6) < 2.0.0
ffmpeg (4.1) < 4.1_1
glib (2.58.1) < 2.58.2
gnu-sed (4.5) < 4.6
gobject-introspection (1.58.1) < 1.58.2
graphite2 (1.3.12) < 1.3.13
gst-plugins-ugly (1.14.4_1) < 1.14.4_2
harfbuzz (2.1.3) < 2.2.0
icu4c (62.1) < 63.1
media-info (18.08.1) < 18.12
meson (0.48.2) < 0.49.0
node@8 (8.12.0) < 8.14.1
opam (2.0.1) < 2.0.2
pixman (0.34.0_1) < 0.36.0
x264 (r2854) < r2917

ploxiln commented 5 years ago

I hacked in a print to get some clue about the formulae causing this loop:

--- a/Library/Homebrew/cmd/upgrade.rb
+++ b/Library/Homebrew/cmd/upgrade.rb
@@ -186,12 +186,13 @@ module Homebrew
   def upgradable_dependents(kegs, formulae)
     formulae_to_upgrade = Set.new
     formulae_pinned = Set.new

     formulae.each do |formula|
       descendants = Set.new
+      puts "considering formula: #{formula.full_name}"

       dependents = kegs.select do |keg|
         keg.runtime_dependencies

the output is confusing because this is a recursion inside a loop, but it eventually did come to a consistent repeating pattern:

...
considering formula: gst-rtsp-server
considering formula: gst-rtsp-server
considering formula: gst-plugins-bad
considering formula: gst-plugins-base
considering formula: gst-libav
considering formula: gst-plugins-bad
considering formula: gst-plugins-good
considering formula: gst-rtsp-server
considering formula: gst-plugins-good
considering formula: gst-rtsp-server
considering formula: webp
considering formula: libtiff
considering formula: little-cms2
considering formula: emacs
considering formula: mpv
considering formula: libsoup
considering formula: gst-plugins-good
considering formula: ghostscript
considering formula: webp
considering formula: libtiff
considering formula: little-cms2
considering formula: emacs
considering formula: mpv
considering formula: libsoup
considering formula: gst-plugins-good
considering formula: ghostscript
considering formula: webp
considering formula: libtiff
considering formula: little-cms2
considering formula: emacs
considering formula: mpv
considering formula: libsoup
considering formula: gst-plugins-good
considering formula: ghostscript
considering formula: webp
considering formula: libtiff
considering formula: little-cms2
considering formula: emacs
considering formula: mpv
... repeats ...

ploxiln commented 5 years ago

I think an interesting question is: how did libtiff get into that output ...

$ brew info libtiff
libtiff: stable 4.0.10 (bottled)
TIFF library and utilities
http://libtiff.maptools.org/
/usr/local/Cellar/libtiff/4.0.10 (246 files, 3.5MB) *
  Built from source on 2018-11-28 at 21:06:13
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/libtiff.rb
==> Dependencies
Required: jpeg (installed)
Optional: xz (installed)

MikeMcQuaid commented 5 years ago

Sorry, given this affects just you we'll accept a pull request for a fix or would need to see this reproducible with no brew doctor warnings. Will reopen if others experience this.

ploxiln commented 5 years ago

eh well it's just HOMEBREW_BUILD_FROM_SOURCE, the "p11-kit" one is erroneous:

$ brew info gnutls
gnutls: stable 3.5.19 (bottled)
GNU Transport Layer Security (TLS) Library
https://gnutls.org/
/usr/local/Cellar/gnutls/3.5.19 (1,104 files, 7.3MB) *
  Built from source on 2018-07-19 at 00:39:41 with: --without-p11-kit
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/gnutls.rb
==> Dependencies
Build: pkg-config (installed)
Required: gmp (installed), libtasn1 (installed), libunistring (installed), nettle (installed)
Recommended: p11-kit (uninstalled)
Optional: guile (uninstalled), unbound (uninstalled)
==> Options
--with-guile
    Build with guile support
--with-unbound
    Build with unbound support
--without-p11-kit
    Build without p11-kit support

Frizlab commented 5 years ago

I have the same issue with a simple brew upgrade. The upgrade command declared upgrading ffmpeg 4.1 -> 4.1_1, rust 1.31.0 -> 1.31.1, gnu-sed 4.5 -> 4.7, libssh 0.8.4 -> 0.8.5, gnutls 3.5.19 -> 3.6.5.

I have the following installed:

adns            ffmpeg          graphite2       libgcrypt       libvo-aacenc        opencore-amr        readline        theora
asciidoc        ffmpeg@2.8      gti         libgpg-error        libvorbis       openh264        rlwrap          tree
autoconf        flac            gtk-doc         libgsm          libvpx          openjpeg        rsync           two-lame
autoconf-archive    fontconfig      harfbuzz        libidn          libxml2         openssl         rtmpdump        unrar
automake        freetype        htop            libidn2         libyaml         openvpn         rubberband      util-linux
bat         frei0r          httpie          libksba         little-cms2     opus            ruby            vapor
boost           fribidi         hugo            libmodplug      lz4         p11-kit         rust            watch
cabextract      game-music-emu      icu4c           libogg          lzo         p7zip           sdl         wavpack
cairo           gdbm            intltool        libpng          makedepend      pango           sdl2            webp
carthage        gdk-pixbuf      itstool         libressl        meson-internal      pass            sl          wget
chromaprint     gettext         jpeg            librsvg         mpg123          pcre            snappy          wine
cimg            git-lfs         jq          libsamplerate       nasm            phash           source-highlight    winetricks
cloc            glib            lame            libsndfile      ncdu            pigz            speedtest-cli       x264
cmake           gmp         leptonica       libsoxr         nettle          pinentry        speex           x265
coreutils       gnu-getopt      lftp            libssh          ninja           pixman          sphinx-doc      xmlto
ctls            gnu-sed         libass          libssh2         nmap            pkg-config      sqlite          xvid
cython          gnupg           libassuan       libtasn1        node            postgresql      swiftlint       xz
dep         gnutls          libbs2b         libtiff         npth            pstree          swig            yasm
docbook         go          libcaca         libtool         ocaml           pv          tcl-tk          youtube-dl
docbook-xsl     gobject-introspection   libcroco        libunistring        ocaml-findlib       python          tesseract       zeromq
doxygen         gpac            libdvdcss       libusb          ocamlbuild      python@2        texi2html       zimg
fdk-aac         gpgme           libffi          libvidstab      oniguruma       qrencode        the_silver_searcher

I also have the HOMEBREW_BUILD_FROM_SOURCE var set to 1.

MikeMcQuaid commented 5 years ago

I also have the HOMEBREW_BUILD_FROM_SOURCE var set to 1.

As the docs note: that’s an unsupported configuration.

Frizlab commented 5 years ago

I got the issue on another computer with the var unset, just when compiling from source.

MikeMcQuaid commented 5 years ago

@Frizlab If someone can provide a simple reproduction without HOMEBREW_BUILD_FROM_SOURCE set we've got something we can look at fixing.

MikeMcQuaid commented 5 years ago

Additionally we need a fuller backtrace in a Gist, thanks.

Frizlab commented 5 years ago

Will try and do that when/if I find the time.

Homebrew / brew

infinite stack recursion in upgradable_dependents #5431

What you were trying to do (and why)

What happened (include command output)

What you expected to happen

Step-by-step reproduction instructions (by running `brew` commands)

Homebrew / brew

infinite stack recursion in upgradable_dependents #5431

What you were trying to do (and why)

What happened (include command output)

What you expected to happen

Step-by-step reproduction instructions (by running brew commands)

Step-by-step reproduction instructions (by running `brew` commands)