Homebrew / homebrew-bundle

📦 Bundler for non-Ruby dependencies from Homebrew, Homebrew Cask and the Mac App Store.
MIT License
5.3k stars 285 forks source link

Topological sort failed #420

Closed benknoble closed 5 years ago

benknoble commented 5 years ago

Similar to #240, I want to run brew bundle dump:

Error: topological sort failed: ["libtiff", "webp"]
Formulae dependency graph sorting failed (likely due to a circular dependency)!

I'm trying to dump everything to a brewfile (for my Dotfiles setup); brew list | column -c 80 gives

ack         jpeg            python
archey          libevent        python3
autoconf        libmodplug      python@2
automake        libmpc          readline
bash            libogg          ruby
bash-completion@2   libpng          scala
brew-cask-completion    libtiff         sdl2
coreutils       libtool         sdl2_image
cowsay          libvorbis       sdl2_ttf
exiftool        libyaml         shellcheck
figlet          lolcat          sl
findutils       lua         smlnj
fortune         luajit          solarus
freetype        make            sphinx-doc
gcc         moreutils       sqlite
gdbm            mpfr            tmux
gettext         nethack         tree
git         openssl         universal-ctags
gitsh           parallel        unrar
gmp         pcre2           vim
gnu-sed         perl            webp
go          physfs          xz
heroku          pkg-config      zelda-roth-se
heroku-node     ponysay         zsdx
htop            proselint       zsxd
hub         pstree
isl         pv

I checked brew info libtiff webp and couldn't see any dependencies that might be circular... they do both depend on jpeg, but I wouldn't have expected that to be an issue.

colindean commented 5 years ago

I read up a little on tsort, the stdlib module that homebrew-bundle uses for topological sorting in order to product the dump output in a sensible order. That error message should indicate the packages that are cyclic, as you also believe.

As a first step, perhaps try brew reinstall libtiff webp to verify that you have valid installations of these packages.

benknoble commented 5 years ago

Will do and report back; thanks.

benknoble commented 5 years ago

Tried the reinstall. I still get the same errors.

MikeMcQuaid commented 5 years ago

Try to use brew deps --tree --installed to look for circular chains.

benknoble commented 5 years ago

I've done that, and will try again tonight, but I don't remember seeing any. Is there an easy way to spot them (i.e. not eyeball-scanning a long piece of output)?

MikeMcQuaid commented 5 years ago

@benknoble I'm not sure, sorry. We'll keep helping you debug here but closing as this isn't a widespread issue.

benknoble commented 5 years ago

@MikeMcQuaid

Unless I'm blind, I'm not seeing any circular dependencies here:

ack

archey

autoconf

automake
└── autoconf

bash

bash-completion@2
└── bash

brew-cask-completion

coreutils

cowsay

exiftool

figlet

findutils

fortune

freetype
└── libpng

gcc
├── gmp
├── isl
│   └── gmp
├── libmpc
│   ├── gmp
│   └── mpfr
│       └── gmp
└── mpfr
    └── gmp

gdbm

gettext

git
├── gettext
└── pcre2

thoughtbot/formulae/gitsh
└── readline

gmp

gnu-sed

go

heroku/brew/heroku
└── heroku/brew/heroku-node

heroku/brew/heroku-node

htop

hub

isl
└── gmp

jpeg

libevent
└── openssl

libmodplug

libmpc
├── gmp
└── mpfr
    └── gmp

libogg

libpng

libtiff
└── jpeg

libtool

libvorbis
└── libogg

libyaml

lolcat
└── ruby
    ├── libyaml
    ├── openssl
    └── readline

lua

luajit

make

moreutils

mpfr
└── gmp

nethack

openssl

parallel

pcre2

perl

physfs

pkg-config

ponysay
├── coreutils
└── python
    ├── gdbm
    ├── openssl
    ├── readline
    ├── sqlite
    │   └── readline
    └── xz

proselint
└── python
    ├── gdbm
    ├── openssl
    ├── readline
    ├── sqlite
    │   └── readline
    └── xz

pstree

pv

python
├── gdbm
├── openssl
├── readline
├── sqlite
│   └── readline
└── xz

python@2
├── gdbm
├── openssl
├── readline
└── sqlite
    └── readline

readline

ruby
├── libyaml
├── openssl
└── readline

scala

sdl2

sdl2_image
├── jpeg
├── libpng
├── libtiff
│   └── jpeg
├── sdl2
└── webp
    ├── jpeg
    └── libpng

sdl2_ttf
├── freetype
│   └── libpng
└── sdl2

shellcheck

sl

smlnj

solarus
├── libmodplug
├── libogg
├── libvorbis
│   └── libogg
├── luajit
├── physfs
├── sdl2
├── sdl2_image
│   ├── jpeg
│   ├── libpng
│   ├── libtiff
│   │   └── jpeg
│   ├── sdl2
│   └── webp
│       ├── jpeg
│       └── libpng
└── sdl2_ttf
    ├── freetype
    │   └── libpng
    └── sdl2

sphinx-doc

sqlite
└── readline

tmux
└── libevent
    └── openssl

tree

universal-ctags/universal-ctags/universal-ctags

unrar

vim
├── gettext
├── lua
├── perl
├── python
│   ├── gdbm
│   ├── openssl
│   ├── readline
│   ├── sqlite
│   │   └── readline
│   └── xz
└── ruby
    ├── libyaml
    ├── openssl
    └── readline

webp
├── jpeg
└── libpng

xz

zelda-roth-se
└── solarus
    ├── libmodplug
    ├── libogg
    ├── libvorbis
    │   └── libogg
    ├── luajit
    ├── physfs
    ├── sdl2
    ├── sdl2_image
    │   ├── jpeg
    │   ├── libpng
    │   ├── libtiff
    │   │   └── jpeg
    │   ├── sdl2
    │   └── webp
    │       ├── jpeg
    │       └── libpng
    └── sdl2_ttf
        ├── freetype
        │   └── libpng
        └── sdl2

zsdx
└── solarus
    ├── libmodplug
    ├── libogg
    ├── libvorbis
    │   └── libogg
    ├── luajit
    ├── physfs
    ├── sdl2
    ├── sdl2_image
    │   ├── jpeg
    │   ├── libpng
    │   ├── libtiff
    │   │   └── jpeg
    │   ├── sdl2
    │   └── webp
    │       ├── jpeg
    │       └── libpng
    └── sdl2_ttf
        ├── freetype
        │   └── libpng
        └── sdl2

zsxd
└── solarus
    ├── libmodplug
    ├── libogg
    ├── libvorbis
    │   └── libogg
    ├── luajit
    ├── physfs
    ├── sdl2
    ├── sdl2_image
    │   ├── jpeg
    │   ├── libpng
    │   ├── libtiff
    │   │   └── jpeg
    │   ├── sdl2
    │   └── webp
    │       ├── jpeg
    │       └── libpng
    └── sdl2_ttf
        ├── freetype
        │   └── libpng
        └── sdl2
benknoble commented 5 years ago

@MikeMcQuaid seems I'm not alone, might be worth re-opening this?

MikeMcQuaid commented 5 years ago

Need to try and find a minimal reproduction case to be able to reproduce this.

MikeMcQuaid commented 5 years ago

@benknoble @heptathlon can you brew update (so you have https://github.com/Homebrew/homebrew-bundle/pull/422) and then rerun brew bundle dump and paste the output here? It should be more enlightening now.

heptathlon commented 5 years ago

hi @MikeMcQuaid and thanks for your efforts. here is the output with the debug commit:

Error: Formulae dependency graph sorting failed (likely due to a circular dependency): ffmpeg: ["lame", "libogg", "libvorbis", "libvpx", "opus", "sdl2", "snappy", "theora", "x264", "x265", "xvid", "xz", "llvm"] x264: ["ffmpeg"]

MikeMcQuaid commented 5 years ago

@heptathlon You should run brew reinstall ffmpeg x264.

MikeMcQuaid commented 5 years ago

(I've updated the message in https://github.com/Homebrew/homebrew-bundle/pull/423 to suggest this automatically now)

heptathlon commented 5 years ago

Now I get a slightly different result: "llvm" is not listed anymore.

Error: Formulae dependency graph sorting failed (likely due to a circular dependency): ffmpeg: ["lame", "libogg", "libvorbis", "libvpx", "opus", "sdl2", "snappy", "theora", "x264", "x265", "xvid", "xz"] x264: ["ffmpeg"] Please run brew reinstall ffmpeg x264 and try again.

MikeMcQuaid commented 5 years ago

@heptathlon What does brew info ffmpeg x264 and brew config output? Try brew uninstall --force ffmpeg x264; brew reinstall ffmpeg x264.

heptathlon commented 5 years ago

brew info ffmpeg x264:

ffmpeg: stable 4.1, HEAD Play, record, convert, and stream audio and video https://ffmpeg.org/ /usr/local/Cellar/ffmpeg/4.1_1 (283 files, 56.4MB) * Built from source on 2019-01-12 at 14:33:25 From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/ffmpeg.rb ==> Dependencies Build: nasm ✔, pkg-config ✔, texi2html ✔ Required: lame ✔, libvorbis ✔, libvpx ✔, opus ✔, sdl2 ✔, snappy ✔, theora ✔, x264 ✔, x265 ✔, xvid ✔, xz ✔ Optional: aom ✘, chromaprint ✘, fdk-aac ✔, fontconfig ✔, freetype ✔, frei0r ✘, game-music-emu ✘, libass ✘, libbluray ✘, libbs2b ✘, libcaca ✘, libgsm ✘, libmodplug ✘, librsvg ✘, libsoxr ✘, libssh ✘, libvidstab ✘, libvmaf ✘, opencore-amr ✘, openh264 ✘, openjpeg ✔, openssl ✔, rtmpdump ✘, rubberband ✘, speex ✘, srt ✘, tesseract ✘, two-lame ✘, wavpack ✘, webp ✔, zeromq ✔, zimg ✘ ==> Options --with-aom Enable AV1 video codec --with-chromaprint Enable the Chromaprint audio fingerprinting library --with-fdk-aac Enable the Fraunhofer FDK AAC library --with-fontconfig Build with fontconfig support --with-freetype Build with freetype support --with-frei0r Build with frei0r support --with-game-music-emu Build with game-music-emu support --with-libass Enable ASS/SSA subtitle format --with-libbluray Build with libbluray support --with-libbs2b Build with libbs2b support --with-libcaca Build with libcaca support --with-libgsm Build with libgsm support --with-libmodplug Build with libmodplug support --with-librsvg Enable SVG files as inputs via librsvg --with-libsoxr Enable the soxr resample library --with-libssh Enable SFTP protocol via libssh --with-libvidstab Enable vid.stab support for video stabilization --with-libvmaf Enable libvmaf scoring library --with-opencore-amr Enable Opencore AMR NR/WB audio format --with-openh264 Enable OpenH264 library --with-openjpeg Enable JPEG 2000 image format --with-openssl Enable SSL support --with-rtmpdump Enable RTMP protocol --with-rubberband Enable rubberband library --with-speex Build with speex support --with-srt Enable SRT library --with-tesseract Enable the tesseract OCR engine --with-two-lame Build with two-lame support --with-wavpack Build with wavpack support --with-webp Enable using libwebp to encode WEBP images --with-zeromq Enable using libzeromq to receive commands sent through a libzeromq client --with-zimg Enable z.lib zimg library --HEAD Install HEAD version

x264: stable r2917, HEAD H.264/AVC encoder https://www.videolan.org/developers/x264.html /usr/local/Cellar/x264/r2917 (11 files, 6.2MB) * Built from source on 2019-01-12 at 14:34:28 From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/x264.rb ==> Dependencies Build: nasm ✔ ==> Options --HEAD Install HEAD version


brew config:

HOMEBREW_VERSION: 1.9.1-10-g4f17e23 ORIGIN: https://github.com/Homebrew/brew.git HEAD: 4f17e2384a9b0ffb51d4b9295f425fc072896064 Last commit: 5 hours ago Core tap ORIGIN: https://github.com/Homebrew/homebrew-core Core tap HEAD: be0d619ee2d9ba8d85b583c3651f97d66ca6cbcd Core tap last commit: 6 hours ago HOMEBREW_PREFIX: /usr/local HOMEBREW_DEV_CMD_RUN: 1 HOMEBREW_LOGS: /Users/john/Library/Logs/Homebrew HOMEBREW_VISUAL: nano CPU: octa-core 64-bit haswell Homebrew Ruby: 2.3.7 => /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.7/bin/ruby Clang: 8.0 build 800 Git: 2.20.1 => /usr/local/opt/git/bin/git Curl: 7.43.0 => /usr/bin/curl Java: 11.0.1, 1.8.0_25, 1.7.0_71 macOS: 10.11.6-x86_64 CLT: 8.2.0.0.1.1480973914 Xcode: 8.0 XQuartz: 2.7.11 => /opt/X11


brew uninstall --force ffmpeg x264; brew reinstall ffmpeg x264

Error: Refusing to uninstall /usr/local/Cellar/x264/r2917 because it is required by libav, which is currently installed. You can override this and force removal with: brew uninstall --ignore-dependencies ffmpeg x264

benknoble commented 5 years ago

After brew reinstall libtiff webp I get

Error: Formulae dependency graph sorting failed (likely due to a circular dependency):
libtiff: ["jpeg", "webp"]
webp: ["jpeg", "libpng", "libtiff"]
Please run `brew reinstall libtiff webp` and try again.

There's clearly something circular there, but I don't understand how

MikeMcQuaid commented 5 years ago

@heptathlon Run brew update; brew uninstall --ignore-dependencies --force x264 ffmpeg; brew install ffmpeg and try again @benknoble Run brew update; brew uninstall --ignore-dependencies --force webp libtiff; brew install webp libtiff and try again

heptathlon commented 5 years ago

Note that I also installed x264 again (missing in instructions): brew update; brew uninstall --ignore-dependencies --force x264 ffmpeg; brew install ffmpeg x264

brew bundle dump: Error: Formulae dependency graph sorting failed (likely due to a circular dependency): libtiff: ["jpeg", "llvm", "webp", "zstd"] webp: ["jpeg", "libpng", "libtiff"] Please run brew reinstall libtiff webp and try again.


brew update; brew uninstall --ignore-dependencies --force webp libtiff; brew install webp libtiff

brew bundle dump: Works! YEAH!

So what was the problem and is there a way for Homebrew to solve it?

Thanks again for helping! 😄

benknoble commented 5 years ago

That did it for me as well. 🎉

Curious about the root cause/why "reinstalling" didn't work, but this did.

MikeMcQuaid commented 5 years ago

Glad you both got it sorted 🎉.

This has just been the result of removing options in Homebrew/homebrew-core. Some revisions probably should have been added and weren't. This will be resolved when https://github.com/Homebrew/homebrew-core/issues/31510 is closed.

Curious about the root cause/why "reinstalling" didn't work, but this did.

brew reinstall will in some cases use the formula file embedded inside the keg. This is often desirable (for boring reasons) but wasn't in this case. I'm adjusting the messaging again in https://github.com/Homebrew/homebrew-bundle/pull/425 so it'll tell you what to do and that is as good as we'll get for resolution for this for now.

Thanks to both of you for staying responsive and helpful. Couldn't have debugged this without you both ❤️