Formulae with opportunistic linkage issues

ilovezfs commented 7 years ago

During the jpeg 9 PR, these formulae ended up opportunistically linked:

(The pull requests indicate a fix, and the commits indicate just that a new bottle was pulled as a workaround.)

[x] argyll-cms (libpng) (#23381)
[x] fdroidserver (libyaml) (https://github.com/Homebrew/homebrew-core/commit/ade3afb42c480df19aaed5bdd18ec9cba8d255c5) (won't fix)
[x] fizmo (freetype) (#23383)
[x] fox (fontconfig) (#23382)
[x] freeswitch (postgresql and xz) (https://github.com/Homebrew/homebrew-core/commit/dfa3b91435d7273042dc8fae3ddaa7410ea33c5b)
[x] gegl (suite-sparse) (#23437)
[x] ghostscript (fontconfig, libidn) (#16497)
[x] graphicsmagick (libwmf) (#23930)
[x] gst-plugins-bad (faad2) (https://github.com/Homebrew/homebrew-core/commit/261104401193e0eaa5c386be6ebd70cbc66d0488)
[x] gst-plugins-ugly (lame) (https://github.com/Homebrew/homebrew-core/commit/f2075b3ce5fe7ffb0cb9761eba1a15fc67f87d47)
[x] imagemagick (libwmf) (#16492)
[x] imagemagick@6 (libwmf) (#23937)
[x] imlib2 (libid3tag) (#23931)
[x] io (theora) (https://github.com/Homebrew/homebrew-core/commit/19c588b0b32150fb9aee43915a139d33fada5d36)
[ ] jbig2enc (giflib) (d659fbdfc81968fee97ed9716f8fa33d3f950436)
[x] mldonkey (libmagic) (#23933)
[x] nagios (libtool) (#23934)
[x] open-scene-graph (openexr, ilmbase, liblas, sdl) (#23991)
[x] podofo (libidn) (#23995)
[ ] rlvm (guichan) (d7cf6b27f806d1bec2bbe7b15520d81156bfaed8)
[x] sane-backends (libpng) (564f1542b23ddc87834942a0ef79a44eaea74224)
[x] vice (mpg123) (3fdc8a47587f54ea3e686dd8f4f15736ed0f2726)
[ ] volatility (gmp) (a49dc6cbc54249601ccf2c0f09b6483550afb04d)
[x] webp (giflib) (#16501)

stale[bot] commented 7 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

JCount commented 7 years ago

Go away @stale[bot], this is most definitely still an issue.

stale[bot] commented 7 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

DomT4 commented 7 years ago

Relax bot. Take another few weeks off.

mwunsch commented 7 years ago

Hello. I came across this issue from a failed attempt to install gst-plugins-bad. Is there a place to better understand what this issue is and what it means for something to be "opportunistically linked". Would love to better understand why these builds are failing.

queengooborg commented 6 years ago

@ilovezfs I'd be happy to help with this, but can you possibly explain what the term "opportunistically linked" means? (Thank you!)

ilovezfs commented 6 years ago

It means that one package (e.g. argyll-cms) ends up linked to the libraries of another package (e.g. libpng) even though there's no direct or recursive dependency of the former on the latter.

iMac-TMP:~ joe$ brew deps argyll-cms
jpeg
libtiff
iMac-TMP:~ joe$ brew install libpng
==> Downloading https://homebrew.bintray.com/bottles/libpng-1.6.34.el_capitan.bottle.tar.gz
Already downloaded: /Users/joe/Library/Caches/Homebrew/libpng-1.6.34.el_capitan.bottle.tar.gz
==> Pouring libpng-1.6.34.el_capitan.bottle.tar.gz
🍺  /usr/local/Cellar/libpng/1.6.34: 26 files, 1.2MB
iMac-TMP:~ joe$ brew install -s argyll-cms
==> Installing dependencies for argyll-cms: jam, jpeg, libtiff
==> Installing argyll-cms dependency: jam
==> Downloading https://homebrew.bintray.com/bottles/jam-2.6.el_capitan.bottle.tar.gz
######################################################################## 100.0%
==> Pouring jam-2.6.el_capitan.bottle.tar.gz
🍺  /usr/local/Cellar/jam/2.6: 4 files, 103.6KB
==> Installing argyll-cms dependency: jpeg
==> Downloading https://homebrew.bintray.com/bottles/jpeg-9b.el_capitan.bottle.tar.gz
Already downloaded: /Users/joe/Library/Caches/Homebrew/jpeg-9b.el_capitan.bottle.tar.gz
==> Pouring jpeg-9b.el_capitan.bottle.tar.gz
🍺  /usr/local/Cellar/jpeg/9b: 20 files, 725.1KB
==> Installing argyll-cms dependency: libtiff
==> Downloading https://homebrew.bintray.com/bottles/libtiff-4.0.9_1.el_capitan.bottle.tar.gz
Already downloaded: /Users/joe/Library/Caches/Homebrew/libtiff-4.0.9_1.el_capitan.bottle.tar.gz
==> Pouring libtiff-4.0.9_1.el_capitan.bottle.tar.gz
🍺  /usr/local/Cellar/libtiff/4.0.9_1: 246 files, 3.5MB
==> Installing argyll-cms
==> Downloading https://www.argyllcms.com/Argyll_V2.0.0_src.zip
######################################################################## 100.0%
==> sh makeall.sh
==> ./makeinstall.sh
🍺  /usr/local/Cellar/argyll-cms/2.0.0: 328 files, 115MB, built in 1 minute 3 seconds
iMac-TMP:~ joe$ brew linkage argyll-cms
System libraries:
  /System/Library/Frameworks/AppKit.framework/Versions/C/AppKit
  /System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices
  /System/Library/Frameworks/AudioToolbox.framework/Versions/A/AudioToolbox
  /System/Library/Frameworks/Carbon.framework/Versions/A/Carbon
  /System/Library/Frameworks/Cocoa.framework/Versions/A/Cocoa
  /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
  /System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics
  /System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices
  /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation
  /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit
  /usr/lib/libSystem.B.dylib
  /usr/lib/libobjc.A.dylib
Homebrew libraries:
  /usr/local/opt/jpeg/lib/libjpeg.9.dylib (jpeg)
  /usr/local/opt/libpng/lib/libpng16.16.dylib (libpng)
  /usr/local/opt/libtiff/lib/libtiff.5.dylib (libtiff)
Undeclared dependencies with linkage:
  libpng
iMac-TMP:~ joe$

queengooborg commented 6 years ago

Ah, I see — that’s what I thought, but I wasn’t 100% sure. I have some ideas on how to fix these, will be making this my day’s project. 😉

ilovezfs commented 6 years ago

Usually it just needs an explicit --without-foo passed to configure.

queengooborg commented 6 years ago

I'll be updating this comment as my own little to-do list, so that way everything can be kept track of! (Legend: 💤 = not started, 🚫 = no solution found, ❌ = could not replicate, 🕐 = working on a solution, ✅ = solution finished and in a PR.)

✅ argyll-cms loads it's own version of libpng if it cannot find a pre-installed version. However, this seems to be the case with all of its dependencies (jpeg, libtiff, and even zlib). Unfortunately, there's no way to specify --without-libpng, which is the final piece of evidence that points towards it being required. #23381

🚫 fdroidserver also seems to be linking to little-cms2.

✅ fizmo actually requires freetype, but when it can't find fontconfig, it loads an old one bundled with X11. As @ilovezfs says, it's better to just require fontconfig. #23383

✅ fox has the same issue as fizmo. #23382

✅ freeswitch also seems to be linking to unixodbc. Both it and postgresql seem to be installing anyways, however. As such, I've decided to simply add these two as dependencies. (I didn't have the issue with xz, possibly patched already?) #23384

✅ gegl also seems to be linking to cairo and jasper. All of them have been added as optional dependencies, and will not link unless requested. #23437

:zzz: rlvm :zzz: volatility

:zzz: gst-plugins-bad (doesn't have --without-XXX options) :zzz: gst-plugins-ugly (doesn't have --without-XXX options) :zzz: jbig2enc (doesn't have --without-XXX options) :zzz: vice (doesn't have --without-XXX options)

ilovezfs commented 6 years ago

@vinyldarkscratch unfortunately the problem is primarily about preventing these from becoming corrupted during CI jobs that happen to involve the other formula, not about giving users an opt-out.

queengooborg commented 6 years ago

That's true -- I'm working on argyll-cms (and learning about what a Jamfile is in the process) at the moment, but I'll get back to fox and see if I can find a good solution for it?

ilovezfs commented 6 years ago

Actually, I'd just require fontconfig because when our fontconfig isn't installed, it just tries to use the random old fontconfig that is bundled with X11.

queengooborg commented 6 years ago

@ilovezfs Ah, that's definitely a good reason -- PR incoming. :wink:

apjanke commented 6 years ago

Ping: fizmo and gegl have been fixed; can be checked off.

apjanke commented 6 years ago

gst-plugins-bad picks up several formulae opportunistically, and I don't see configure controls for them. That'll be a larger job to fix.

$ brew install -s gst-plugins-bad
...
$ brew linkage gst-plugins-bad
...
Undeclared dependencies with linkage:
  cairo
  faad2
  glib
  gstreamer
  libpng
  pango

Similar with gst-plugins-ugly.

$ brew linkage gst-plugins-ugly
...
Undeclared dependencies with linkage:
  glib
  gstreamer
  lame

ilovezfs commented 6 years ago

I think we should ignore whatever doesn't have a --without-foo

apjanke commented 6 years ago

Fair enough. I'll skip over those.

apjanke commented 6 years ago

Remaining formulae:

gst-plugins-bad: has no --without-LIB option gst-plugins-ugly: has no --without-LIB option io: cmake jbig2enc: has no --without-* switches for its lib dependencies. open-scene-graph: cmake podofo: cmake rlvm: uses SCons; gonna have to research that. vice: has no --without-mpg123 option volatility: Python, has several linkages; postponing that one till I hear about fdroidserver

The CMake-based formulae require hacking the CMakelists file for each lib, the way open-scene-graph does for ffmpeg:

    # Turning off FFMPEG takes this change or a dozen "-DFFMPEG_" variables
    if build.without? "ffmpeg"
      inreplace "CMakeLists.txt", "FIND_PACKAGE(FFmpeg)", "#FIND_PACKAGE(FFmpeg)"
    end

Do you want fixes for those?

ilovezfs commented 6 years ago

The CMake-based formulae require hacking the CMakelists file

These can almost always be addressed with a CMake arg.

apjanke commented 6 years ago

You're right. -DCMAKE_DISABLE_FIND_PACKAGE_<packagename> does it; it was just a bit hard to find that in docs. Added PRs for io and open-scene-graph.

ilovezfs commented 6 years ago

Usually you can run cmake -LAH or cmake -LAH .. just after the cmake step in a formula to get a dump of most of the available variables, including undocumented ones.

MikeMcQuaid commented 5 years ago

Closing as @ilovezfs isn't around to maintain the list any more.

Homebrew / homebrew-core

Formulae with opportunistic linkage issues #16531