jonchang
commented
3 years ago I have a script going to get build logs for all of the formulae which we attempted to build but don't have bottles for.
Closed fxcoudert closed 3 years ago
jonchang
commented
3 years ago I have a script going to get build logs for all of the formulae which we attempted to build but don't have bottles for.
derrabus
commented
3 years ago Regarding php@7.2, this is mainly an Xcode 12 compatibility problem. @SMillerDev has worked on this, see #61828. Given that php 7.2 will be EOL in less than two weeks, I wonder if it's worth patching the package, though.
SMillerDev
commented
3 years ago Just remove it already IYAM.
sharris40
commented
3 years ago vapoursynth needs somebody to look into a licensing problem before the relevant pull request (https://github.com/Homebrew/homebrew-core/pull/59828) can proceed. It’s been stale for almost a month. I can make an interim pull request that just fixes the audit failures and doesn’t update the package.
carlocab
commented
3 years ago
- exa: missing pandoc
exa only needs pandoc for building head, so I think this can be bottled as is. Incidentally, building head will also install the exact same exa binary as the stable version, so I'm not really sure what head is doing there. I'll work on the formula so it actually installs a development version when it's been bottled.
issyl0
commented
3 years ago A fix for vault's test is in #65037.
Totktonada
commented
3 years ago jonchang
commented
3 years ago It might need to depend on curl-openssl?
carlocab
commented
3 years ago I've nailed down the problem with bottling odin: https://github.com/odin-lang/Odin/issues/792
This should be fixed in a future version, but in the meantime, I'm trying to see if upstream will provide a patch.
Moisan
commented
3 years ago distcc HEAD builds fine with an up to date libiberty resource. Based on https://github.com/distcc/distcc/issues/401, upstream seems to be about to tag a new release so we should be able to bump it soon.
jonchang
commented
3 years ago I've got pypy seeming to build locally and have opened https://github.com/Homebrew/homebrew-core/pull/65854 to see if it'll bottle.
mitchblank
commented
3 years ago I opened a PR that should fix cvs: #65954
mitchblank
commented
3 years ago Over the last couple days I took a look through some of the formula in @jonchang's "Big list" above... either packages I have some past familiarity with or things that just looked interesting:
Fixed and already merged:
PRs open but not merged yet:
Nominated for removal:
Builds fine for me, not sure what the bottling problem was:
Unchecked things in @jonchang's "Big list" that seem to have already been resolved (Big Sur bottles currently exist):
mitchblank
commented
3 years ago A word of caution about perl-based formula -- I've been having a hell of a time with them.
First there is the issue where MakeMaker seems to have changed how it lays out the directory by default. I've seen that a couple times already, and have what I think is a workaround even though I don't understand the underlying problem (e.g. #66367 and #66370)
Today I took what I thought would be a quick look at asciiquarium (a very important formula, I know...) After literally hours of digging through the perl module make system trying to figure out why its curses bindings aren't building correctly I came across Perl-Toolchain-Gang/ExtUtils-MakeMaker#381 -- basically Apple's own /usr/bin/perl is broken for detecting external libs via MakeMaker so any perl library using that framework to build itself is likely going to have problems.
At the root, this is the same type of issue as things like #66304 where the build scripts assume they can probe for the *.dylib files but they now only exist to dlopen()'s world.
Basically, if you see a broken formula that relies on MakeMaker be afraid.
mitchblank
commented
3 years ago Hit a few more.
Fixed and already merged:
PRs open but not merged yet:
Investigated:
Builds fine for me, not sure what the bottling problem was:
Also have leads on some other root causes that might affect multiple formulae:
Term::ReadKey failures in 3 formulae: https://github.com/Homebrew/homebrew-core/pull/66485#issuecomment-741131566
mbcoguno
commented
3 years ago portmidi: #66648 mame: #66718
fxcoudert
commented
3 years ago I've updated the list at the top. I've also started bottling jobs for:
BrewTestBot
commented
3 years ago :x: @fxcoudert bottle request for dungeon failed.
BrewTestBot
commented
3 years ago :x: @fxcoudert bottle request for airshare failed.
BrewTestBot
commented
3 years ago :x: @fxcoudert bottle request for upx failed.
mitchblank
commented
3 years ago I really don't understand what makes BrewTestBot unhappy. I just looked at those three logs.zip and don't have any idea why they "failed"
carlocab
commented
3 years ago @mitchblank try looking at the logs in the CI interface (the raw logs may be easier to search). I've found that sometimes useful information shows up there which don't make it to the log archives.
mitchblank
commented
3 years ago BTW abcde (on the top list) should be OK now that its dependency cdrtools is fixed. I just build/tested it locally.
mitchblank
commented
3 years ago @mitchblank try looking at the logs in the CI interface (the raw logs may be easier to search). I've found that sometimes useful information shows up there which don't make it to the log archives.
Ah OK. Yeah that is somewhat more informative.
open3 is used in the test code; I don't have the ruby experience to say for sure$PWD?fxcoudert
commented
3 years ago @mitchblank disabling tests under CI is a no, always. The tests are meant to be run on CI in most cases, that's how we test that we don't break things.
upx uses weird techniques for binary compression, and they break at every new macOS version (because they do dirty tricks with system-reserved features)
mitchblank
commented
3 years ago OK, upx seems legitimately broken. brew test fails manually on it and:
$ cp /usr/local/bin/upx .
$ chmod +w upx
$ upx -1 ./upx
[...]
$ ./upx -V
[1] 26947 killed ./upx -VIssue already open upstream https://github.com/upx/upx/issues/424
carlocab
commented
3 years ago Yes, I think the tests really are there for CI more than anything else, especially now that (I believe) the vast majority (or, if not, plurality) of PRs are bump-formula-prs, which no longer require the checklist. (Not to mention that the checklists are easy to ignore and fill up by mistake.)
mitchblank
commented
3 years ago xhyve apparently has a branch with Big Sur fixes: https://github.com/machyve/xhyve/issues/207 ...although arm64 will still have more work to do: https://github.com/machyve/xhyve/issues/206
mitchblank
commented
3 years ago I was going to look at some of the top issues but first I thought I should go through and see which open ones have already-known or investigated stuff. This is an edited copy of @fxcoudert's list (with a few lines removed where Big Sur bottles have recently appeared)
Is there somewhere better (wiki-like?) where we could host such a list so people could add their own per-package comments?
11043 mono -- @fxcoudert has #65662 in progress
3631 upx -- upstream issue: upx/upx#424
3153 hyperkit -- see moby/hyperkit#297
2709 testdisk -- unit test sandbox issue: #66311
2621 mingw-w64
2101 boost-python -- note that this builds against system python2 as opposed to boost-python3 formula which is happy wrt Big Sur
1999 iperf -- upstream fix exists https://sourceforge.net/p/iperf2/code/ci/ae0284e4058916acd5807e78b751134c9c3a74e4/
1740 haskell-stack -- @chenrui333 has looked at updating but hit a build issue #62939
1727 net-snmp
1631 dotnet
1468 po4a -- @chenrui333 has #66522 open. TermReadKey user so probably similar fix to innotop
1450 dpkg -- depends_on "po4a"
1368 pypy3 -- depends_on "pypy"; @chenrui333 has #65821 open
1299 mysql-client@5.7
804 sane-backends -- depends_on "net-snmp"
769 mpich
705 pypy
682 llvm@9
654 docker-machine-driver-xhyve -- presumably blocked on xhyve, but doesn't have a depends_on?!?
653 berkeley-db@4
605 crystal -- #65221 is open but will add dependency back on llvm@9
576 distcc -- @mitchblank sent a fix upstream; supposedly new release imminent? distcc/distcc#401
573 mpd
568 rethinkdb
492 llvm@8
489 global
477 thrift@0.9
448 xdotool
424 couchdb -- depends_on "erlang@22", "spidermonkey"
420 gron
417 erlang@22
416 minimal-racket
414 xhyve -- upstream has a branch with Big Sur fixes, maybe wait for release: machyve/xhyve#207
396 cloog
345 eye-d3
338 collectd -- depends_on "net-snmp"
326 foremost
324 git-annex -- @chenrui333 has #64947 open
313 winexe
296 spidermonkey
295 xctool
279 libjson-rpc-cpp
278 connect
270 volatility
270 minidlna -- @mprince has #65760 open
267 mapnik -- @iMichka has #65895 in progress
263 tarantool -- build scripts having trouble detecting curl+ssl? https://github.com/Homebrew/homebrew-core/issues/65000#issuecomment-731336505
262 vcprompt -- @fxcoudert is adding a mirror with #65374
254 czmq
252 imagesnap
248 asciiquarium -- #66485 open
244 elm
240 gnu-prolog
238 llvm@7
218 fakeroot -- @claui as #64906 open
215 bmon
211 jython
204 vsftpd
179 neko
176 bitcoin -- depends_on "berkeley-db@4"
174 mogenerator
171 gnu-cobol
156 mesos -- @chenrui333 has #66205 open
155 haxe -- depends_on "neko"
153 bit
147 bbftp-client
147 balena-cli -- @alebcay has #63674 open
143 swift -- @chenrui333 has #61414 open
142 antlr@2
129 vera++
128 cryptominisat
126 erlang@21
124 subversion@1.8
120 zssh -- @mitchblank has #66397 open
114 scriptcs -- depends_on "mono"
114 keepassc
113 homebank -- depends_on "libofx"
112 kpcli -- another perl module issue, likely not dissimplar to either the Curses or TermReadKey issues
111 makensis -- depends_on "mingw-w64"
111 hledger -- @simonmichael has #66599 open; also depends_on "haskell-stack"
110 insect -- depends_on "psc-package", "pulp", "purescript"
110 crf++
105 cabocha -- depends_on "crf++"
97 libofx -- depends_on "open-sp"
96 tasksh
93 nyancat -- build and tested fine for me, probably CI issue with pty use
93 coturn
89 open-sp
89 ejabberd -- depends_on "erlang@22"
88 kite
87 ldc -- depends_on "llvm@9", @bayandin has #61117 open to update dependency
85 appledoc
82 agda
80 libmaa
80 idris
79 ace
78 plplot
76 pcre++ -- CI can't download package but URL seems fine https://github.com/Homebrew/homebrew-core/issues/66360#issuecomment-742637228
76 csound -- depends_on "faust"
75 hdf5@1.8
75 clozure-cl
74 cataclysm
74 argyll-cms
73 hardlink-osx
73 dafny -- depends_on "mono"; also some changes from @davidcok with #66278
72 dict -- depends_on "libmaa"
71 shntool
69 feedgnuplot -- @mitchblank has #66370 (perl MakeMaker PREFIX issues)
68 include-what-you-use -- depends_on "llvm@9"; @mprince is updating version with #65850
67 dar -- depends_on "upx"
66 dashing -- @hyuraku has #65648 open to update version
64 ike-scan
64 cadaver
60 xsane -- depends_on "sane-backends"
59 kube-aws
58 launchdns
56 pg_top
56 mpack
55 innotop -- @mitchblank has #66699 open
54 planck
53 rt-audio
53 quazip
52 libpst
52 clang-format@8
51 websocketd
51 faust
50 mydumper
49 ponyc
49 pam_yubico -- depends_on "ykclient"
48 jnettop
47 odin -- odin-lang/Odin#792
46 wrangler -- depends_on "erlang@22"
45 ykclient
45 purescript
43 st -- likely similar fix to feedgnuplot: https://github.com/Homebrew/homebrew-core/pull/66370#issuecomment-740856994
43 pcapplusplus
42 tcc -- badly broken wrt Big Sur https://savannah.nongnu.org/bugs/index.php?59640
39 makepkg -- @chenrui333 has update #65488; but also depends_on "fakeroot"
38 nng
37 ykneomgr
37 mariadb-connector-odbc
36 airshare -- build/test OK, but test doesn't work in CI sandbox: https://github.com/Homebrew/homebrew-core/issues/65000#issuecomment-743343848
35 docker-swarm
34 trafficserver
34 cassandra-reaper
33 ssdb
32 mpir
31 apcupsd
30 mysql++
29 nauty
28 dfc
28 bluepill -- #66592 open
27 pdftoipe
26 lemon -- build/test manually worked for me
26 curlpp
24 task-spooler
23 p0f
22 ssed
22 orbit
22 liberasurecode -- depends_on "jerasure"
21 wangle
21 remind
21 gcore
21 exim -- depends_on "berkeley-db@4"
20 nu-smv
20 networkit
20 logtalk -- depends_on "gnu-prolog"
20 cucumber-cpp
20 amap -- build/test worked ok for me manuallyy
20 alure
18 speech-tools
18 pacparser
18 augustus
17 echoprint-codegen
17 chuck
16 pipgrip
16 git-cal -- likely similar fix to feedgnuplot: https://github.com/Homebrew/homebrew-core/pull/66370#issuecomment-740856994
16 austin
16 asuka
15 zpaq -- #66620
15 lasi
15 cstore_fdw
14 visp
14 rinetd
14 cvs-fast-export
14 cspice
14 cassandra@2.1
13 oq -- depends_on "crystal"
13 libzdb
13 eprover
13 check_postgres -- likely similar fix to feedgnuplot: https://github.com/Homebrew/homebrew-core/pull/66370#issuecomment-740856994
13 allureofthestars
12 voldemort
12 sng
12 pulp -- depends_on "purescript"
12 creduce -- depends_on "llvm@9"; also TermReadKey user so probably similar issue to innotop
11 hashlink -- depends_on "haxe"
11 git-remote-hg
11 dungeon -- build/test is fine, but test fails under CI for some reason: https://github.com/Homebrew/homebrew-core/issues/65000#issuecomment-743343848
10 nmh
10 jerasure
9 tpp -- @mitchblank has #66517 in progress but build failed in CI for some reason
9 gambit
9 ezstream
9 dnsprobe
8 tarsnap-gui
8 pnetcdf
8 notmuch-mutt -- another perl module issue, likely not dissimplar to either the Curses or TermReadKey issues
8 libcddb
8 get-flash-videos
7 texmath
7 libpano
7 herrie
7 fragroute
7 enigma
7 crystal-icr -- depends_on "crystal"
6 sfst
6 serialosc
6 plowshare -- depends_on "spidermonkey"
5 zyre -- depends_on "czmq"
5 zorba
5 unyaffs
5 rtptools -- @mitchblank has #66556 in progress
5 rancher-compose
5 pilosa
5 arturo
4 virtualpg
4 snappystream
4 rex -- TermReadKey user so probably similar fix to innotop
4 psc-package -- depends_on "purescript"
4 pod2man -- @mitchblank has #66367 (perl MakeMaker PREFIX issues)
4 nvc -- depends_on "llvm@7"
4 mrboom -- @chenrui333 has #66461 in progress
4 dlite
4 cosi
4 bsponmpi
4 bigloo
4 atomist-cli
3 xrick
3 traildb
3 temporal_tables
3 pony-stable -- depends_on "ponyc"
3 ncp
3 mysql-sandbox -- likely similar fix to feedgnuplot: https://github.com/Homebrew/homebrew-core/pull/66370#issuecomment-740856994
3 harbour
3 autopano-sift-c -- depends_on "libpano"
2 quex
2 pokerstove
2 inform6
2 hqx
2 grt
2 gel
2 ficy
2 fbi-servefiles
2 exult
2 docker2aci
2 clipsafe
2 bond -- @mprince has #65587 in progress, but also depends_on "haskell-stack"
1 xaric
1 whohas
1 tmpreaper
1 tenyr
1 reprepro -- depends_on "berkeley-db@4"
1 oclgrind -- depends_on "llvm@9"
1 libming
1 libevhtp
1 imagejs
1 cpl
1 cmt
1 cedille -- depends_on "agda"
1 beansdb
1 b43-fwcutter
0 skafos
0 qsoas
0 pakchois
0 minbif
0 mdxmini
0 libswiften
0 libsbol
0 eventql
0 corral -- depends_on "ponyc"Took a swing at net-snmp since it's near the top of the list: #66832
Might let us also unblock sane-backends and collectd which are fairly high up as well
mitchblank
commented
3 years ago gnu-prolog fix: #66833
mitchblank
commented
3 years ago Took a quick look at "mpd" -- built OK but brew test is broken. Looks like free_port didn't do what was expected:
==> Testing mpd
exception: Failed to bind to '127.0.0.1:55363'; Failed to bind socket: Address already in usesound familiar to anyone?
mitchblank
commented
3 years ago bmon fix: #66847
carlocab
commented
3 years ago Tried a dirty hack (which didn't work) to fix neko here: https://github.com/Homebrew/homebrew-core/pull/66819
I think the problem is that cmake can't find system libiconv on Big Sur for some reason. I think this has to do with the disappearance of system libraries from the file system in Big Sur, but it seems CMake was supposed to have already fixed that? So not really sure what the deal is.
mitchblank
commented
3 years ago Yeah I was looking at neko too... there are at least 3 or 4 different problems I found. Still sort of poking at it, but it's a tricky one.
carlocab
commented
3 years ago So I tried adding depends_on "libiconv" if MacOS.version >= :big_sur, which fixed the problem of not being able to find libiconv.
Then some other error popped up, but this happened during a ninja run. Ninja runs a lot of things in parallel, so now it's become really difficult to figure out where exactly the error came up.
Logs here in case anyone wants to have a look: neko-log.tar.gz and https://github.com/Homebrew/homebrew-core/runs/1544327700
mitchblank
commented
3 years ago Well this mostly works:
--- a/Formula/neko.rb
+++ b/Formula/neko.rb
@@ -20,6 +20,7 @@ class Neko < Formula
depends_on "bdw-gc"
depends_on "mbedtls"
depends_on "openssl@1.1"
+ depends_on "libiconv"
depends_on "pcre"
def install
@@ -27,6 +28,19 @@ class Neko < Formula
%r{https://downloads.mariadb.org/f/},
"https://downloads.mariadb.com/Connectors/c/"
+ %w{lock_try lock_acquire lock_release}.each do |func|
+ inreplace %w{vm/neko.h.in vm/threads.c libs/ui/ui.c}, " " + func, " HOMEBREW_" + func
+ end
+ %w{lock_acquire lock_release}.each do |func|
+ inreplace %w{vm/others.c}, "\t" + func, "\tHOMEBREW_" + func
+ end
+
+ inreplace "libs/std/process.c", "<stdlib.h>", "<stdlib.h>\n#include <signal.h>"
+
+ %w{RunApplicationEventLoop QuitApplicationEventLoop}.each do |func|
+ inreplace "libs/ui/ui.c", func, "extern void #{func}(void); #{func}"
+ end
+...obviously those inreplace need to be turned into real upstream patches; I've just used them as a temporary hack.
There are still warnings about MSG_NOSIGNAL that look like a bug to me, too.
No idea if those ui changes really work either, but it does compile... :shrug:
carlocab
commented
3 years ago @mitchblank did you see if haxe and hashlink will build with your hacked fix of neko?
carlocab
commented
3 years ago haxe and hashlink have Big Sur bottles here: https://github.com/Homebrew/homebrew-core/pull/66819
However, these seem to depend on a neko Catalina bottle, so not sure if that counts.
If that's a problem, it might be worth leaving a note in the neko formula that haxe and hashlink need to be rebottled when neko gets a Big Sur bottle?
mitchblank
commented
3 years ago Sorry, was busy the last couple days with Real Life. I opened PRs on neko for some of the fixes and made a new homebrew PR: #66975
This also fixes the libiconv thing by convincing mariadb-connector to link against the system iconv/libz rather than adding a dependency
mitchblank
commented
3 years ago @mitchblank did you see if
haxeandhashlinkwill build with your hacked fix ofneko?
Just tested both with #66975 -- build and test of each worked
mitchblank
commented
3 years ago couple others I looked at tonight:
carlocab
commented
3 years ago I've been able to build and test Agda locally. PR here: https://github.com/Homebrew/homebrew-core/pull/67014
Leont
commented
3 years ago First there is the issue where MakeMaker seems to have changed how it lays out the directory by default. I've seen that a couple times already, and have what I think is a workaround even though I don't understand the underlying problem (e.g. #66367 and #66370)
MakeMaker hasn't changed, but probably Apple's perl config has. Can you give the the output of perl -V:siteprefixexp -V:installsitelib on both MacOS 10.15 and on MacOS 11?
Today I took what I thought would be a quick look at asciiquarium (a very important formula, I know...) After literally hours of digging through the perl module make system trying to figure out why its curses bindings aren't building correctly I came across Perl-Toolchain-Gang/ExtUtils-MakeMaker#381 -- basically Apple's own /usr/bin/perl is broken for detecting external libs via MakeMaker so any perl library using that framework to build itself is likely going to have problems.
At the root, this is the same type of issue as things like #66304 where the build scripts assume they can probe for the *.dylib files but they now only exist to dlopen()'s world.
I think I have a solution for that, I hope we can release that in the coming days. That does require upgrading MakeMaker as a first step.
carlocab
commented
3 years ago macOS 11
Homebrew perl:
❯ /usr/local/bin/perl -V:siteprefixexp -V:installsitelib
siteprefixexp='/usr/local/Cellar/perl/5.32.0';
installsitelib='/usr/local/Cellar/perl/5.32.0/lib/perl5/site_perl/5.32.0';system perl:
❯ /usr/bin/perl -V:siteprefixexp -V:installsitelib
siteprefixexp='/usr';
installsitelib='/Library/Perl/5.28';Yeah, it looks like siteprefixexp is what changed. Here is a 10.15 system perl:
% /usr/bin/perl -V:siteprefixexp -V:installsitelib
siteprefixexp='/usr/local';
installsitelib='/Library/Perl/5.18';I've been meaning to get back to the perl issues soon; been distracted by other things. I'm pretty sure the bulk of the problems will be fixed if we just change to using INSTALL_BASE consistently. Really our needs are pretty simple (just install the same way you normally would except into .../Cellar/pkg/version instead of /usr or /usr/local. For a "normal" autoconf package that just means repointing --prefix but for MakeMaker it seems INSTALL_BASE is the closer analog
mitchblank
commented
3 years ago That does require upgrading MakeMaker as a first step.
Most of these perl packages currently work with the system perl, so that is potentially tricky. One option would be to start requiring that they be built against homebrew perl5 instead, although that does increase the baggage required for some otherwise-tiny packages.
Based on what I've seen so far we can probably work around most of the issues with the system perl, but we'll see if we hit any real roadblocks.
carlocab
commented
3 years ago One option would be to start requiring that they be built against homebrew perl5 instead
If it's just a build dependency then it should be fine, I think. Though we should still avoid adding it unnecessarily. Are there any circumstances where using Homebrew perl to build could turn it into a runtime dependency?
mitchblank
commented
3 years ago Pretty sure it would be a runtime dependency in that case, since the perl modules in resources would end up compiled against that copy of perl5.
We already have a massive issue that's systematically tracking and collating all Big Sur bottling failures: https://github.com/Homebrew/homebrew-core/issues/64785 In this issue, I want to track a prioritised list of failures among our top-downloaded formulas, so we track progress and participation.
This top post will be edited over time to remain accurate. Discussion and offers of help can go below.
List of formulas missing a Big Sur bottle (and number of montly installs):