search

Homebrew / homebrew-core

🍻 Default formulae for the missing package manager for macOS (or Linux)
https://brew.sh
BSD 2-Clause "Simplified" License
13.23k stars 12.05k forks source link

Big Sur Intel mass bottling tracking issue #64785

Closed jonchang closed 3 years ago

jonchang commented 3 years ago

If you are a Homebrew user on macOS Big Sur:

We do not provide support for this released but not yet supported version. You will encounter build failures with some formulae. Please create pull requests instead of asking for help on Homebrew's GitHub, Discourse, Twitter or any other official channels. You are responsible for resolving any issues you experience while you are running this released but not yet supported version.

Maintainers, our self-hosted runners for macOS 11.0 Big Sur on Intel are now up and running. To build a bottle on Big Sur, run:

brew dispatch-build-bottle --macos 11.0 --upload --issue=64785 FORMULA

This will:

  1. Run brew test-bot on that formula for a single macOS version (11.0)
  2. Upload and commit the bottle hash (e.g. https://github.com/Homebrew/homebrew-core/commit/a1e2f9abc36d7d738e6fd20c1c50ad6cd5720294)
  3. Post in this issue if there are any build failures.

Ensure that all dependencies of the formula you are requesting a bottle for already have bottles of their own. This will help avoid frankenbottles (where bottles are built using dependencies bottled for earlier versions of macOS). Consider using FX's script: https://github.com/Homebrew/homebrew-core/issues/64785#issuecomment-727192929

Note that the Big Sur runners do not have cask osxfuse, adoptopenjdk8, or xquartz installed. If any formulae had undeclared dependencies on these Requirements, they'll need to be updated or deprecated/disabled/removed, as appropriate.

I've also been running a script that automatically dispatches new bottling jobs if it sees we have idle runners. Think of it as a background job that will ensure that we are always at 100% utilization of the Big Sur runners. Any pull request or dispatch job will make it stop filling the queue, so that way you won't have wait to test something that you want to work on.

I've restricted this issue to repository collaborators as it is likely to attract many requests for support. Once we get the most popular formulae migrated this should be unlocked.

BrewTestBot commented 3 years ago

:x: @jonchang bottle request for openssl@1.1 failed.

jonchang commented 3 years ago

openssl@1.1 manually published in 548542b987c67f961539b1952d2f23fbe441d4dc

BrewTestBot commented 3 years ago

:x: @jonchang bottle request for ant failed.

BrewTestBot commented 3 years ago

:x: @jonchang bottle request for icu4c failed.

BrewTestBot commented 3 years ago

:x: @jonchang bottle request for libunistring failed.

fxcoudert commented 3 years ago

A shell script (run without argument) that tells you what formulas are ready to be bottled, in order of the analytics list:

#!/bin/bash

cd /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula

top=`brew info --analytics | cut -c 9-60 | grep -v '[(\/\-]'`

# List of formulas that are bottled (or bottle unneeded)
bottled=`egrep -l '(sha256.* => :big_sur|bottle :unneeded)' *.rb | sed 's/\.rb$//' | xargs`
bottled=" $bottled "

# One argument: the formula to check
function check_deps() {
  test -e $1.rb || return
  if [[ "$bottled" == *" $1 "* ]]; then
    return
  fi

  deps=`grep '^ *depends_on "' $1.rb | sed -e 's/^ *depends_on "//' -e 's/".*//'`
  for dep in $deps ; do
    if [[ "$bottled" == *" $dep "* ]]; then
      true
    else
      return
    fi
  done
  echo "$1"
}

for i in $top ; do
  check_deps $i
done

It's ugly shell, but it works

BrewTestBot commented 3 years ago

:x: @fxcoudert bottle request for isl failed.

jonchang commented 3 years ago

Any luck getting cmake to build?

BrewTestBot commented 3 years ago

:x: @jonchang bottle request for homebrew/core/cmake failed.

jonchang commented 3 years ago

Ok, the fully qualified name didn't work either.

BrewTestBot commented 3 years ago

:x: @fxcoudert bottle request for itstool failed.

jonchang commented 3 years ago

Cmake build: https://github.com/Homebrew/homebrew-core/actions/runs/363147364

BrewTestBot commented 3 years ago

:x: @fxcoudert bottle request for faad2 failed.

fxcoudert commented 3 years ago

bottle request for faad2 failed

Hum, audit failures about homepage being unreachable block bottling… gonna be annoying, that

BrewTestBot commented 3 years ago

:x: @fxcoudert bottle request for lz4 failed.

BrewTestBot commented 3 years ago

:x: @fxcoudert bottle request for cocoapods failed.

jonchang commented 3 years ago

I'm not sure why we're seeing this error, it was already fixed last month: https://github.com/Homebrew/ruby-macho/pull/264

==> brew audit cocoapods --online --git --skip-style
==> FAILED
Error: Failed to read Mach-O binary: /usr/local/Cellar/cocoapods/1.10.0/libexec/extensions/universal-darwin-20/2.6.0/ffi-1.13.1/ffi_c.bundle
Error: Unrecognized Mach-O load command: 0x80000034
BrewTestBot commented 3 years ago

:x: @fxcoudert bottle request for telnet failed.

BrewTestBot commented 3 years ago

:x: @fxcoudert bottle request for hicolor-icon-theme failed.

jonchang commented 3 years ago

https://github.com/Homebrew/brew/pull/9115 for cocoapods

jonchang commented 3 years ago

Qt bottle: https://github.com/Homebrew/homebrew-core/runs/1399724540?check_suite_focus=true

ERROR at //build/config/mac/mac_sdk.gni:80:5: Script returned non-zero exit code.
exec_script("//build/mac/find_sdk.py", find_sdk_args, "list lines")
^----------
Current dir: /private/tmp/qt-20201114-71251-zayscx/qt-everywhere-src-5.15.1/qtwebengine/src/core/release/
Command: /usr/bin/python2 /private/tmp/qt-20201114-71251-zayscx/qt-everywhere-src-5.15.1/qtwebengine/src/3rdparty/chromium/build/mac/find_sdk.py --p
--print_bin_path 11.0
Returned 1.
stderr:

Traceback (most recent call last):
File "/private/tmp/qt-20201114-71251-zayscx/qt-everywhere-src-5.15.1/qtwebengine/src/3rdparty/chromium/build/mac/find_sdk.py", line 127, in <modul

print(main())
File "/private/tmp/qt-20201114-71251-zayscx/qt-everywhere-src-5.15.1/qtwebengine/src/3rdparty/chromium/build/mac/find_sdk.py", line 96, in main
raise Exception('No %s+ SDK found' % min_sdk_version)
Exception: No 11.0+ SDK found

See //build/config/sysroot.gni:65:3: whence it was imported.
import("//build/config/mac/mac_sdk.gni")
^--------------------------------------
See //BUILD.gn:76:5: which caused the file to be included.
"//net:net_unittests",
^--------------------
Project ERROR: GN run error!
make[3]: *** [sub-gn_run-pro-make_first] Error 3
make[2]: *** [sub-core-make_first] Error 2
make[1]: *** [sub-src-make_first] Error 2
make: *** [module-qtwebengine-make_first] Error 2

Qtwebengine will need this chromium patch: https://github.com/chromium/chromium/commit/cdd96213435c7cb21042e84720d9343ca35b37cf#diff-195dca8233761e55ae153d3c49dca30f6ffa6abc0422af9d65c86c8c8e445569

Pull request: https://github.com/Homebrew/homebrew-core/pull/64799

fxcoudert commented 3 years ago

Erlang failure:

  You are natively building Erlang/OTP for a later version of MacOSX
  than current version (11.0.1). You either need to
  cross-build Erlang/OTP, or set the environment variable
  MACOSX_DEPLOYMENT_TARGET to 11.0.1 (or a lower version).

Upstream report https://bugs.erlang.org/browse/ERL-1407

fxcoudert commented 3 years ago 
xmlto:
  * Files were found with references to the Homebrew shims directory.
    The offending files are:
      bin/xmlto

This will block most of gtk

BrewTestBot commented 3 years ago

:x: @fxcoudert bottle request for mono failed.

BrewTestBot commented 3 years ago

:x: @fxcoudert bottle request for gstreamer failed.

fxcoudert commented 3 years ago 
gstreamer:
  * HEAD: The URL https://anongit.freedesktop.org/git/gstreamer/gstreamer.git is not a valid git URL

Will fix in https://github.com/Homebrew/homebrew-core/pull/64805

BrewTestBot commented 3 years ago

:x: @fxcoudert bottle request for lftp failed.

fxcoudert commented 3 years ago 
vde:
  * python modules have explicit framework links
    These python extension modules were linked directly to a Python
    framework binary. They should be linked with -undefined dynamic_lookup
    instead of -lpython or -framework Python.
      /usr/local/opt/vde/lib/python2.7/site-packages/vdeplug_python.so

🐍 2 🔫 💀 https://github.com/Homebrew/homebrew-core/pull/64809

BrewTestBot commented 3 years ago

:x: @fxcoudert bottle request for screenresolution failed.

fxcoudert commented 3 years ago

screenresolution has:

==> FAILED
Error: unknown or unsupported macOS version: :snow_leopard

I have no idea where it's coming from

BrewTestBot commented 3 years ago

:x: @fxcoudert bottle request for rclone failed.

fxcoudert commented 3 years ago

rclone:

/Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/github.com/billziss-gh/cgofuse@v1.4.0/fuse/host_cgo.go:116:10: fatal error: 'fuse.h' file not found
#include <fuse.h>
         ^~~~~~~~
1 error generated.
jonchang commented 3 years ago

screenresolution has:

==> FAILED
Error: unknown or unsupported macOS version: :snow_leopard

I have no idea where it's coming from

This is the git audit stepping too far back into pre-history to find an earlier version of screenresolution. I'll add it manually.

jonchang commented 3 years ago

rclone:

/Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/github.com/billziss-gh/cgofuse@v1.4.0/fuse/host_cgo.go:116:10: fatal error: 'fuse.h' file not found
#include <fuse.h>
         ^~~~~~~~
1 error generated.

Looks like this had an implicit dependency on osxfuse, which I didn't install on the Big Sur runners.

jonchang commented 3 years ago

rust first-pass build errors:

BrewTestBot commented 3 years ago

:x: @fxcoudert bottle request for gf-complete failed.

BrewTestBot commented 3 years ago

:x: @fxcoudert bottle request for ffmpeg failed.

BrewTestBot commented 3 years ago

:x: @fxcoudert bottle request for faac failed.

BrewTestBot commented 3 years ago

:x: @fxcoudert bottle request for libelf failed.

BrewTestBot commented 3 years ago

:x: @fxcoudert bottle request for gstreamer failed.

BrewTestBot commented 3 years ago

:x: @fxcoudert bottle request for lz4 failed.

BrewTestBot commented 3 years ago

:x: @fxcoudert bottle request for faad2 failed.

BrewTestBot commented 3 years ago

:x: @fxcoudert bottle request for cabal-install failed.

fxcoudert commented 3 years ago

cabal-install fails, and it's probably not specific to Big Sur:

Setup: Encountered missing or private dependencies:
base >=4.5 && <4.14
BrewTestBot commented 3 years ago

:x: @fxcoudert bottle request for orc failed.

BrewTestBot commented 3 years ago

:x: @fxcoudert bottle request for dep failed.

dep:
  * GitHub repo is archived

This is going to be a problem, because we need that one.

BrewTestBot commented 3 years ago

:x: @alebcay bottle request for telnet failed.

BrewTestBot commented 3 years ago

:x: @fxcoudert bottle request for isl failed.

BrewTestBot commented 3 years ago

:x: @fxcoudert bottle request for lz4 failed.

BrewTestBot commented 3 years ago

:x: @fxcoudert bottle request for macvim failed.

error: The linked and embedded framework 'Sparkle.framework' is missing one or more architectures required by this target: arm64. (in target 'MacVim' from project 'MacVim')