AVX instructions in Big Sur bottles crash when run in Rosetta 2

[Tyrubias]

Bug report

Please note we will close your issue without comment if you delete, do not read or do not fill out the issue checklist below and provide ALL the requested information. If you repeatedly fail to use the issue template, we will block you from ever submitting issues to Homebrew again.

• [x] ran brew update and can still reproduce the problem?
• [x] ran brew doctor, fixed all issues and can still reproduce the problem?
• [x] ran brew gist-logs <formula> (where <formula> is the name of the formula that failed) and included the output link?
• [x] if brew gist-logs didn't work: ran brew config and brew doctor and included their output with your issue?

What you were trying to do (and why)

I was trying to run python3 from the formula python@3.9 on my M1 MacBook Air under Rosetta 2.

What happened (include command output)

When trying to run the interactive REPL or any Python program, the interpreter crashes with the error illegal hardware instruction. Logs can be found here.

Command output

  [1]    2892 illegal hardware instruction  python3

  

What you expected to happen

Python 3 should run under Rosetta 2

Step-by-step reproduction instructions (by running brew install commands)

$ brew install python@3.9
$ python3

Of note is that fact that postinstall fails as well.

[lutzroeder]

@fxcoudert @carlocab is there anything that can be done to rebuild python which we know is broken while we are figuring out what else might be broken?

[redblobgames]

I'm getting the illegal instruction error when running emcc or em++ from the emscripten package, but it's not in carlocab's list of affected bottles, so this may affect more packages than that list.

Edit: false alarm; this is because emcc runs python

[jonchang]

Formulae bottled between brew versions 2.7.0-32-g379371f and 2.7.0-100-gda0d7ef (inclusive) will be affected by this issue. This depends on when the formula was bottled, not when it was merged into homebrew-core, so we'll need to treat everything after Dec 23 as possibly suspect.

% git log --since='Wed Dec 23 09:19:33 2020 +0000' --reverse --oneline | head -n1
c6a6bb8f23 kumactl 1.0.4

I found 235 formulae with Big Sur bottle changes between c6a6bb8f23 and f0fc30fb7. We can download all extant bottles for these formulae and examine the install receipt to identify which brew version created that bottle.

brew fetch abseil agda aliyun-cli allureofthestars apcupsd appledoc asio aws-google-auth bartycrouch bbftp-client bdw-gc benthos bitcoin bond borgbackup braid broot bullet bumpversion cabocha cargo-audit cargo-watch check_postgres chkrootkit chuck clash cocoapods coturn counterfeiter crystal-icr cubejs-cli cucumber-cpp cucumber-ruby cvs-fast-export dartsim dash dcm2niix dfc diceware dict direnv dmagnetic dmalloc dnscontrol dnsx doctest doxygen dpkg dvc efl efm-langserver eksctl fastlane ficy fig2dev flow-cli folderify freerdp futhark gcalcli geographiclib get-flash-videos ghc ghostscript git-annex git-cal git-delta git-remote-codecommit git-revise gitless glab glow gnu-cobol graphicsmagick grpc h2spec hamlib harfbuzz haskell-stack hatari hebcal hiredis hledger homebank hqx htmldoc iam-policy-json-to-terraform imagejs imagemagick imagemagick@6 immudb inko ircd-hybrid jpeg-turbo jql jupyterlab k3sup kubevela kumactl lasi lazygit lemon libarchive libcddb libepoxy libgpg-error libhandy libjson-rpc-cpp libofx libpst libstfl libstrophe libxslt localstack macosvpn macvim magic_enum mailutils makensis mariadb-connector-c mdbtools micronaut mikutter mkvtomp4 moarvm mockery molecule monolith mosquitto mozjpeg mpg123 mpir mpop mps-youtube msmtp mysql-sandbox navi ncmpc ncmpcpp neko newrelic-cli newsboat nexus nfpm ngs nicotine-plus nmh notmuch nqp ocrmypdf okteto open-mpi openalpr openvdb optipng oq orbit orientdb p0f pacparser pagmo pakchois passenger pc6001vx pcre++ pdfcpu pdftoipe pg_top pgpool-ii pgroonga plowshare pnpm promtail pugixml pulumi pyinvoke python@3.9 qbs radare2 rakudo raylib remind reprepro restview rethinkdb rinetd robot-framework root rqlite rst-lint ruby ruby@2.7 scipy scour sdl2 shallow-backup shntool speech-tools sphinx-doc spidermonkey sqlc sshuttle st statik streamlink stress-ng supervisor swiftformat tasksh terraform_landscape tin tokei trailscraper trash-cli travis twine-pypi util-linux vcpkg vice vim vit vitetris vsftpd weechat whistle wimlib wolfssl x265 xterm ydcv you-get youtube-dl youtube-dlc zpaq zyre
parallel 'gtar xf $(brew --cache {}) --wildcards "*INSTALL_RECEIPT.json" --to-stdout | jq -r .homebrew_version | xargs echo {}' ::: abseil agda aliyun-cli allureofthestars apcupsd appledoc asio aws-google-auth bartycrouch bbftp-client bdw-gc benthos bitcoin bond borgbackup braid broot bullet bumpversion cabocha cargo-audit cargo-watch check_postgres chkrootkit chuck clash cocoapods coturn counterfeiter crystal-icr cubejs-cli cucumber-cpp cucumber-ruby cvs-fast-export dartsim dash dcm2niix dfc diceware dict direnv dmagnetic dmalloc dnscontrol dnsx doctest doxygen dpkg dvc efl efm-langserver eksctl fastlane ficy fig2dev flow-cli folderify freerdp futhark gcalcli geographiclib get-flash-videos ghc ghostscript git-annex git-cal git-delta git-remote-codecommit git-revise gitless glab glow gnu-cobol graphicsmagick grpc h2spec hamlib harfbuzz haskell-stack hatari hebcal hiredis hledger homebank hqx htmldoc iam-policy-json-to-terraform imagejs imagemagick imagemagick@6 immudb inko ircd-hybrid jpeg-turbo jql jupyterlab k3sup kubevela kumactl lasi lazygit lemon libarchive libcddb libepoxy libgpg-error libhandy libjson-rpc-cpp libofx libpst libstfl libstrophe libxslt localstack macosvpn macvim magic_enum mailutils makensis mariadb-connector-c mdbtools micronaut mikutter mkvtomp4 moarvm mockery molecule monolith mosquitto mozjpeg mpg123 mpir mpop mps-youtube msmtp mysql-sandbox navi ncmpc ncmpcpp neko newrelic-cli newsboat nexus nfpm ngs nicotine-plus nmh notmuch nqp ocrmypdf okteto open-mpi openalpr openvdb optipng oq orbit orientdb p0f pacparser pagmo pakchois passenger pc6001vx pcre++ pdfcpu pdftoipe pg_top pgpool-ii pgroonga plowshare pnpm promtail pugixml pulumi pyinvoke python@3.9 qbs radare2 rakudo raylib remind reprepro restview rethinkdb rinetd robot-framework root rqlite rst-lint ruby ruby@2.7 scipy scour sdl2 shallow-backup shntool speech-tools sphinx-doc spidermonkey sqlc sshuttle st statik streamlink stress-ng supervisor swiftformat tasksh terraform_landscape tin tokei trailscraper trash-cli travis twine-pypi util-linux vcpkg vice vim vit vitetris vsftpd weechat whistle wimlib wolfssl x265 xterm ydcv you-get youtube-dl youtube-dlc zpaq zyre

There are 172 formulae that will need to be rebuilt. We need to a few code changes to handle bottle rebuilds and then I'll kick off the jobs, which will hopefully pass without incident.

version	formula
2.5.10-31-g63b03fb	scipy
2.5.11	open-mpi
2.6.2-41-gd73fc48	neko
2.7.0	mpir
2.7.0-19-gfc305cc	pg_top
2.7.0-19-gfc305cc	root
2.7.0-24-g53ef74f	bdw-gc
2.7.0-24-g53ef74f	geographiclib
2.7.0-24-g53ef74f	molecule
2.7.0-24-g53ef74f	optipng
2.7.0-26-gc8210ac	counterfeiter
2.7.0-26-gc8210ac	kumactl
2.7.0-26-gc8210ac	libgpg-error
2.7.0-26-gc8210ac	mariadb-connector-c
2.7.0-26-gc8210ac	p0f
2.7.0-26-gc8210ac	vsftpd
2.7.0-32-g379371f	libepoxy
2.7.0-32-g379371f	nmh
2.7.0-35-g943ee39	appledoc
2.7.0-35-g943ee39	git-delta
2.7.0-35-g943ee39	pcre++
2.7.0-35-g943ee39	tasksh
2.7.0-37-g7d46fb8	bitcoin
2.7.0-37-g7d46fb8	dpkg
2.7.0-39-gd6692d2	bbftp-client
2.7.0-39-gd6692d2	cabocha
2.7.0-39-gd6692d2	crystal-icr
2.7.0-39-gd6692d2	cvs-fast-export
2.7.0-39-gd6692d2	dict
2.7.0-39-gd6692d2	libofx
2.7.0-39-gd6692d2	makensis
2.7.0-39-gd6692d2	reprepro
2.7.0-39-gd6692d2	zpaq
2.7.0-39-gd6692d2	zyre
2.7.0-41-ge041468	dash
2.7.0-41-ge041468	gnu-cobol
2.7.0-41-ge041468	hebcal
2.7.0-41-ge041468	immudb
2.7.0-41-ge041468	inko
2.7.0-41-ge041468	k3sup
2.7.0-41-ge041468	kubevela
2.7.0-41-ge041468	mailutils
2.7.0-41-ge041468	mosquitto
2.7.0-41-ge041468	msmtp
2.7.0-41-ge041468	newsboat
2.7.0-41-ge041468	qbs
2.7.0-41-ge041468	sqlc
2.7.0-44-gcb8e9a6	ghostscript
2.7.0-44-gcb8e9a6	navi
2.7.0-44-gcb8e9a6	newrelic-cli
2.7.0-44-gcb8e9a6	pagmo
2.7.0-44-gcb8e9a6	pnpm
2.7.0-44-gcb8e9a6	sdl2
2.7.0-53-ge6e76a4	braid
2.7.0-53-ge6e76a4	bullet
2.7.0-53-ge6e76a4	dartsim
2.7.0-53-ge6e76a4	dcm2niix
2.7.0-53-ge6e76a4	efl
2.7.0-53-ge6e76a4	freerdp
2.7.0-53-ge6e76a4	homebank
2.7.0-53-ge6e76a4	libpst
2.7.0-53-ge6e76a4	libxslt
2.7.0-53-ge6e76a4	magic_enum
2.7.0-53-ge6e76a4	pulumi
2.7.0-53-ge6e76a4	radare2
2.7.0-53-ge6e76a4	streamlink
2.7.0-53-ge6e76a4	wolfssl
2.7.0-56-g78e5ad0	h2spec
2.7.0-56-g78e5ad0	mockery
2.7.0-56-g78e5ad0	ncmpcpp
2.7.0-56-g78e5ad0	oq
2.7.0-56-g78e5ad0	pc6001vx
2.7.0-56-g78e5ad0	pgpool-ii
2.7.0-56-g78e5ad0	shntool
2.7.0-58-gab1d399	ncmpc
2.7.0-58-gab1d399	remind
2.7.0-58-gab1d399	speech-tools
2.7.0-64-ge69a54d	pacparser
2.7.0-64-ge69a54d	util-linux
2.7.0-66-ga2ae6f8	jupyterlab
2.7.0-66-ga2ae6f8	rqlite
2.7.0-68-g5818b28	dmagnetic
2.7.0-68-g5818b28	efm-langserver
2.7.0-68-g5818b28	lazygit
2.7.0-68-g5818b28	python@3.9
2.7.0-68-g5818b28	spidermonkey
2.7.0-78-g2424e58	lemon
2.7.0-78-g2424e58	pugixml
2.7.0-78-g2424e58	twine-pypi
2.7.0-80-g1f80e57	aliyun-cli
2.7.0-80-g1f80e57	broot
2.7.0-80-g1f80e57	dmalloc
2.7.0-84-g8ac02c9	imagemagick
2.7.0-84-g8ac02c9	imagemagick@6
2.7.0-84-g8ac02c9	libhandy
2.7.0-84-g8ac02c9	libstrophe
2.7.0-84-g8ac02c9	tokei
2.7.0-86-g5143b86	bartycrouch
2.7.0-86-g5143b86	bumpversion
2.7.0-86-g5143b86	diceware
2.7.0-86-g5143b86	eksctl
2.7.0-86-g5143b86	folderify
2.7.0-86-g5143b86	gcalcli
2.7.0-86-g5143b86	micronaut
2.7.0-86-g5143b86	okteto
2.7.0-86-g5143b86	pdfcpu
2.7.0-86-g5143b86	sphinx-doc
2.7.0-86-g5143b86	swiftformat
2.7.0-88-g96c28d2	chkrootkit
2.7.0-88-g96c28d2	flow-cli
2.7.0-88-g96c28d2	get-flash-videos
2.7.0-88-g96c28d2	gitless
2.7.0-88-g96c28d2	nexus
2.7.0-88-g96c28d2	orbit
2.7.0-88-g96c28d2	plowshare
2.7.0-88-g96c28d2	raylib
2.7.0-88-g96c28d2	whistle
2.7.0-90-g5521c76	apcupsd
2.7.0-90-g5521c76	chuck
2.7.0-90-g5521c76	lasi
2.7.0-90-g5521c76	mpop
2.7.0-90-g5521c76	orientdb
2.7.0-90-g5521c76	pakchois
2.7.0-90-g5521c76	pdftoipe
2.7.0-90-g5521c76	tin
2.7.0-92-g7b68d21	dfc
2.7.0-96-gd9064eb	aws-google-auth
2.7.0-96-gd9064eb	benthos
2.7.0-96-gd9064eb	borgbackup
2.7.0-96-gd9064eb	coturn
2.7.0-96-gd9064eb	cucumber-cpp
2.7.0-96-gd9064eb	doctest
2.7.0-96-gd9064eb	git-remote-codecommit
2.7.0-96-gd9064eb	git-revise
2.7.0-96-gd9064eb	glow
2.7.0-96-gd9064eb	hiredis
2.7.0-96-gd9064eb	htmldoc
2.7.0-96-gd9064eb	libjson-rpc-cpp
2.7.0-96-gd9064eb	moarvm
2.7.0-96-gd9064eb	nqp
2.7.0-96-gd9064eb	openvdb
2.7.0-96-gd9064eb	rakudo
2.7.0-96-gd9064eb	vit
2.7.0-98-g3f4427e	hamlib
2.7.0-98-g3f4427e	notmuch
2.7.0-98-g3f4427e	promtail
2.7.0-100-gda0d7ef	abseil
2.7.0-100-gda0d7ef	asio
2.7.0-100-gda0d7ef	cargo-watch
2.7.0-100-gda0d7ef	cocoapods
2.7.0-100-gda0d7ef	cucumber-ruby
2.7.0-100-gda0d7ef	dnscontrol
2.7.0-100-gda0d7ef	dvc
2.7.0-100-gda0d7ef	grpc
2.7.0-100-gda0d7ef	hqx
2.7.0-100-gda0d7ef	imagejs
2.7.0-100-gda0d7ef	jql
2.7.0-100-gda0d7ef	libarchive
2.7.0-100-gda0d7ef	libcddb
2.7.0-100-gda0d7ef	libstfl
2.7.0-100-gda0d7ef	macosvpn
2.7.0-100-gda0d7ef	macvim
2.7.0-100-gda0d7ef	mdbtools
2.7.0-100-gda0d7ef	mkvtomp4
2.7.0-100-gda0d7ef	mps-youtube
2.7.0-100-gda0d7ef	nfpm
2.7.0-100-gda0d7ef	ngs
2.7.0-100-gda0d7ef	nicotine-plus
2.7.0-100-gda0d7ef	passenger
2.7.0-100-gda0d7ef	pyinvoke
2.7.0-100-gda0d7ef	restview
2.7.0-100-gda0d7ef	rinetd
2.7.0-100-gda0d7ef	robot-framework
2.7.0-100-gda0d7ef	rst-lint
2.7.0-100-gda0d7ef	ruby
2.7.0-100-gda0d7ef	scour
2.7.0-100-gda0d7ef	shallow-backup
2.7.0-100-gda0d7ef	sshuttle
2.7.0-100-gda0d7ef	statik
2.7.0-100-gda0d7ef	supervisor
2.7.0-100-gda0d7ef	terraform_landscape
2.7.0-100-gda0d7ef	trailscraper
2.7.0-100-gda0d7ef	travis
2.7.0-100-gda0d7ef	vim
2.7.0-100-gda0d7ef	vitetris
2.7.0-100-gda0d7ef	weechat
2.7.0-100-gda0d7ef	ydcv
2.7.0-100-gda0d7ef	youtube-dlc
2.7.0-102-gbcbf585	agda
2.7.0-102-gbcbf585	allureofthestars
2.7.0-102-gbcbf585	bond
2.7.0-102-gbcbf585	cargo-audit
2.7.0-102-gbcbf585	check_postgres
2.7.0-102-gbcbf585	clash
2.7.0-102-gbcbf585	cubejs-cli
2.7.0-102-gbcbf585	direnv
2.7.0-102-gbcbf585	dnsx
2.7.0-102-gbcbf585	doxygen
2.7.0-102-gbcbf585	fastlane
2.7.0-102-gbcbf585	ficy
2.7.0-102-gbcbf585	fig2dev
2.7.0-102-gbcbf585	futhark
2.7.0-102-gbcbf585	ghc
2.7.0-102-gbcbf585	git-annex
2.7.0-102-gbcbf585	git-cal
2.7.0-102-gbcbf585	glab
2.7.0-102-gbcbf585	graphicsmagick
2.7.0-102-gbcbf585	harfbuzz
2.7.0-102-gbcbf585	haskell-stack
2.7.0-102-gbcbf585	hatari
2.7.0-102-gbcbf585	hledger
2.7.0-102-gbcbf585	iam-policy-json-to-terraform
2.7.0-102-gbcbf585	ircd-hybrid
2.7.0-102-gbcbf585	jpeg-turbo
2.7.0-102-gbcbf585	localstack
2.7.0-102-gbcbf585	mikutter
2.7.0-102-gbcbf585	monolith
2.7.0-102-gbcbf585	mozjpeg
2.7.0-102-gbcbf585	mpg123
2.7.0-102-gbcbf585	mysql-sandbox
2.7.0-102-gbcbf585	ocrmypdf
2.7.0-102-gbcbf585	openalpr
2.7.0-102-gbcbf585	pgroonga
2.7.0-102-gbcbf585	rethinkdb
2.7.0-102-gbcbf585	ruby@2.7
2.7.0-102-gbcbf585	st
2.7.0-102-gbcbf585	stress-ng
2.7.0-102-gbcbf585	trash-cli
2.7.0-102-gbcbf585	vcpkg
2.7.0-102-gbcbf585	vice
2.7.0-102-gbcbf585	wimlib
2.7.0-102-gbcbf585	x265
2.7.0-102-gbcbf585	xterm
2.7.0-102-gbcbf585	youtube-dl
2.7.0-102-gbcbf585	you-get

[vsevolodgarkusha]

I've just used arch -x86_64 brew install -s python and python works again on my M1.

thanks! It works for me!

[jonchang]

We've revision bumped and merged all 34 formulae with 1000+ monthly active users per analytics. The remaining 138 will get rebuilds happening in the background as we churn through the queue. Once a formula has been rebuilt, if you are using Rosetta you can simply brew reinstall <FORMULA> to get a rebuilt bottle. Thanks for identifying this issue!

[MikeMcQuaid]

Thanks @jonchang and @fxcoudert for working on this 🎉