Closed mistydemeo closed 3 years ago
My preference is /opt/homebrew
for Homebrew on Apple Silicon. (I'm not on the TSC.)
Thanks for starting this discussion, Misty!
Some non-Homebrew tools also install libraries and binaries in
/usr/local
, for the same reason Homebrew does, and so there are sometimes conflicts between them.
I'd say /opt/homebrew
(or really anything other than /usr/local
) is preferable in this case.
/usr/local/bin
is in the defaultPATH
I think we can still accomplish this by adding /etc/paths.d/homebrew
.
My preference is /opt/homebrew
(I’m not a TSC member).
Early feedback from people outside of Homebrew has been largely positive. I’m looking forward to getting one of the most often-cited pain points solved.
Sorry fat fingers
I think we can still accomplish this by adding
/etc/paths.d/homebrew
.
Any chance how to get launchd to pick this up, too, so GUI apps can inherit it?
That would keep brewed commands (and scripts that use them) working when launched from Xcode or a text editor.
Eventually, the need for /usr/local as a Rosetta installation of Homebrew will go away and Apple Silicon users will have only one installation.
I am unsure about this. Some software might never compile under ARM. So we will always need 2 prefixes, one for ARM, one for Rosetta.
I am unsure about this. Some software might never compile under ARM. So we will always need 2 prefixes, one for ARM, one for Rosetta.
You could use a subdirectory under /opt/homebrew
indicating the architecture to separate the installation trees, e.g. something like
/opt/homebrew/x86_64
/opt/homebrew/arm_apple # ...or whatever the standard name for the apple silicon is
I've bee using homebrew with a different prefix for many years now (/opt/homebrew
, accidentally), and setting up the PATH via /etc/paths
or a dedicated file in /etc/paths.d/
works fine. TeXLive uses the same mechanism, for example, XQuartz as well (I believe).
My preference is /opt/homebrew for Homebrew on Apple Silicon. (I'm not on the TSC.)
Can you tell me more, from a Linuxbrew perspective, how well this works for users? Do users get confused by libraries and headers not being found by compilers by default?
I am unsure about this. Some software might never compile under ARM. So we will always need 2 prefixes, one for ARM, one for Rosetta.
Rosetta isn't staying around forever. If it's like PowerPC support, we'll only have support for running Intel binaries on ARM for another few years.
Consequently, the question here is: once Apple Silicon Homebrew becomes the primary installation on Apple Silicon Macs, which should be prefix be?
/opt/homebrew
.
Reasoning: we've talked a bunch in the past about whether we should move the default prefix away from /usr/local
because:
/usr/local
/usr/local
The main reason I can see against is that e.g. RubyGems won't automatically find things in /usr/local
but, in my experience, this has been becoming more and more of a thing since we use keg-only and versioned formulae more liberally.
You could use a subdirectory under
/opt/homebrew
indicating the architecture to separate the installation trees, e.g. something like
@cmarquardt Thanks! That's something we considered but it feels inelegant to me given the existing locations. I've locked this thread not because of your (insightful) comment but just generally to keep this discussion limited to Homebrew maintainers.
I am unsure about this. Some software might never compile under ARM. So we will always need 2 prefixes, one for ARM, one for Rosetta.
Agreed @iMichka.
/usr/local/bin
is in the defaultPATH
I think we can still accomplish this by adding
/etc/paths.d/homebrew
.
In my experience it's less of an issue with PATH
and more of an issue with "default compiler lookup paths" for things like RubyGems that don't want to configure this.
In my experience it's less of an issue with PATH and more of an issue with "default compiler lookup paths" for things like RubyGems that don't want to configure this.
Yep - it's the library lookup path and the header lookup path from the compiler, not the executable PATH
.
Agreed @\iMichka.
As I mentioned earlier, Rosetta is only staying around for awhile. We should plan for it for the next 2-3 years but we shouldn't have plans that include its existence past that. 😄
As I mentioned earlier, Rosetta is only staying around for awhile. We should plan for it for the next 2-3 years but we shouldn't have plans that include its existence past that
Remember what happened with windows XP or Internet Explorer 6 ? 😄 I think it will stay for a long time as a lot of software will never get ported.
I think it will stay for a long time as a lot of software will never get ported.
A lot of PowerPC software never got ported and Rosetta 1 went away anyway. Same deal with 32-bit software support. Apple are pretty willing to yank away the rug after having given a few years' advance warning. I don't see Rosetta 2 as being any different from previous cases!
Making a comment because locking a conversation also blocks reactions (why?).
I fully agree with @mistydemeo. Apple will say bye-bye to Intel as soon as they can and won’t shed a single tear for software that no longer compiles. Tim might’ve said they would be supporting Intel Macs “for years to come”, but apparently that applied to OS support on older Macs, not making new machines (not finding the source, but Greg Joswiak mentioned it in a recent interview).
As for HB’s primary installation, no opposition to /opt/homebrew
.
I'm fine with what we came up then:
I think there is a general agreement on /opt/homebrew
.
@Homebrew/tsc, please vote by ticking your name below and change the prefix to your preferred option.
/opt/homebrew
)/opt/homebrew
)/opt/homebrew
)/usr/local
)/opt/homebrew
)I think /opt/homebrew
is the least painful of choices, so let's go with that. We'll need to document it clearly, and it will break a ton of stuff (in particular, a lot of Python packages look for libraries in /usr/local/lib
).
Looks like we have a consensus on /opt/homebrew
! I'm glad we had this discussion, and I think we've come to a good consensus.
Based on this: Mike's already done the initial work towards the /opt/homebrew
prefix on Apple Silicon, so we just need to build on that existing framework.
I'll leave this issue open for the time being so we can document the followup steps. We can close it once we have PRs for action items. Here's a proposed set of items:
/opt/homebrew
when running under Apple Silicon. (Currently we bail out on Apple Silicon due to the experimental nature of support.)brew.sh
or parent process always execs itself in x86_64
mode in a /usr/local
prefix on Apple Silicon./opt/homebrew/bin
is in the user's default PATH
(via /etc/paths.d/homebrew
?), or ensure we've documented that users must do this manually.Users would need to add /opt/homebrew
to more environment variables than PATH
in order to build against these packages, can we consider providing /opt/homebrew/setup.*sh
scripts that would configure environment variables (PATH
, LIBRARY_PATH
, DYLD_LIBRARY_PATH
, CMAKE_PREFIX_PATH
, PKG_CONFIG_PATH
, etc) for the current shell? I work with a robotics software framework that installs to /opt
and provides setup.*sh
scripts like this, and I find them very convenient. It's perhaps more essential in that case, as they have a separate installation prefix for each release (/opt/ros/melodic
, /opt/ros/noetic
, etc), but I think it could still be useful for homebrew as well.
Hmm, would brew sh
in this case add the appropriate paths?
Running commands via brew bundle exec
should also help make sure they're found.
Hmm, would
brew sh
in this case add the appropriate paths?Running commands via
brew bundle exec
should also help make sure they're found.
Those are both good suggestions for building packages in the homebrew environment, which is what my initial comment mentioned, but upon thinking further, there's also the matter of runtime when you would also want DYLD_LIBRARY_PATH
to be set if there are any plugins in /opt/homebrew
. If it's only two environment variables to enable runtime (PATH
and DYLD_LIBRARY_PATH
) then I don't think we need an entire script for that.
one quibble I have with brew sh
that would keep me from using it more regularly is that it wipes out any environment variables I might have set in my profile, but that's by design I suppose
configure environment variables (PATH, LIBRARY_PATH, DYLD_LIBRARY_PATH, CMAKE_PREFIX_PATH, PKG_CONFIG_PATH, etc)
That's the main downside of leaving /usr/local
: many build scripts and compiler tools look there by default. It was cited as an upside by some, though, so maybe we don't want to mess with variables other than PATH
. Also, the number of possible variables to add is endless, as each build system has its own.
(DYLD_LIBRARY_PATH
should not be necessary, that's for libraries copied outside of their install path, which brew does fix for us automatically.)
Update the installer to install in
/opt/homebrew
when running under Apple Silicon. (Currently we bail out on Apple Silicon due to the experimental nature of support.) Document and publicized the transition, including providing help on how to adapt to the change.
We should hold off on this until we are planning on providing official "support". This means having the majority of packages bottled and building.
Ensure
brew.sh
or parent process always execs itself inx86_64
mode in a/usr/local
prefix on Apple Silicon.
I've discussed that in https://github.com/Homebrew/brew/pull/9287#pullrequestreview-538260454 but it's worth noting also my comments in https://github.com/Homebrew/brew/pull/9126.
TL;DR: what we care about is installing things in the "wrong" prefix and not running e.g. brew info
or brew config
as the "wrong" architecture.
Either ensure that
/opt/homebrew/bin
is in the user's defaultPATH
(via/etc/paths.d/homebrew
?), or ensure we've documented that users must do this manually.
brew doctor
already enforces this.
Users would need to add
/opt/homebrew
to more environment variables thanPATH
in order to build against these packages, can we consider providing/opt/homebrew/setup.*sh
scripts that would configure environment variables (PATH
,LIBRARY_PATH
,DYLD_LIBRARY_PATH
,CMAKE_PREFIX_PATH
,PKG_CONFIG_PATH
, etc) for the current shell?
brew shellenv
is already used on Homebrew on Linux for this purpose.
DYLD_LIBRARY_PATH
doesn't work how you'd expect on modern macOS so isn't worth setting. CMAKE_PREFIX_PATH
and PKG_CONFIG_PATH
are already (at least partially) handled by our installations of both and the lack of system installations.
That's the main downside of leaving
/usr/local
: many build scripts and compiler tools look there by default. It was cited as an upside by some, though, so maybe we don't want to mess with variables other thanPATH
. Also, the number of possible variables to add is endless, as each build system has its own.
This is already a common problem with keg-only software (e.g. OpenSSL most notably) with pretty simple solutions at this point. As long as Homebrew handles our own usage and provides the necessary paths with helpers like brew shellenv
, brew --prefix wget
, etc. then the rest is up to other software to consume.
(
DYLD_LIBRARY_PATH
should not be necessary, that's for libraries copied outside of their install path, which brew does fix for us automatically.)
I have needed to set DYLD_LIBRARY_PATH
when loading a plugin that links to other libraries in a non-standard path, but it is a bit of an edge case.
brew shellenv
is already used on Homebrew on Linux for this purpose.
Great! That's just what I was looking for.
DYLD_LIBRARY_PATH
doesn't work how you'd expect on modern macOS so isn't worth setting.
If you're referring to system integrity protection, yeah that makes it really confusing. My most recent experience involved wondering why env | grep DYLD
would not show any values for DYLD_LIBRARY_PATH
when echo $DYLD_LIBRARY_PATH
does (it's because I was using /usr/bin/env
which is protected by SIP).
I have needed to set DYLD_LIBRARY_PATH when loading a plugin that links to other libraries in a non-standard path, but it is a bit of an edge case
In Catalina and later, I think it's almost unusable (disabled as a security risk). My point was: if something needs to set DYLD_LIBRARY_PATH
to work, it's basically a bug and we should treat it as such.
Heads up that the bottle code might need a small tweak if you introduce a new prefix: see https://github.com/Homebrew/brew/pull/9310 where I stumbled over an issue with bottles having different prefixes.
Feature suggestion
We need to determine which prefix we'll use as the default prefix for Apple Silicon-based Macs on a permanent basis.
A detailed description of the proposed feature
In #9117, we switched to a new prefix of
/opt/homebrew
for installations on Apple Silicon. This was written and shipped with heroic speed to help prevent strange issues with bleeding edge users on the first consumer Apple Silicon Macs. I'd like to make sure we have a conversation about our longterm plans here before we commit to something. This is an aspect with potential consequences for users, so it's important we've fully weighed the pros and cons before we come to a final decision.Today, Homebrew's prefix is
/usr/local
, with the repository in/usr/local/Homebrew
. In the initial phase of Apple Silicon migration, in order to avoid using the same prefix for both Intel and ARM binaries, we have two prefixes instead:/usr/local
for Intel, and/opt/homebrew
for Apple Silicon. Eventually, the need for/usr/local
as a Rosetta installation of Homebrew will go away and Apple Silicon users will have only one installation.Way back in the mists of history,
/usr/local
was chosen for a few reasons:/usr/local/bin
is in the defaultPATH
, so most things a user will use will end up seeing Homebrew-installed tools without configuration./usr/local/include
,/usr/local/lib
and/usr/local/lib/pkgconfig
by default, so non-Homebrew software will find Homebrew-installed libraries without configuration.These reasons were all in contrast to MacPorts, which installed its software in
/opt/local
./usr/local
was chosen in response to the issues users were experiencing at the time. We have also received some feedback from users who would prefer a different default prefix, namely:/usr/local
, for the same reason Homebrew does, and so there are sometimes conflicts between them.Consequently, the question here is: once Apple Silicon Homebrew becomes the primary installation on Apple Silicon Macs, which should be prefix be?
/usr/local
/opt/homebrew
We have a bit of time before we start creating our first bottles, so let's make sure we've discussed this in detail before then.
The motivation for the feature
Ensuring the best user experience.
How the feature would be relevant to at least 90% of Homebrew users
Eventually, close to 100% of Homebrew users will be using Apple Silicon.
What alternatives to the feature have been considered
See the discussion above.