docker cask tries to put files in /usr/local/bin instead of /opt/homebrew/bin on apple silicon machines

charlieok commented 9 months ago

Verification

[X] I understand that if I ignore these instructions, my issue may be closed without review.
[X] I have retried my command with --force.
[X] I ran brew update-reset && brew update and retried my command.
[X] I ran brew doctor, fixed as many issues as possible and retried my command.
[X] I have checked the instructions for reporting bugs.
[X] I made doubly sure this is not a checksum does not match / SHA256 mismatch error (do not open an issue before trying to open a PR to fix first).

Description of issue

These lines in the docker cask attempt to place files in /usr/local/bin, a hardcoded path:

https://github.com/Homebrew/homebrew-cask/blob/10f49a445c3295218ce178bbc3bcd8a8e17de474/Casks/d/docker.rb#L59-L76

However, my homebrew install is entirely in /opt/homebrew , as is the norm on Apple Silicon machines. I have nothing in /usr/local and don't want homebrew to put anything there.

https://docs.brew.sh/FAQ#why-is-the-default-installation-prefix-opthomebrew-on-apple-silicon

It would make sense, I think, to have these use #{HOMEBREW_PREFIX} just like the lines directly underneath.

Command that failed

brew install --cask docker --verbose --debug

Output of command with `--verbose --debug`

Error: Failure while executing; `/usr/bin/sudo -E -- /bin/mkdir -p -- /usr/local/bin` exited with 1. Here's the output:
Sorry, user homebrew_admin is not allowed to execute '/bin/mkdir -p -- /usr/local/bin' as root on Cs-M2-MacBook-Pro.local.

This failed because I had not given the homebrew_admin user permission to run mkdir as root. I was taking a look to decide if it should have that permission, and then I saw that it was attempting to add files to /usr/local/bin which is not the path it should be using, so that's the issue I'm reporting here. In this case, the restricted permission helped me notice the incorrect path.

Output of `brew doctor` and `brew config`

homebrew_admin@Cs-M2-MacBook-Pro ~ % brew doctor
Your system is ready to brew.

homebrew_admin@Cs-M2-MacBook-Pro ~ % brew config
HOMEBREW_VERSION: 4.1.22
ORIGIN: https://github.com/Homebrew/brew
HEAD: 251a098fbf1c02ca95f53b4806e5b6da8d5e4a80
Last commit: 3 days ago
Core tap JSON: 29 Nov 08:02 UTC
HOMEBREW_PREFIX: /opt/homebrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_MAKE_JOBS: 12
Homebrew Ruby: 2.6.10 => /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby
CPU: dodeca-core 64-bit arm_blizzard_avalanche
Clang: 15.0.0 build 1500
Git: 2.39.3 => /Library/Developer/CommandLineTools/usr/bin/git
Curl: 8.1.2 => /usr/bin/curl
macOS: 14.1.1-arm64
CLT: 15.0.0.0.1.1694021235
Xcode: N/A
Rosetta 2: false

Output of `brew tap`

homebrew_admin@Cs-M2-MacBook-Pro ~ % brew tap
homebrew_admin@Cs-M2-MacBook-Pro ~ %

(nothing)

razvanazamfirei commented 9 months ago

This is primarily a docker issue.

We've had a few discussions when we first ran into the problem. See https://github.com/Homebrew/homebrew-cask/pull/147086#issuecomment-1546322878. https://github.com/Homebrew/brew/pull/15413

charlieok commented 9 months ago

I'm not sure I understand. Your second link is to a comment that is making essentially the same point I'm making. That comment is not answered by any opposing considerations.

I think the correct behavior here for homebrew is to stick to its stated system of where files belong. If docker turns out to actually need, or have code that insists on, a hardcoded path like /usr/local/bin, then I'd agree that would be a docker issue that docker should fix.

As a user of homebrew, I should at least have a way to say "No really, put these files in $HOMEBREW_PREFIX". Given the way this cask is written, it doesn't seem like I do, but please let me know if there is one.

bevanjkay commented 9 months ago

I think the correct behavior here for homebrew is to stick to its stated system of where files belong. If docker turns out to actually need, or have code that insists on, a hardcoded path like /usr/local/bin, then I'd agree that would be a docker issue that docker should fix.

I believe this is the case here.

charlieok commented 9 months ago

Even in that case, I still want to be able to tell homebrew to stick to $HOMEBREW_PREFIX

SMillerDev commented 9 months ago

Even in that case, I still want to be able to tell homebrew to stick to $HOMEBREW_PREFIX

You can if you maintain a copy of this cask in your own tap.

I think the correct behavior here for homebrew is to stick to its stated system of where files belong.

It does. Formulae go in /opt/homebrew on ARM and casks go wherever they damn well please. We can't control where upstream wants tools to go, and in this case it would break docker just so people can feel good about it being in a different directory. I don't think that's worth the breakage.

charlieok commented 9 months ago

Ah, ok, that makes sense. So casks are a bit of a wild west already.

Now I'm wondering if I could modify this particular cask to watch for an option passed from the command line and/or an environment variable, which would tell it to force putting these files in HOMEBREW_PREFIX instead of wherever docker might think they should be.

bevanjkay commented 9 months ago

@charlieok For a customised variant you best option may be to host a version in your own tap, then you can configure it in a way that is perfect for you.

charlieok commented 9 months ago

Sharing some updated thoughts in case they're useful to anyone else following a similar path.

This started a snowball that wound up with me thinking in a completely different direction. Like, if docker desktop wants to place files that way, I'm soured on the whole idea of using docker desktop at all.

The comment about how formulae are handled more strictly while "casks go wherever they damn well please" set me looking at docker formulae. But upon installing the 'docker' formula, it seemed I had a docker CLI client but no docker deamon brew service.

Maybe there is a step I missed that would have gotten a dockerd brew service, but while casting about for that I was faced with the fact that docker really isn't the central thing for containers anyway here in 2023 (see kubernetes deprecation of dockershim and containerd as its default container runtime). So I'm increasingly liking the idea of a setup built around containerd instead of dockerd, with an ecosystem of tools that fit well with that choice (e.g. nerdctl as client CLI). So now I'm looking at Lima which seems made for that use case on macOS.

I realize this is now far afield of the original issue. I still think it's bad that a cask tried to put files in the wrong place, but I get that cask maintainers need to work with whatever the upstream project has published. I guess I have to take that as a known downside of the cask approach and deal with it ¯_ (ツ)_/¯. If anyone wants to close this issue, that's fine with me.

bevanjkay commented 9 months ago

@charlieok The difference is that a cask is just a way to supply software in essentially the same form as if you downloaded it directly form the developer (but in an automated way). Whereas all formula are built from source by Homebrew, meaning that configurations can be adjusted (when absolutely necessary) to provide the best experience for the end user.

In this case, it is a choice by the developers of the Docker Desktop application that the cli tools are installed in /usr/local/bin.

https://docs.docker.com/desktop/settings/mac/#advanced-1 It looks like after installation you can choose to have Docker Desktop place the CLI tools in $HOME/.docker/bin instead, under the "advanced" tab in settings.

charlieok commented 9 months ago

Reflecting back on this, I want to apologize for the whole "this is wrong" tone, but also explain a bit more the source of my original frustration. This was born from the frequently asked questions about sudo and permissions used with homebrew.

Changing a system-level directory like /usr/local to belong to one particular user instead seemed kludgy to me, so I took special notice of this paragraph in the FAQ:

"If you need to run Homebrew in a multi-user environment, consider creating a separate user account specifically for use of Homebrew."

https://docs.brew.sh/FAQ#why-does-homebrew-say-sudo-is-bad

And so I did exactly that, and made a user called homebrew_admin. Since homebrew's convention for apple silicon was to use /opt/homebrew, making that directory owned by homebrew_admin seemed like a great way forward. That choice of path seemed like homebrew saying "this new processor architecture (apple silicon) presents an opportunity for us to move to a better fileystem path going forward". It made sense: /opt is a traditional location for package managers to use, and this path has "homebrew" explicitly in it. Make homebrew_admin the owner of /opt/homebrew. That (and /Applications for casks) is where it will put its stuff, and I can keep things relatively scoped around that.

That worked for a number of casks until I got to docker, and then I hit an error when it tried to put stuff in /usr/local. That prompted a reaction of "homebrew_admin, via brew casks, is overstepping its permissions". But of course there are other pieces of software that offer to install command line helpers in /usr/local, and if the pattern with casks is to go ahead and add those at install time, the same logic would apply to any app in that category.

So what I'm left with is how to make this work with the "homebrew has its own user" setup method. Do I make homebrew_admin the owner of /usr/local/*? Or just make sure it has write permissions there, and it will probably own files that it puts there? (Presumably other non-homebrew things will also be doing stuff in /usr/local) Or maybe homebrew_admin should be allowed to sudo as root in order to run the commands it attempts (mkdir in the case above)?

Homebrew / homebrew-cask