[feature request] homebrew formula for macOS

[baggiponte]

Hi! I am super hyped about this project and I am pretty sure a homebrew formula would make it even more popular and easy to install. This is the macOS mirror of #148. I have never tried making a homebrew formula, but I can dig into that and help, of course!

[jjallaire]

Hi there! Thanks for the request. We don't have expertise in homebrew recipes but we'd certainly welcome the contribution :-)

We do have 4 major external binary dependencies (each of which have some version sensitivity). They all have a way to install via homebrew (3 in the official repo and 1 hosted on GH):

• pandoc: https://formulae.brew.sh/formula/pandoc
• deno: https://formulae.brew.sh/formula/deno
• esbuild: https://formulae.brew.sh/formula/esbuild
• dart-sass: https://github.com/sass/homebrew-sass

You can see our current versions of these dependencies here: https://github.com/quarto-dev/quarto-cli/blob/main/configuration#L10-L14

The tricky thing is that I'm not sure whether Homebrew will allow you to embed private versions of these binaries within the quarto installation. Failing that, I'm also not sure whether Homebrew allows multiple side-by-side binaries with different versions (e.g. multiple versions of deno the one we depend on and the latest one). In any case, we currently assume that these binaries are available in our bin directory and if they need to be elsewhere for homebrew then some code changes might be necessary.

All other dependences are JS libraries that are checked in to source control.

We do currently create a tarball-style installation for the Mac in this script: https://github.com/quarto-dev/quarto-cli/blob/main/package/src/macos/installer.ts

[baggiponte]

Thank you for collecting the resources! I'll have a look into that. I don't really get what you mean by "I'm not sure whether Homebrew will allow you to embed private versions of these binaries within the quarto installation" - you refer to version conflicts and make sure that installing different versions of - say - deno might mess up other binaries and/or the PATH?

[jjallaire]

Yes, exactly. If we are using an older version of deno and another project is using a newer version. They may or may not have a way to reconcile this.

Note that we will soon be updating to the most recent version of deno at which point perhaps just taking whatever is there will work.

On Mon, Dec 6, 2021 at 1:32 PM baggiponte @.***> wrote:

Thank you for collecting the resources! I'll have a look into that. I don't really get what you mean by "I'm not sure whether Homebrew will allow you to embed private versions of these binaries within the quarto installation" - you refer to version conflicts and make sure that installing different versions of - say - deno might mess up other binaries and/or the PATH?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/quarto-dev/quarto-cli/issues/203#issuecomment-987044420, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAZPR5FUV3J6L3BKBRMM2TUPT6S5ANCNFSM5JKYZ5PA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[baggiponte]

Hi, sorry for going AWOL. Just installed quarto with homebrew, I guess I can close the issue - thanks for tackling it! One small question: since it's a CLI utility and not (yet) a GUI/app, should quarto be a formula rather than a cask, much alike pandoc?

[jjallaire]

Yes, apparently you have a Homebrew recipe that just runs a MacOS pkg installer (so none of the issues I mentioned above are concerns).

Note that this version won't always reflect the very latest release. The maintainer of it, @rundel, will update it with some frequency but it likely won't track the very latest (since we release daily and homebrew won't appreciate that level of churn).

[rundel]

Quarto is installed as a cask because it is distributed as a pkg and not a source distribution. I made the initial PR to get it added but now anyone can quickly submit a PR to get the version bumped via

brew bump-cask-pr quarto --version 0.3.43

just replace the version number with the new one and it should automatically make the PR (modulus some Git configuration and ruby package installation). The maintainers tend to be super responsive and I've usually had things merged within minutes to a couple of hours.

[baggiponte]

Very interesting, thank you - will make sure to run the command if I notice the formula's not up to date.

I was wondering, can we have a cask installed in $(brew --prefix), i.e. in the same place as a formula?

[cbrnr]

@jjallaire do you have any plans to provide a normal brew formula (i.e. not a cask)? I'm asking because maybe the version sensitivities are already gone and it would now be easier to create a formula?

[jjallaire]

I don't think version sensitivities will go away anytime soon (esp. w/r/t the fact that pandoc does create backwards incompatible releases from time to time). Not sure what @rundel's take on this is and whether formula still allow for version pinning? Note that we are going to create a stable v1.0 release sometime next month which could be a target for this (however if we can't pin version w/ formula I think it still might be a non-starter).

[cbrnr]

I'm not an expert in the inner workings of Homebrew, but it does support versioned formulae: https://docs.brew.sh/Versions.html

However, this means that if quarto needs a specific version of one of its dependencies, that formula must exist.

Ideally, quarto should always use the latest versions available in Homebrew. Would it make sense to try to get rid of the version pins that it currently has? I think that would definitely make it easier to adapt to future changes. What are the version pins and why are they needed?

[jjallaire]

We can't in general get rid of version pins because multiple projects we depend on (pandoc and deno) still make backwards incompatible changes with some frequency. We could have a formula that takes latest of these with the proviso that it "should work" (and then fall back to a version in the case that we discover an incompatibility). Not sure if a formula that "should work" is better than one that "will work" though?

[cbrnr]

I think this might be worth a try. However, assuming that the current versions of pandoc and deno do work now, I would not pin them yet. If they make a breaking change, I'd first try to adapt quarto (sometimes only little changes are necessary), and only if that is not an option I'd create a pinned version formula (but only after complaining to the package developers 😄). All of this would make quarto a nice command line tool that's easy to install on macOS (brew cask is also pretty easy already, but requires the activation of the cask tap and the download of quite a lot of package metadata, and it is installed in an unconventional location then).

[jjallaire]

Okay, that seems sensible. @rundel, what do you think of turning quarto into a formula rather than a cask?

[rundel]

Have a formula version would be lighter weight and wouldn't require the pkg which might be a small quality of life improvement.

I'm not an expert on all things homebrew but I don't think that the quarto formula would need to be versioned but rather the dependencies (pandoc, etc.) if quarto is using non-latest versions of anything. Seems like a non-trivial amount of work but would be happy to test things out if that would be helpful.

[pat-s]

Hi, I would second the need for a formula. Given that quarto is a CLI and not a GUI app, having it installed into /Applications feels wrong.

I don't have/know of any other app which places a directory into /Applications to expose a CLI in the end.

An additional benefit would be that quarto could be updated/installed without admin priviliges which apps going into /Applications need.

[cbrnr]

I completely agree @pat-s, but you can actually install the app as a normal user. You need to select "Install for me only" in the installer, then you don't need admin privileges and the app gets installed into ~/Applications. Still weird, so a proper formula would be really great.

[rundel]

I was just playing around with this a little bit and it looks like it should be pretty straight forward to make this work as a formula using the *-macos.tar.gz releases since these include the necessary bin/ and share/ folders which can just be installed into the appropriate location in the homebrew cellar. I'm not 100% sure that the homebrew folks will allow this pre-built version to be used as a formula but it also looks straight forward to use the configure and build steps from the GitHub actions to get these same folders. Basic proof of concept formula is here: https://gist.github.com/rundel/68c9cd6ea10622fd7eb958a55fec4b80

I have run into one issue with how quarto determines the quarto script path which is not working with how homebrew symlinks the quarto script. I'll describe the issue below, but I'm happy to also open a separate issue if that would be more helpful.

quarto installed via the above formula is located as follows:

» which quarto
/opt/homebrew/bin/quarto

» ls -la `which quarto`
lrwxr-xr-x  1 rundel  admin  35 Jun  2 12:11 /opt/homebrew/bin/quarto -> ../Cellar/quarto/0.9.504/bin/quarto

running this quarto results in

» quarto
/opt/homebrew/bin/quarto: line 81: cd: /../share: No such file or directory
/opt/homebrew/bin/quarto: line 106: /tools/deno: No such file or directory

I believe that this is due to how the SCRIPT_PATH variable is being determined in the code here: https://github.com/quarto-dev/quarto-cli/blob/208ff8f68cb9d27ea2603a70882537f78f47fdf6/package/scripts/common/quarto#L4-L10 It seems like (on my system at least) the $DIR in line 8 returns nothing and using $SCRIPT_PATH instead was working for me (but I may be missing an edge case).

@jjallaire

[jjallaire]

@dragonstyle could you help out with this?

[dragonstyle]

Great work, that is indeed incorrect and SCRIPT_PATH is correct. Thanks for pinning this down - I made a commit to correct this so the latest version should work properly.

Thank you @rundel!

[cbrnr]

I just wanted to add that casks are "reserved" for GUI applications, which quarto certainly isn't. Quote from https://github.com/Homebrew/homebrew-cask:

Homebrew Cask extends Homebrew and brings its elegance, simplicity, and speed to the installation and management of GUI macOS applications such as Atom and Google Chrome.

I'm reading a little bit into Homebrew, but currently I don't know enough to be of any help in creating a formula unfortunately.

[rundel]

I haven't had much of a chance to mess with this in the last couple of weeks, but given the current release approach there is not an obvious path forward with making a quarto formula. The primary issue is that a formula must be source only which is not compatible with the current releases which contain deno, esbuild, and pandoc binaries. I have a working proof of concept formula that uses the quarto-*-macos.tar.gz releases installed to the homebrew cellar but this cant be accepted into homebrew because of the above restrictions.

A formula may be possible if someone wants to dig into the packaging system and replicate the steps into a homebrew compatible build / install process but that is beyond me for the moment.

[cbrnr]

But I assume building quarto from scratch with the latest versions must be possible, as mentioned in #1174? And if deno needs to be pinned to 1.22, Homebrew does have support for this (e.g. there's python@3.8, python@3.9, python@3.10, etc., so I'm guessing this must also be possible for deno, which BTW is available at 1.22 rn).

[pat-s]

I believe one could always make a personal tap which would not be blocked by the offical requirements of the homebrew-core tap as mentioned before. Even if this would be just a temporary solution, I'd highly appreciate it as cask updates are more cumbersome than formula ones and also always require sudo priviliges.

[rundel]

I hadn't considered the personal tap as an option - and it would work I think (brew audit will still complain about the binaries and non-native architecture if on M1). I've got a rudimentary formula working and everything seems fine on my system:

brew tap rundel/quarto-cli
brew install quarto

if you want to try it - happy to let the quarto folks take over the repo or create their own if they are good with this type of approach.

[baggiponte]

I just realised by using @rundel's tap that the official one symlinks the quarto binary to /usr/local/bin. This works for Intel macs, but does not for M1 - there, brew installs under /opt/homebrew/bin. EDIT: this belongs more to #1661, so I added a possible solution there.

[cbrnr]

I wanted to revive this issue and ask if providing an official (not private) Homebrew formula would be more feasible now, given that the latest Quarto release should build with any recent Deno version? I think the main obstacle was that Quarto required very specific Deno versions (and possibly also specific version for other dependencies, but I'm not sure), but I think this has changed a lot and maybe it would be a good time to take another stab at it?

[cscheid]

Homebrew formula would be more feasible now, given that the latest Quarto release should build with any recent Deno version?

Sorry, but what action on our part causes you to believe that?

I think the main obstacle was that Quarto required very specific Deno versions (and possibly also specific version for other dependencies, but I'm not sure), but I think this has changed a lot and maybe it would be a good time to take another stab at it?

That hasn't changed. In fact, we're currently blocked from upgrading our Deno version because of what we think is a Deno.command bug wrt output stream redirections.

[cbrnr]

No particular reason other than the time that has passed. I naively assumed that Deno might have matured enough for Quarto to remove any upper bounds (given that 1.5 is now using 1.41, which is almost the most recent 1.43).

[cscheid]

Unfortunately it's not there yet, and the stdlib changes incompatibly very often from release to release as well, and that requires additional coordination. But, just to set expectations here:

1. I wouldn't expect "works with any version of Deno" for at least ~2 years
2. We consider "works with any version of Deno" a nice to have, but we are not planning on spending significant engineering effort behind it.