npm & pip installers

[OJFord]

Per the readme that you're interested in adding other installers/checkers/dumpers, I think a great one would be npm and pip.

Very many npm packages do not have homebrew formulae that on other systems would be installed by the native package manager, and as a result are awkward to handle on macOS. For example, python-lsp-server & svelte-language-server; they make sense to be system-wide but presently that means doing pip install python-lsp-server and npm install -g svelte-language-server. Homebrew does have, for example terraform-ls; so now we have language servers managed by three different package managers for no particular reason.

[jacobbednarz]

pip and npm are probably widely used enough to warrant some investigation. unfortunately, i'm not one of those who would use this and don't install either of these package types globally. when talking about a per repository basis, there are already good options so i'd question what the goal of this would be and where homebrew-bundle would do a better job than the existing options.

[OJFord]

Perhaps I'm misunderstanding, but 'the existing options' are like

• nix-darwin
• system 'npm install -g ...' in brew bundle
• manually/user's own script

? There isn't really a good way for a homebrew/bundle user to do it.

[jacobbednarz]

npm has package.json and pip has requirements.txt for this sort of thing. bundle doesn't have this capability today but i personally don't see where it fits in the ecosystem against the existing options.

[colindean]

Homebrew treats its Python as a runtime dependency for Python apps, not something for the end user to use directly. This, despite common usage in this way, is perilous because a new Python version can break existing base-level package installations and virtual environments alike. I reckon NPM works the same way, although I have very little experience with it.

The problem with installing packages from pip and npm like this is apparent then: Which Python or Node installation should they be installed against? That uncertainty seems to really complicate the expression of intent manifest in the Brewfile.

Something worth thinking a little more about for the Python side of this is pipx. Homebrew's pipx formula seems to have some thought about Python version changes, so it might be unaffected by the aforementioned problem.

[MikeMcQuaid]

npm has package.json and pip has requirements.txt for this sort of thing. bundle doesn't have this capability today but i personally don't see where it fits in the ecosystem against the existing options.

This. If we wanted to do this it would be less "list all npm modules in Brewfile" and more "have a DSL for the Brewfile that runs e.g. npm install.

Homebrew treats its Python as a runtime dependency for Python apps, not something for the end user to use directly. This, despite common usage in this way, is perilous because a new Python version can break existing base-level package installations and virtual environments alike. I reckon NPM works the same way, although I have very little experience with it.

@colindean This isn't correct, sorry; please hold off giving this advice to folks, thanks ❤️

good options so i'd question what the goal of this would be and where homebrew-bundle would do a better job than the existing options.

This is the primary question that would need answered here.

Which Python or Node installation should they be installed against? That uncertainty seems to really complicate the expression of intent manifest in the Brewfile.

This is fairly easy to solve given Homebrew's existing integration with peenv and nodenv.

[OJFord]

package.json & requirements.txt are a good point, unlike Brewfiles they're not typically used for system packages so it just didn't cross my mind.

But then we still run into the issue of the system python being installed by brew as 'externally managed', so it can't be installed to without --break-system-packages. It seems weird to me to do that but not then provide a mechanism for installing to it - even those formulae that exist (e.g. python-requests) are deprecated as don't do that.

On Arch for example pacman installs python similarly, but then also packages python- (and nodejs-) libs for installing them.

If I want (for example) requests available system-wide, what am I supposed to do? Have some 'system' venv somewhere that I activate in every shell and pretend?

[MikeMcQuaid]

package.json & requirements.txt are a good point, unlike Brewfiles they're not typically used for system packages so it just didn't cross my mind.

System-wide installation of NodeJS/Python libraries does not play nicely with Homebrew so we don't recommend it really.

It seems weird to me to do that but not then provide a mechanism for installing to it

That's because you're not meant to: you're meant to install to e.g. a virtualenv or the Python modules for your user.

If I want (for example) requests available system-wide, what am I supposed to do?

Not do that.

Have some 'system' venv somewhere that I activate in every shell and pretend?

That or install this library as the user(s) running the shell(s).

[OJFord]

Is there a single other package manager that behaves like this, on any platform? You can do what you want obviously, I'm just saying it's surprising behaviour.

That or install this library as the user(s) running the shell(s).

I'm not sure what that means, just to call the venv I described $(whoami) instead of 'system'? Sure, but that's just a filename, that doesn't change anything.

[MikeMcQuaid]

Is there a single other package manager that behaves like this, on any platform?

Bundler, for example, tends to default to making everything project specific rather than system-wide.

Homebrew Bundle is heavily influenced by that. I'm the primary maintainer/contributor by a pretty long way and it is strongly influenced by my opinions and experience on how best to use package managers.

With Homebrew itself: the vast majority of other package managers on other platforms require you to do sudo whatever install and Homebrew does not. Because we don't have the root/sudo boundary to cross, we do things a bit differently.

Regardless: even if every package manager were to do it differently: that does not compel us to do it their way.

I'm not sure what that means, just to call the venv I described $(whoami) instead of 'system'?

I've never intestinally created a virtualenv so: no idea.

I have installed lots of NPM modules/RubyGems in projects and Python, Ruby, Perl modules just as my current user under e.g. $HOME/.gem, $HOME/.pip, though. Installing things systemwide is to be avoided when possible.

[binaryben]

I have installed lots of NPM modules/RubyGems in projects and Python, Ruby, Perl modules just as my current user under e.g. $HOME/.gem, $HOME/.pip, though. Installing things systemwide is to be avoided when possible.

I can't speak to Ruby and to a lesser degree Python (I'm basically inclined to agree with you for Python), but this is not completely accurate for NPM. Some tools are designed (or at least supported) to be installed globally with NPM. Serve by Vercel is one example of the top of my head I use in this way. I think NPM should be supported as an installer type.

My main concern would be how to manage actually getting NPM installed in the first place as there are a number of ways. I'm currently trialing PNPM for example to install Node/NPM. Many use nvm/n/volta. I'd recommend leaving that to the user though to ensure npm is available on the system (probably installed by Homebrew anyway)

[MikeMcQuaid]

Some tools are designed (or at least supported) to be installed globally with NPM.

They may be supported/designed that way but, speaking from experience, it's definitely not the optimal way to use said tools with Homebrew itself.

[OJFord]

So surely Homebrew is the place for a solution to what is a Homebrew problem?

Have some extra managed environment for non-dependency packages, or one per package (also solving conflicts between formulae's depenencies which presumably already has the same issue?) or whatever.

[binaryben]

They may be supported/designed that way but, speaking from experience, it's definitely not the optimal way to use said tools with Homebrew itself.

Can you elaborate for my understanding please? Are you referring to Homebrew wouldn't have the ability to install to a Homebrew specific bin directory? If so, that is something I hadn't considered. That would require setting up a mechanism like how pnpm manages packages which would be tedious

[MikeMcQuaid]

So surely Homebrew is the place for a solution to what is a Homebrew problem?

I disagree this is a Homebrew-specific problem. It will apply to any package manager where there's a systemwide or user-based installation path.

Can you elaborate for my understanding please?

If you can avoid it (e.g. by installing in the relevant project or user directory): not installing things into brew --prefix will avoid potential issues on upgrades, uninstalls, etc.

Some tools are designed (or at least supported) to be installed globally with NPM.

Also worth noting: unlike pure libraries: Homebrew already packages many of these tools e.g. https://formulae.brew.sh/formula/tailwindcss

[binaryben]

If you can avoid it (e.g. by installing in the relevant project or user directory): not installing things into brew --prefix will avoid potential issues on upgrades, uninstalls, etc.

I'm still not sure I understand. Why would using Brewfile to install an NPM module mean it needs to be installed to brew --prefix?

Also worth noting: unlike pure libraries: Homebrew already packages many of these tools e.g.

It sure does. Here is a list of some of the ones that are not pure libraries and are not packaged as formulas from my bootstrap file though:

pnpm i -g @softwaretechnik/dbml-renderer
pnpm i -g stegcloak
pnpm i -g serve
pnpm i -g surge
pnpm i -g vercel@latest
pnpm i -g @shopify/cli
pnpm i -g netlify-cli

It would be great to include these in the brewfile. They're the only things not managed by brewfile now.

[MikeMcQuaid]

I'm still not sure I understand. Why would using Brewfile to install an NPM module mean it needs to be installed to brew --prefix?

That's the global NPM installation location for an NPM installed with Homebrew.

Here is a list of some of the ones that are not pure libraries and are not packaged as formulas from my bootstrap file though

It's probably worth creating formulae for some of these, either for your own use or for submitting to homebrew/core.