Open refack opened 7 years ago
Fwiw the osx installer is in equal disarray
Have you seen nodejs/installer?
On Jul 14, 2017 12:18 PM, "Refael Ackermann" notifications@github.com wrote:
So for some unknown historical reason node core has a Windows installer (probably to appease the MS gods)... After a recent conversation with @ljharb https://github.com/ljharb, IMHO the installer should be in this WG's focus, since it's essentially a "management-challenged" version manager.... We are fielding issues as:
- Allow multiple installs of Node on Windows with msi https://github.com/nodejs/node/issues/4603
- Stop the installer deleting my symlink for AppData\Roaming\npm https://github.com/nodejs/node/issues/7882
- Windows installer: Per user install (feature request) https://github.com/nodejs/node/issues/13830 and the list goes on... https://github.com/nodejs/node/issues?utf8=%E2%9C%93&q=is%3Aissue%20label%3Awindows%20label%3Ainstall
Which bring up the same issues that are discussed here, like:
- where do we put the global node_modules?
- Should we install npm or not, and where?
- Does core want to officially support side-by-side multi-version, and if so how to switch versions (i.e. version manager yes or no)?
- How to detect which version are installed?
Not to diminish from this WG's efforts in the least, and not intending to step on anyone's toes!!! It's just that our questions are less "academic" since they are motivated by issues opened by users, since the Windows installer is unfortunately an existing core feature ๐คฆโโ๏ธ So we need to answer them sooner rather than later... BTW: Personally my prefered answer would have been "The Windows installer is deprecated, use the zipped package or a userland version manager" but it got push-back nodejs/node#4603 (comment) https://github.com/nodejs/node/issues/4603#issuecomment-313275229
/cc @tniessen https://github.com/tniessen @nodejs/platform-windows https://github.com/orgs/nodejs/teams/platform-windows
โ You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/nodejs/version-management/issues/16, or mute the thread https://github.com/notifications/unsubscribe-auth/AAecV0pyiI6pYJZy0SazNTJ4A22Etuujks5sN06YgaJpZM4OYIi6 .
Fwiw the osx installer is in equal disarray
Have you seen nodejs/installer?
Ohh yeah sorry I meant to mention the osx installer.
Have you seen nodejs/installer?
I keep forgetting it... But again it has the benefit of cooking behind the scenes while the Windows and osx installers are out there causing headache... /cc @nodejs/electron-installer
So fwiw this is pretty close to the conclusion we reached at at the last collaborators Summit
We need to completely overhaul our installers, and likely move towards a version manager.
Likely move away from /user/local, and try and standardize where stuff loves to make it easier to move between tools!
The latest work has been happening in the installer repo, but we should perhaps make a more directed effort. This should like be a joint effort between build and release
On Jul 14, 2017 12:21 PM, "Refael Ackermann" notifications@github.com wrote:
Fwiw the osx installer is in equal disarray
Have you seen nodejs/installer?
Ohh yeah sorry I meant to mention the osx installer.
โ You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/nodejs/version-management/issues/16#issuecomment-315336236, or mute the thread https://github.com/notifications/unsubscribe-auth/AAecV6SqOkNVtjphjrfpIPWL5_ytZfUmks5sN08jgaJpZM4OYIi6 .
Hey, I think our install should have the following properties:
I think it's worth saying that overall our installers - while in disarray - work pretty well. Especially compared to other platforms and stacks (installing Node is much simpler than installing Python+Flask for example which is one of the simpler stacks).
I'm -1 on any option that doesn't include NPM as well (since it is a vital tool for developing Node.js with our small core). I'm fine with revisiting letting the user pick a package manager - but honestly NPM works well enough for most users with most use cases IMO anyway.
@refack a replacement has to happen in the background... I have yet to hear of a sustainable way of building on top of what we currently have. I totally support driving more aggressively towards this happening, we just need people to invest time
On Jul 14, 2017 12:32 PM, "Benjamin Gruenbaum" notifications@github.com wrote:
Hey, I think our install should have the following properties:
- Installable from package managers on the 3 biggest platforms (choco on windows, brew on linux, apt/pacman/whatever on linuxes).
- Installable as an installer on Windows and Mac where this sort of installation is popular. For what it's worth the windows/mac installers do a pretty good job today - and for the 95% of users they work pretty well.
- A standalone installer would be nice (that doesn't touch anything global) for all three platforms - personally I just grab the source and build when I need this but I don't think many users do that.
I think it's worth saying that overall our installers - while in disarray
- work pretty well. Especially compared to other platforms and stacks (installing Node is much simpler than installing Python+Flask for example which is one of the simpler stacks).
I'm -1 on any option that doesn't include NPM as well (since it is a vital tool for developing Node.js with our small core). I'm fine with revisiting letting the user pick a package manager - but honestly NPM works well enough for most users with most use cases IMO anyway.
โ You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/nodejs/version-management/issues/16#issuecomment-315337959, or mute the thread https://github.com/notifications/unsubscribe-auth/AAecVw8tyHDsK8K9r6XODAvBpKCaIBLZks5sN1G8gaJpZM4OYIi6 .
@refack a replacement has to happen in the background... I have yet to hear
Agreed, my question is what do we do meanwhile? Improve the existing installers? Recommend userland VerMan (nvm
/ nvs
) ? Say: wait for "The Installer"?
BTW: As I see it any work we do meanwhile on the windows (MSI) installer, could and should be harnessed the "The Installer" and any VerMan
@refack I'll just say this clearly one more time - I don't think the situation with the windows installer at the moment is all that bad - and neither is the situation with the macOS installer. I'd like any new installer you work on to be at least as easy to use as the existing installers since environment set up is a huge problem in my experience for new users.
(Of course - I agree that we can do a better job with both the installers and I think you're doing an amazing job being on top of the windows platform, the installer, and issues people have - which historically Node didn't do as good of a job with as other platforms.)
@refack I'll just say this clearly one more time - I don't think the situation with the windows installer at the moment is all that bad - and neither is the situation with the macOS installer. I'd like any new installer you work on to be at least as easy to use as the existing installers since environment set up is a huge problem in my experience for new users.
Our (@tniessen and I's) efforts are not to replace, but only to improve the existing Installer.
P.S. MSI installers packages (which the existing Installer is) are officially supported by Microsoft, use OS APIs, and are dependent on by our users https://github.com/nodejs/node/issues/4603#issuecomment-313324044. So AFAICT even "The Installer" will need to live side-by-side with.
There are geniune problems with the widows installer regarding how it handles the non directory and inability to remove files. We are also limited in expanding functionality.
On Jul 14, 2017 12:52 PM, "Refael Ackermann" notifications@github.com wrote:
@refack https://github.com/refack I'll just say this clearly one more time - I don't think the situation with the windows installer at the moment is all that bad - and neither is the situation with the macOS installer. I'd like any new installer you work on to be at least as easy to use as the existing installers since environment set up is a huge problem in my experience for new users.
Our (@tniessen https://github.com/tniessen and I's) efforts are not to replace, but only to improve the existing Installer
โ You are receiving this because you commented.
Reply to this email directly, view it on GitHub https://github.com/nodejs/version-management/issues/16#issuecomment-315341323, or mute the thread https://github.com/notifications/unsubscribe-auth/AAecVwrvW3T6yUZQn2HAg0kO-J2SVJANks5sN1Z-gaJpZM4OYIi6 .
There are geniune problems with the widows installer regarding how it handles the non directory and inability to remove files. We are also limited in expanding functionality.
@refack I'll just say this clearly one more time - I don't think the situation with the windows installer at the moment is all that bad - and neither is the situation with the macOS installer. I'd like any new installer you work on to be at least as easy to use as the existing installers since environment set up is a huge problem in my experience for new users.
Just to add, Installers are hammers not surgical scalpels. They assume stuff about your system, and they read/write/delete system files in an irreversible way. Any bug can lead to a catastrophe for the user.
I think I can say, without a doubt, that whatever version management solution we end up eventually coming up with and shipping as part of node will have the following characteristics:
I'm sure there are more things that can go on this list, but that's just some things off the top of my head.
Other thoughts:
node_modules
must not be shared across node versions; every version of node needs a separate reinstall of packages, for compilation reasons if nothing else (there's also packages that dynamically load different JS based on which node version it's running in, and this could easily happen at postinstall time too).npm
comes with node always. It should never be installed separately, and it should always start out initially as whatever version ships with that stock node version. If people want an option to npm install -g npm
at the same time, that's great, but that should be the mechanism by which it's upgraded.node -v
detects which version is installed.I think there are two important questions that need to be answered before we can move forward to proposing solutions:
Is it a requirement that Node installers use the OS-specific installation technologies? For Windows, many people, especially system administrators, expect software to be delivered via MSI packages. MSI installers have consistent command-line options, can be logged, inventoried, deployed via group policy, etc. I'm less familiar with installers on the Mac, but I'd imagine there are similar considerations. Are there benefits of moving away from the OS installation technologies that would outweigh the drawbacks?
Should Node installers participate in / be aware of version management? Currently version manager tools do not actually use the platform specific installer packages; they always download and extract the .tar.gz
/.zip
packages directly (as far as I know). And then the version-manager needs special-case logic to deal with any Node version that might have been installed separately.
Everything written here is my option and open to debate ๐
Preamble: I don't like the Windows MSI installer. I think it's unfortunate, but we're stuck with it. It being a part of core is doubly unfortunate. Ideally it would have been a 3rd party product like nodesource/distributions
, nvm
or nvs
. Less ideally but better then the status quo, it should be a separate project like libuv
, or npm
and not an afterthought of core
.
I think this WG's existence is indicative of that we can all agree that an Installer's requirements and knowledge-base is very different from those of the "pure" node platform.
@ljharb, My perspective re the state of current Windows installer
- the global node_modules must not be shared across node versions;
This is not true for the current installer, hence a bug need fixing.
- npm comes with node always.
Why? (not arguing I'm just lacking context).
- side-by-side different versions is version management, and should not be supported until the solution mentioned above is ready.
That's a bit of a pickle. In that sense the Windows Installer already is a de-facto degenerate VerMan. Which we need to keep supporting... I need to split some hairs here, regarding "enabled" vs. "supported". Side-by-side versions is a scenario that has always been enabled and maybe even supported, and has great value. e.g. my computer:
IMHO the Installer could enable that (even as an undocumented feature not "Officially" supported) just by working right. I agree that it's not a scenario that we want to "Support". But I don't see a reason not to enable it. Let's discuss...
- node -v detects which version is installed.
A corollary of (3)... I'm not sure I see the significance of that. As I see it the bare node
command is just a convenience...
@jasongin
- Is it a requirement that Node installers use the OS-specific installation technologies?
Unfortunately MSI use is grandfathered in. Even if the VerMan will not use the MSI, AFAICT the MSI will have to continue being available. Maybe we could spin it off as a sub-project... I don't know, but hope...
- Should Node installers participate in / be aware of version management?
I see no reason why not, even ad hoc. Using a global registry like in PEP514, or a some other means of discovery.
Oooffffff that turned out long... To sum. IMHO we (node-core) are already in the version management business like it or not ๐คท
Is it a requirement that Node installers use the OS-specific installation technologies?
I would assume so, yes, you have outlined very good reasons yourself.
@refack and I started to work on a solution for https://github.com/nodejs/node/issues/4603, which has been open for 1.5 years without any progress.
Should Node installers participate in / be aware of version management?
Not necessarily IMO. The problem discussed in https://github.com/nodejs/node/issues/4603 is not that there is no way to switch between different versions, users can do that on their own. But our current MSI actively prevents different node.js versions from being installed at the same time, so users need to do the whole setup on their own as soon as they want to install multiple different versions, including permissions, npm and PATH configuration.
Our idea was to identify each node.js release as a separate "product" (MSI terminology) and put these products into folders C:\Program Files\nodejs\<version>
, each having its own node_modules
. Each version can be installed, modified and uninstalled independently, without having to use any kind of version manager, not even to set the PATH.
@ljharb suggested not to do any platform-specific work in this area, so we were curious about this WG's plans to solve the problems on Windows in the foreseeable future.
Our idea was to identify each node.js release as a separate "product" (MSI terminology) and put these products into folders C:\Program Files\nodejs\
, each having its own node_modules. Each version can be installed, modified and uninstalled independently, without having to use any kind of version manager, not even to set the PATH. @ljharb suggested not to do any platform-specific work in this area, so we were curious about this WG's plans to solve the problems on Windows in the foreseeable future.
Transcript of @tniessen and I's conversation https://gist.github.com/refack/e9d1371a1c2530b7ff48a432fafea1af
@refack Intentional enabling is support - as opposed to unintentional enabling.
npm comes with node always.
Why? (not arguing I'm just lacking context).
Because that's how node works, modulo the option to exclude npm - npm
also requires node, so it doesn't make sense to install it without node, and npm is node-version-sensitive, so it's important to correlate npm and node versions. There's just no scenario where you need npm without node (altho there may be scenarios where you want node without npm).
This is not true for the current installer, hence a bug need fixing.
Agreed.
That's a bit of a pickle. In that sense the Windows Installer already is a de-facto degenerate VerMan. Which we need to keep supporting...
Things that are unintentionally enabled aren't things we necessarily need to keep supporting.
That's a bit of a pickle. In that sense the Windows Installer already is a de-facto degenerate VerMan. Which we need to keep supporting...
Things that are unintentionally enabled aren't things we necessarily need to keep supporting.
I'd be very happy if we can get consensus on that ๐
Because that's how node works, modulo the option to exclude npm - npm also requires node, so it doesn't make sense to install it without node, and npm is node-version-sensitive, so it's important to correlate npm and node versions. There's just no scenario where you need npm without node (altho there may be scenarios where you want node without npm).
A bit of an academic discussion if we agree that the global node_modules
has to be in 1-1 correlation with an installed node version but...
In MSI terms one can define a dependency chain where one "product" (in this case npm
) can only be installed by another product (in this case node
), but later could be uninstalled/fixed/updated independently. If the parent is uninstalled that propagates to uninstalling the dependant.
That seems to describe our case very well, and it may solve some implementation details with the Windows installer, so we would like to keep that option open.
This also might give a better sense of the responsibility boundary between node
and npm
...
If node is uninstalled, npm will stop working.
npm
can't exist independent of node
, and it makes no sense to try to create a world where that's the case.
If node is uninstalled, npm will stop working.
npm can't exist independent of node, and it makes no sense to try to create a world where that's the case.
Yes agreed, but we can enforce that via MSI product dependencies.
Just an example, for stuff currently installed on my computer:
Two kinds of dependencies:
P.S. it's also an example of side-by-side versions:
I think only npm should be managing npm; so I wouldn't expect npm to show up in that installed program list.
I think only npm should be managing npm; so I wouldn't expect npm to show up in that installed program list.
I assume you mean node in the first use of npm.
That's the debatable point since npm
is managed by npm.com
but it does not appear in the "Installed Programs" ๐ even though it's simply piggy backing onto the node
installer.
But I'm not gonna bike-shed over that until I see an immediate and substantial benefit in splitting them.
So I guess the real question is, is "side-by-side installation" something node core officially supports (prior to any version management scenario), or merely enables (inconsistently across platforms)?
If the former, then obviously the Windows installer should add it, and there should be documentation for every platform on how to do it - and then a number of other questions in this thread become relevant.
If the latter, however, then it's totally fine if one platform enables it and another does not - because it's out of scope for node core - and the rest of this thread is moot as it relates to current node policy.
I'd argue it's solidly the latter - does anyone believe it's the former?
we can enforce that via MSI product dependencies.
Visual Studio has a sophisticated external installer that orchestrates all the MSI dependencies. (I worked on parts of it ~10 years ago.) That functionality is not at all a built-in to MSI. So it would take considerable work to replicate.
I'd argue it's solidly the latter - does anyone believe it's the former?
My personal 2ยข - we should make the installer as minimal as possible while still being consistent. There are a few features it has I would actually love to deprecate. If we have consensus around that I'd be happy to convey that to the users and point them to best-of-bread userland solutions.
Re: side-by-side (SxS) I have no problem disavowing "SxS Installation" but "SxS existence" is a super important feature (implying 0 global state).
Also while "The Installer" is cooking I'd lobby for spinning off the Windows (and probably the macOS) installer to a sub-project. Under this WG's responsibility (as "The Installer" should be)
So I guess the real question is, is "side-by-side installation" something node core officially supports (prior to any version management scenario), or merely enables (inconsistently across platforms)?
Neither of these options would prevent the Windows installer from allowing side by side installation, would it? It does not make the situation worse than it is (as far as I know), so I don't fully understand your opposition to this.
@tniessen yes. Intentionally adding support for something is very different from it having been an emergent property. If something is not "supported", then it must not ever be added intentionally, but it also need not be removed if it happens to exist.
To clarify; intentionally adding support for a feature makes it supported.
Meaning: ๐ True ๐ Bad ๐ Not happy that it's true
If something is not "supported", then it must not ever be added intentionally, but it also need not be removed if it happens to exist.
Okay, I see your point. Then I guess it is up to this WG to decide whether side-by-side installation on Windows should be "supported" or the current behavior should be kept, effectively actively preventing the installation of different versions. Even with third-party version managers in place, IMO side-by-side installation on Windows using official MSIs is justified.
If something is not "supported", then it must not ever be added intentionally, but it also need not be removed if it happens to exist.
Why can't you ever add something that's unsupported? We have an entire class of core APIs that are experimental
(i.e. unsupported), and that stance precludes us ever testing anything before we ship a stable version. I'm also not sure why it matters if you meant to add something.
Things that are unintentionally enabled aren't things we necessarily need to keep supporting. To clarify; intentionally adding support for a featureย makesย it supported.
If a significant number of people rely on a feature it basically becomes supported no matter what the original intent was (or whether it was accidentally added).
So I guess the real question is, is "side-by-side installation" something node core officially supports (prior to any version management scenario), or merely enables (inconsistently across platforms)?
I think it's something that node core is considering officially supporting, as it's clearly a very common use case (hence this Discussion Group right?) However we'd definitely want to experiment first.
@gibfahn surely it's intended to be supported in some sense; my question is, prior to the VM group releasing something is it something node core wishes to support?
My guess is no.
Regarding experimental apis, installer features aren't APIs, so it's a different beast.
I did some complex Windows MSI package 15 years ago... I can't believe that you can't more easily build one today for Node !
@doodadjs please reconsider your comment as I don't feel it adds something constructive to the discussion. Node is already installed by an MSI installer - and this discussion is about how doing a better job with edge cases (like multiple installs) and is about specifying behavior.
If you have extensive experience with windows packaging - I would love it if you weighed in on the issues discussed above.
@benjamingr By reading the comments, I had the feeling that people wanted no MSI at all
My experience with MSI is very old. But I remember you can group things in modules (don't remember the exact term) then include them in MSI. Everything has a UUID that you must keep, or sometimes discard. I also remember that you can embed executables and call them in your MSI. You can also build interfaces, etc., etc. But like I said, that's 15 years ago.
I'm probably barging in here, but considering what I went through these past 2 weeks just trying to set up Node on a Windows machine that will need to build multiple native modules at a new client of mine, I'd say the Windows installer should probably include an installation of all required build tools (windows-build-tools project just isn't enough), this should probably include the c++ redist package, python 2.7 and the MASM files (ml64.exe. and ml.exe with their dependent dlls). It was a nightmare finding working solutions on the web and setting them up correctly (especially the MASM files which get installed only if you create a VS c++ project and add the correct build dependency) I know it sounds like I'm whining, and that's because I AM. Seriously, "ease of use" should still be a priority, shouldn't it?
@RobiFerentz i don't think you'll find any disagreement on any of that :-D but I think that's probably tangential to the question of whether the Windows installer should be able to install multiple versions at once; your comment seems more about how well it installs one version.
@ljharb I know, but whatever decision is made, I think there are more pressing issues with Node on Windows than single or multiple version installations (besides the actual idea of developing a node app on windows which in itself is problematic :gun:).
@RobiFerentz have you tried windows-build-tools
? It was created by Microsoft employees to try to ease the pain of setting things up on Windows. It's documented in the node-gyp README, but it should probably be somewhere more prominent, like the website.
@gibfahn I mentioned the project in my first comment. They're just not enough, and there's plenty of other crap to work around depending on your machine architecture, which VS Studio version you are using, etc ad nauseam.
@ljharb @RobiFerentz - no arguments... the pain of native builds is substantial. One factor that we may need to consider within this group is if and how native build tools traverse different versions and architectures.
For example, Windows still needs to support both 32 and 64-bit architectures. For native modules, supporting both architectures per version can potentially mean a huge amount of build tool redundancy (multiple versions of VS for older versions, isolating windows-build-tools
per version). This can (at least with older versions of node) translate to a large footprint of unusable levels.
While I agree the pain @RobiFerentz addresses focuses on a single version, the problem compounds with multiple versions. I fear we're treading in "shared global node_modules" territory, in the sense that sharing build tools would be more efficient (if it could even work), but opens a floodgate for potential problems.
I don't have a good answer for this, but I do think it's something we need to keep on our radar. Even if the Windows installer ultimately handles all of this for a single install, we still have to address what the implications are across multiple installs.
While I agree the pain @RobiFerentz addresses focuses on a single version, the problem compounds with multiple versions. I fear we're treading in "shared global node_modules" territory, in the sense that sharing build tools would be more efficient (if it could even work), but opens a floodgate for potential problems.
I don't have a good answer for this, but I do think it's something we need to keep on our radar. Even if the Windows installer ultimately handles all of this for a single install, we still have to address what the implications are across multiple installs.
Not necessarily. Native modules (๐ญ) depend on node-gyp
and the "node sdk" (i.e. *-headers.tar.gz) which are backward compatible with node version 0.8 and up. And only need one toolset (Preferably VS2015, but as far as I recall some stuff will work with VS2013, and VS2017 is fully compatible)
P.S. windows-build-tools
use npm
only as a bootstrapper, eventually they are installed globally, and there's no need to require('windows-build-tools')
at any level
P.P.S. I'm compiling on a simpler and more complete windows-build-tools
that could theoretically be node_modules "local"
Just stumbled across this thread, wanted to mention that ps-nvm actually does download and use the msi distributions on Windows to install multiple Node versions side-by-side:
(x-posting from https://github.com/nodejs/node/issues/4603)
Sooo, I'm thinking:
Actually, none of the points in 3. are contradictory, and I happen to (intentionally) support all of them.
From nodejs/node#4603, 3.a appears to be consensus. Does anyone object to 3.a?
Awesome, that's settled, then :)
So for some unknown historical reason node core has a Windows installer (probably to appease the MS gods)... After a recent conversation with @ljharb, IMHO the installer should be in this WG's focus, since it's essentially a "management-challenged" version manager.... We are fielding issues as:
Which bring up the same issues that are discussed here, like:
node_modules
?npm
or not, and where?core
want to officially support side-by-side multi-version, and if so how to switch versions (i.e. version manager yes or no)?Not to diminish from this WG's efforts in the least, and not intending to step on anyone's toes!!! It's just that our questions are less "academic" since they are motivated by issues opened by users, since the Windows installer is unfortunately an existing core feature ๐คฆโโ๏ธ So we need to answer them sooner rather than later... BTW: Personally my prefered answer would have been "The Windows installer is deprecated, use the zipped package or a userland version manager" but it got push-back https://github.com/nodejs/node/issues/4603#issuecomment-313275229
/cc @tniessen @nodejs/platform-windows