search

nodejs / TSC

The Node.js Technical Steering Committee
591 stars 133 forks source link

Investigate moving `nvm` into the foundation #96

Closed ljharb closed 7 years ago

ljharb commented 8 years ago
MylesBorins commented 8 years ago

Super +1 on this happening. I don't necessarily have answer to all of those questions but I believe that is the point of the incubation process.

williamkapke commented 8 years ago

I do not think the TSC is currently accepting applications for Mentorship/Incubation/TLPs at this time. See: https://github.com/nodejs/TSC/issues/73#issuecomment-200501604 . But, I perhaps this is just to have a "what if" conversation without the formalities...

From what I understand, TLPs are something that has been determined to be "vital to the node ecosystem" and is at risk of becoming unstable without assistance. Express being a current example.

Maybe you can provide more context as to why the Node.js Foundation's resources & assistance are desired/needed?

are there any conflicts of interest, or messaging concerns, created by this project living in the foundation?

This would make it a "blessed" project which has been discussed & debated by the TSC before. I can't speak for all of the points of views on that- so someone else will need to elaborate on it.

jasnell commented 8 years ago

@williamkapke ... I'm not sure this is something that we would need to spin up too much process around... that is, I don't think we'd have to treat it as a top level project any more than we treat citgm as a top level project (for instance) or the docker-node stuff. It can simply be a tool that lives within the nodejs org.

the key question for me would be: are there other competing similar packages that would be disadvantaged by moving this into the org?

I think I'm +1 on this.

Fishrock123 commented 8 years ago

I think we should ship a version manager. (But I'm pretty sure there are others who disagree.)

Or, at the very least, bundle one like we bundle npm.

I'm not sure that correlates to bringing an existing one into the foundation, and probably not at the current time.

williamkapke commented 8 years ago

ah. Rather than being a TLP, this would be something under the Core's domain... maybe...?

are there other competing similar packages that would be disadvantaged

I'll throw https://www.npmjs.com/package/n out there to be noted.

jasnell commented 8 years ago

If we bundled one I'd definitely want to bring it into the foundation first. I've always found it mildly irritating that we don't already ship a version manager.

MylesBorins commented 8 years ago

The one advantage to nvm compared to most other node version managers is that it is implemented in POSIX. This has the disadvantage of not necessarily supporting windows, but it does make it very portable, and can run without having to have node installed first

With the windows ubuntu subsystem perhaps there is a way to get it working, as long as nvm is the only running in that subsystem to install node on the main system (as bash on windows is still not able to support npm).

mikeal commented 8 years ago

Even if it weren't bundled it would benefit from being more integrated into core development.

Does anyone know market share between nvm, n, and others node version managers?

MylesBorins commented 8 years ago

Doing some scientific research... please retweet

https://twitter.com/thealphanerd/status/725909274753261568

jasnell commented 8 years ago

according to numbers on npm...

nvm:

n:

mikeal commented 8 years ago

nvm isn't really installed by npm tho, so they won't have great data there.

MylesBorins commented 8 years ago

nvm is not generally downloaded by npm... @ljharb do you have the clone numbers from github?

jasnell commented 8 years ago

Yes, I know ;-) ... I'm adding to the market research, not providing all of it ;-)

williamkapke commented 8 years ago

https://www.npmjs.com/package/nvm doesn't have a valid link to the github repo which might turn people away.

ljharb commented 8 years ago

The one on npm is broken and from a legacy package - so none of its numbers are useful.

However, nvm is used on travis-ci for every single nodejs project, and I'm going to go out on a limb and say that it has more than 197K test runs in a month.

thefourtheye commented 8 years ago

@ljharb As nvm is already a popular project, with active contributors and maintainers, what do you think it would get out of coming under the Node.js umbrella?

P.S: I am not against this movement, but I am just trying to understand the motivation better. Please ignore if it is not appropriate to discuss it here.

ljharb commented 8 years ago

I think that the node core team and I already cooperate sufficiently to ensure that things continue to go smoothly between node and nvm, and I think the project has enough notoriety on its own that it doesn't need any bump from being "blessed" in any form.

However, I'm particularly hoping to benefit from a proper governance process, and to ensure that the project survives if I move on (not that I plan to for the time being) and if the current repo owner isn't available or willing to resume maintaining it.

While nvm is currently not "at risk" in any capacity I'm aware of, it seems to me to be better to plan in advance, rather than waiting until such a problem appears (like Express, for example).

As for other projects, the other two node version managers I'm aware of that have significant usage are indeed n, followed by nave - all three of these projects are for different (albeit slightly overlapping) use cases, so I don't see them as competitors, more as alternatives. If nvm were to join the foundation, and one of the alternatives wished to as well, I wouldn't personally see that as a conflict.

orangemocha commented 8 years ago

I am a bit concerned about the 'blessing' effect. To be honest, I am a bit concerned about nvm's lack of Window support.

And FWIW, if we were to bless a version manager, I kind of like nodist's approach better as it uses shim and supports version switching per directory, per app, in addition to per machine.

ljharb commented 8 years ago

@orangemocha fwiw, nvm now works on the latest BashOnWindows (although node itself doesn't yet).

nvm supports version switching per directory as well, via .nvmrc and a manual nvm use, we've just made the choice not to obtrusively modify cd for users - but there's examples in our readme of how to easily do this.

orangemocha commented 8 years ago

My comment was a bit of raw brain dump, sorry. To better summarize my feedback:

1) If we are not trying to bless a version manager, let's think about what other motivations would be served by nvm joining the Foundation. 2) If we think that Node needs to ship with a version manager, and hence we want to bless one, let's make sure that it has the 'right' architecture for Node, or the roadmap to make it so.

Fishrock123 commented 8 years ago

cc @nodejs/ctc @nodejs/tsc

mscdex commented 8 years ago

Yes, platform compatibility is one issue that comes to mind (mostly Windows doesn't seem to be a first class citizen in the version managers I've seen).

nebrius commented 8 years ago

I'm +1 on @orangemocha's suggestions re: Windows. I'm wary of officially blessing a version manager if it doesn't support/has weak support for Windows.

That said, bringing a version manager into the foundation could be a good opportunity to get first class support for Windows into such a version manager.

evanlucas commented 8 years ago

Don't forget fish shell either...unfortunately it does not work with fish (I know there is a port of it, btw)

ljharb commented 8 years ago

I'm not opposed to adding Windows or non-POSIX support to nvm, especially as part of the node foundation - primarily it's been an issue of contributor time, as well as the ease of automated testing.

evanlucas commented 8 years ago

@ljharb that's good to know. That would be awesome (for me, at least) Thanks!

Fishrock123 commented 8 years ago

Don't forget fish shell either...unfortunately it does not work with fish (I know there is a port of it, btw)

Most of us who use fish just call into bash for these things? Idk

ljharb commented 8 years ago

A fish implementation for nvm would likely just be a wrapper around nvm itself that calls into it via bash, which is how most of the existing fish wrappers are implemented.

coreybutler commented 8 years ago

I want to toss https://github.com/coreybutler/nvm-windows into the mix. It's grown in popularity... reasonably quickly (~150 Github stars to 1300+ in the last few months). The README lays out the approach to version management, which is a bit different from some of the other version managers (including the original nvm). The key bullet points are:

  1. The core code base is 100% Go. No dependencies*. Designed for cross-platform uniformity.
  2. Uses PATH, pointing to a changeable symlink for version switching.

*There are some helper scripts specifically for Windows, most of which I believe Go has now natively addressed (i.e. these should be able to go away).

By not having any dependencies on node or npm, it remains flexible & independent of those code bases. The obvious downside is contributors need to know Go.

Roadmap

While NVM4W currently only supports Windows, I am actively developing Mac & *nix versions from the same code base. Go has been excellent for this. There will be a name change when I reach the next release point.

Frankly, the code is the easy part. The time suck has been preparing installers & proper builds across Appveyor, Travis, homebrew/chocoloatey/yum/etc. I've struck a deal with BitRock to simplify this to a degree, but the project would benefit from having a maintainer for each type of installer (specifically non-visual installers like chocolatey).

I also really like the auto-update capabilities Go can potentially provide for a CLI tool, so I've been exploring that.

Usage

NVM4W ships with "installer" and "no-setup" versions (for those who want to make their own silent installer). According to my Github stats this morning, v1.1.0 has 37,908 installer downloads & 9,954 "no-setup" downloads. This only includes usage since v4.x.x was released. However; I suspect these numbers to not accurately represent real usage. I believe they're low, because I'm constantly getting questions about enterprise deployments through Active Directory, mirroring downloads, or other on-premise distribution means. That can't really be tracked. It also amazes me how many devs are stuck on older versions of Windows... point being the recent inclusion of bash in Windows will take some time before reaching enterprise folks.

Foundation Blessing

If the foundation doesn't adopt/bless a specific tool, it should still set standards. For example, the original Joyent repo didn't have a clean way of obtaining version data, which lead to the ugliness that is nodedistro. That's an iron.io script that scrapes data from the releases page. Now that releases are published in a nice JSON feed, this is no longer a problem (switching is a top priority on my roadmap). The point is foundation-blessed standards and a means of understanding them will drive development.

One "sort of" outstanding issue is naming. NVM4W still uses "stable" and "unstable" since I haven't yet had time to incorporate LTS & Current. I don't think there's much that can be done about that now, but it's indicative of how changing a standard will impact workflow.

The blessing of the foundation will impact user trust. We all know decisions are often based on the "official" choice of the authoritative body, especially within enterprise.

As @ljharb mentioned, governance would benefit a number of projects (NVM4W included). I believe there are multiple "right" approaches to version management. Has anyone considered some sort of standards compliance "certification" process?

Bundling

While the concept of version management could live within the foundation, it should not be bundled with node itself. In other words, don't ship it with a download. It would need to be a separate tool that traverses multiple versions... bundling it with each version of node just doesn't make sense.

Project Impact & Opinions

By blessing a specific project, you'll be blessing a specific approach. In my personal experience, there are two major opinions driving usage of any of the version managers. It boils down to simplistic vs deep workflow integration. Individuals seem to like deeper integration while teams like less (or the ability to customize their own workflow, given a set of simplistic tools).

Examples From my Experience

When I made NVM4W, my goal was to mimic the experience of replacing node with a newer/older version (i.e. uninstall/reinstall) on demand. I didn't want the tool to influence how developers wrote or deployed apps. Since NVM4W was made well after nvm, and intentionally more minimalistic, there were lots of questions about cherry picking versions for a specific process or terminal, .nvmrc files, etc. Instead of focusing on which version of node was installed, feature requests started circulating around managing the entire dev environment, app development/dependencies, DevOps tasks, etc. I've also had equally divided feedback arguing version management should be system-wide vs per-process.

Managing npm modules across versions is a pain due to sheer size/quantity. One corporate user mentioned his workstations took nearly 30 minutes to boot up because he kept multiple versions of node installed, each with a very healthy collection of npm modules, all within a roaming profile being copied over a slow internal network. This has lead to several enthusiastic conversations from people who want a "shared global node_modules" to reduce the footprint problem, as well as several who are adamantly opposed to that.

Sorry for the novel. This is just a brain dump and some initial gut reactions. If there is anything I can offer in this process, I'd be happy to. I would love to see version management addressed in some sort of official capacity.

joshgav commented 8 years ago

At the end of yesterday's CTC meeting (notes) it was requested that this be added to tsc-agenda for the upcoming meeting.

I don't have rights in this repo, could someone from @nodejs/tsc please label accordingly so TSC members can form opinions for discussion? Thanks!

addaleax commented 8 years ago

@joshgav Done. :)

jasnell commented 8 years ago

Personally, I would like to see this happen. Usage of nvm in the ecosystem is significant and the utility is obvious. Yes there are cross-platform concerns but that's something that can be addressed incrementally. Pulling this in just makes sense and once we're further along with ensuring cross-platform support, I would love to see us actually bundle nvm with the node distribution itself.

What I would propose is this: let's create a version-manager team and repo. The existing nvm code can be one of the possible inputs to bootstrap that work but we can extend and offer to the other popular version-managers to also get involved. It's perfectly ok to start with multiple codebases as a starting point. If they decline, then that's fine, we can move forward with nvm as the base... if they jump in, then we do some initial work to reconcile.

There doesn't need to be a ton of process around this and there shouldn't be. It doesn't need to be its own top level project, there doesn't even need to be a formal WG chartered. We can simply choose to create a repo and start collaborating on something.

ljharb commented 8 years ago

@jasnell if that is coupled with bringing nvm into the foundation in the meantime, I'm all for it! However I think that the parallel work of preparing a bundle-able version manager shouldn't stand in the way of the concrete benefits for both nvm and the foundation for nvm to be brought into the fold in the meantime.

bnoordhuis commented 8 years ago

As for other projects, the other two node version managers I'm aware of that have significant usage are indeed n, followed by nave - all three of these projects are for different (albeit slightly overlapping) use cases, so I don't see them as competitors, more as alternatives.

Can you elaborate on what you feel the differences between them are? In my (possibly naive) view the primary use case for all three is switching between node versions.

jasnell commented 8 years ago

Oh yeah, I agree. As I said, we shouldn't need to have much process around this at all and it should be possible to bootstrap something quickly. Obviously, however, we need to get the opinion of the rest of the @nodejs/tsc so hopefully others will weigh in. This will be on next weeks meeting agenda so hopefully we'll have decision at least by then!

ljharb commented 8 years ago

@bnoordhuis the bot factoid on freenode says "nvm ( http://nvm.sh / #nvm ) lets you change node versions per-user/per-shell. nave ( https://www.npmjs.com/package/nave ) is for per-user/per-SUBshell. n ( https://www.npmjs.com/package/n ) is for a single, global, system-wide node."

jasnell commented 8 years ago

@ljharb .. one clarification point that I may have just missed previously: do all of the current contributors/maintainers of nvm agree to moving the repo into the nodejs github org? We would not be able to move it in without that agreement. Also, are there are additional requirements such as Travis CI support? (ok, that's two clarification points)

ljharb commented 8 years ago

@jasnell yes, I've discussed it with the repo owner. travis-ci support is great but I'm already running on the free tier, so the paid tier would just make my tests much faster. It would be ideal to be able to use the node build infrastructure to run nvm's tests on all supported platforms, but imo that's not a blocker, just a potential future resource i could utilize.

jasnell commented 8 years ago

Ok. There's an ongoing discussion about whether we want to open the orgs repos up to Travis in general because apparently doing so requires granting Travis access to the private restricted repos also (which we generally don't want to do). So that may be a bit complicated. We should queue up a CTC/TSC conversation for that separately tho.

joshgav commented 8 years ago

nvm could also potentially handle VM choice, e.g. V8, Chakra, Duktape, etc.

It could also be made aware of the engines tag in package.json and automatically install that version for use with that directory/app, e.g. to start, run nvm install && npm install.

Under the Foundation it'll be easier to account for changes and dependencies in core within nvm, or vice versa.

ljharb commented 8 years ago

For solely exploratory purposes, I've created https://github.com/creationix/nvm/pull/1203 to propose adding the required documents to nvm itself. I've also added a road map.

rvagg commented 8 years ago

ftr I flagged this to the board at the last meeting, simply as a matter of courtesy so it wouldn't come as a surprise if we moved forward with this.

rvagg commented 7 years ago

I've been thinking about how to move forward with this and I have a suggestion that might stop us from continually kicking the too-hard can down the road every 2 weeks. Thanks to @joshgav for giving me a private nudge on this.

I propose that we set up an _investigatory group_ to consider the whole area of version management. "Runtime Management" is a term @joshgav is using, that may be appropriately broad enough for the subject area in question. I'm thinking that @ljharb and @joshgav would be two appropriate facilitators to get the ball rolling, perhaps others would like to put up their hand to get into the thick of this too. It'll most likely be made up of people favourable to the idea of bringing on nvm or doing something similar, and that's OK, because part of its role would be to make the case to the TSC (and by extension the CTC as it would be appropriate that we get broad buy-in for any strategy going forward).

Here's my suggestion for tasking and scope of this investigatory group:

The final deliverable for this investigatory group will simply be a case for action here, or a decision that there is no case to be made. If a case is presented, it will need to be complete enough and persuasive enough to get the TSC on board so the group should consult widely to capture any concerns.

jasnell commented 7 years ago

I'm not convinced that we really need to investigate this much further. I think this is something we should just do. Tools such as nvm make it easier to install and update across Node.js versions -- we already build an installer, a version manager should be included as part of that effort.

I don't believe we really need a formal working group for this. A github team and a repo with a governance policy that's identical to core's would be just fine. This is how we've been running projects like citgm and it's been working out just fine.

The key concrete outcome would be a version manager that works well on all of the platforms we support -- so obviously there's a chunk of work that would need to be done with nvm to make that happen.

Let's just move forward with this and things will evolve. If it does not evolve fast enough or if problems emerge, then we can deal with those if/when they do become an issue.

Fishrock123 commented 7 years ago

@rvagg I don't think adding this new level of process overhead will help here.

If a case as you outlined must be made, I think those of supporting it can already do so. I don't remember that ever being the case though, or we probably could have already done that.

Consider what typically happens when we suggest upgrading node version or having new people install Node.js. More often than not, we just tell them to use nvm. that is telling.

Unlike @jasnell though I do think it probably should have a WG. It's separate enough from everything else that I think it would be a good idea. This would probably also help with preserving some of nvm's existing ideals and values.

Where does https://github.com/nodejs/installer fit in to this picture?

There's been a significant amount of talk in there about just making it use nvm under the hood.

ljharb commented 7 years ago

My hope is that nvm (and other version managers who desire to) can enter the foundation in parallel with a version management investigation/working group, as opposed to being dependent on the actions or progress of one.

tjwebb commented 7 years ago

I'm one of the maintainers of n. (thanks @williamkapke for inviting us over here)

I think this is something we should just do.

There are two distinct approaches that we could take, following the lead of other runtimes:

  1. Manage version via OS package Some languages (e.g. python, java) provide their runtimes by OS package, and the version is managed there. Compiled languages offer version management via compiler. Has there been any interest in making the node builds provided through various OS package managers (e.g. apt, yum) more "official"? Is this an approach this TSC is interested in?
  2. Manage version via utility (nvm, n) This, I guess, is the "Ruby" approach. A separate utility is installed to govern the version of the runtime. Would this approach constitute a change in packaging, such as when I go to nodejs.org and download "node", will I be getting the version manager, which will then install node? Or will the packaging remain as it is

Has there been any discussion on this? Or are we simply presupposing the second approach?

MylesBorins commented 7 years ago

@tjwebb afaik releasing through official package maintainers is often done at the whim of the package maintainers themselves. Debian for example has specific versions that they ship in apt due to a variety of reasons (stability, openssl, dynamic linking).

I for one would very much like to see us improve relationships with official package maintainers for distros and get more modern versions of the platform supported, but I do see that as a separate problem from having a versions managing utility... as it is often not intuitive or obvious how to install previous versions using a tool like apt.

coreybutler commented 7 years ago

In response to:

The key concrete outcome would be a version manager that works well on all of the platforms we support -- so obviously there's a chunk of work that would need to be done with nvm to make that happen.

I have been working on a new project based off my original Golang code from NVM for Windows that now works on macOS (@evanlucas - I've been meaning to get this code over to you). It should work on Linux too, but I haven't had enough time to build tests yet (I've been swamped with a new Electron app).

Fishrock123 commented 7 years ago

Some languages (e.g. python, java) provide their runtimes by OS package, and the version is managed there. Compiled languages offer version management via compiler. Has there been any interest in making the node builds provided through various OS package managers (e.g. apt, yum) more "official"? Is this an approach this TSC is interested in?

Not particularly. Many Linux distros for example have very different ideas of stability and as such offer only vary outdated versions.