search

nodejs / user-feedback

Node.js User Feedback Initiative
https://github.com/nodejs/user-feedback
56 stars 18 forks source link

Upgrading Problems - Can we help? #116

Closed mcollina closed 5 years ago

mcollina commented 6 years ago

As Node.js 10.x becomes LTS next week,now is a good time to consider upgrading.

We understand that there are stometimes issues when migrating to a newer Node.js major version. One of the things we have heard is that this can be due to a module in the dependency chain that is not being updated.

We would like to gather some feedback and better understand how we can improve the migration experience.

Sme questions we would like to float to the community:

styfle commented 6 years ago

One way to improve the upgrade experience is to provide a tool that could statically analyze (most) of a package's source code and it's dependencies to detect if it will run on a specific version of Node. This could give higher confidence when upgrading because you will get errors prior to running in production.

Today, I use TypeScript to type check my code as well as interfaces for dependencies that my code interacts with, but this does not help me (much) when upgrading node because any of my direct/transitive dependencies might be using a deprecated or removed feature from Node.

mcollina commented 6 years ago

(I've updated the text of the issue to be more clear!)

joesepi commented 6 years ago

I'm happy to help shepherd this forward, perhaps in tandem with #77. What are next steps?

One meta question around these types of surveys -- should we establish a template and guide for building a user feedback survey initiative? Is the process for establishing a template/guide started/stalled in #53? Perhaps I should start there. :)

mcollina commented 6 years ago

That’d be fantastic Joe! Do you see this as more of a timed survey because of Node 10 LTS? I think it could be a very good marketing message for the project as well to recommend updating.

art2cool commented 6 years ago

Thanks, it's awesome!

dylang commented 6 years ago

Is there a simple way to explain the benefits of upgrading to leadership?

For example:

Could something like this be useful?

Pick your current version to see the improvement

version Security Performance Notes
0.0.10 baseline baseline <- Your version, released 7 years ago!
4.x.y 4 fixes 5% faster released 6 years ago, support ended 5 years ago.
6.x.y 6 fixes 10% faster released 4 years ago, 80% of the top 1000 downloaded node modules published this year do not support this version.
8.x.y 8 fixes 25% faster released 2 years ago, 50% slower than the current release.
10.x.y 20 fixes 100% faster released this week. LTS for the next 2 years.

Note: The above is a hypothetical chart to illustrate an concept.

Even that chart might be too complex. My goal is something a engineer can fit in a Slack message to convince leadership why they should want their teams to upgrade.

ljharb commented 6 years ago

(providing some feedback to help shape the survey)

The reason companies don’t upgrade is often not a lack of desire, but a lack of funding - money, time, people. Upgrading node (or rails, or ruby, or any “platform” technology) is risky, even when it comes with benefits.

One thing that sharply decreases the risk of upgrading is when all dependencies and APIs that we use can be updated to a version that supports both the version of node we are on, and the one we want to upgrade to. For example, when airbnb updated from node 0.10 to node 4, anything that forced a dependency update at the same time as the upgrade made the process harder (so anything that didn’t simultaneously support node 4 and node 0.10, so that we could ship those updates on 0.10, derisking the node 4 upgrade).

The point I’m driving at is that if the node project wants to make it easier for people to upgrade node, it should encourage the ecosystem to retain backwards compatibility with older node versions for as long as it can, and it should backport features as far back as it can, to make the actual platform upgrade as easy (as small, as de-risked) as possible.

joesepi commented 6 years ago

@mcollina I think a timed survey would be good to address Node 10 LTS and get us a snapshot but then I think it could be converted to an ongoing long term feedback collection process.

And if we take that approach, we can use our established process to get a survey out without too much delay/difficulty.

mhdawson commented 6 years ago

I'm +1 on a survey as well.

mcollina commented 6 years ago

+1

Il giorno ven 2 nov 2018 alle 15:57 Michael Dawson notifications@github.com ha scritto:

I'm +1 on a survey as well.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nodejs/user-feedback/issues/96#issuecomment-435407567, or mute the thread https://github.com/notifications/unsubscribe-auth/AADL4xF4lSr4UZw_nw7uvxE8qh_IYQlOks5urF1wgaJpZM4X8FHo .

mramato commented 6 years ago

While Node has great change logs, it can be frustrating to figure out what's new or significant between major LTS releases. Since we exclusively use LTS, we didn't pay close attention to 10.x changes while it was non-LTS. It's tedious to go back and figure out what's different, especially when some changes/features have already been backported.

It would be great to have an official high-level summary of new and notable changes between 8.x and 10.x. Something I can email my team and say, "10.13.0 is the new LTS, here are the major highlights/breaking changes you should be aware of from 8.12.0" Sure, there are third-party news sites that try and do this, but I always find them outdated or otherwise lacking.

Even the official release blog provides no real information for people coming from 8.x LTS other than 10.x is the new LTS: https://nodejs.org/en/blog/release/v10.13.0/

targos commented 6 years ago

I tried to start an initiative a while ago: https://github.com/nodejs/node/issues/18836 The problem is that I couldn't find the time to work on it myself and it stalled.

alexfernandez commented 6 years ago

At my current company (Devo) we do massive event aggregation, ingesting many terabytes of data per day. As heavy Node.js users, our biggest concern is always performance. For instance there's this issue with tls.createSecurePair() which we tracked down and merged a benchmark for; it makes it quite slow to first open a connection and then upgrade it to TLS. The proposed alternative new TLSSocket() not only requires major design changes to how we process data but also has horrible performance. Granted that this is a corner case on a deprecated function call, but for us it means almost halving bandwidth on TLS connections with realistic packet sizes.

This is just one issue which we discovered almost by accident, and it means in practice that we are stuck with 8.x for the foreseeable future. On major upgrades there might be other similar issues lurking, which may require extensive testing to uncover and which will probably only arise in production. I also want to stress that the usual behavior of newer versions is to get faster with time, which is quite welcome, and it is only in exceptional cases like this one that we see performance regressions that go under the radar.

mcollina commented 6 years ago

@alexfernandez thanks for reporting, would you mind sharing a comparative benchmark?

alexfernandez commented 6 years ago

Sure! Running this benchmark: results are for MB transferred on a clear connection secured using tls.createSecurePair(), with a realistic packet size of 1024 bytes.

Node.js version MB transferred
8.12.0 2050
9.11.2 1796
10.13.0 1069
11.1.0 1180

As you can see they drop to around half with 10.x and later. More complete results were reported on the issue. With 10.x and later the results are similar to those obtained with new TLSSocket(), which were historically quite worse than with tls.createSecurePair(). I suppose this is good news, since it means that with some buffering new TLSSocket() should possibly be made quite faster for everyone! :smile:

mcollina commented 6 years ago

Can you open an issue on nodejs/node and tag me? This is something we should look into.

Il giorno sab 3 nov 2018 alle 10:45 Alex Fernández notifications@github.com ha scritto:

Sure! Running this benchmark https://github.com/nodejs/node/blob/v10.x-staging/benchmark/tls/secure-pair.js: results are for MB transferred on a clear connection secured using tls.createSecurePair(), with a realistic packet size of 1024 bytes. Node.js version MB transferred 8.12.0 2050 9.11.2 1796 10.13.0 1069 11.1.0 1180

As you can see they drop to around half with 10.x and later. More complete results were reported on the issue https://github.com/nodejs/node/issues/20263#issuecomment-435458327. With 10.x and later the results are similar to those obtained with new TLSSocket(), which were historically quite worse than with tls.createSecurePair(). I suppose this is good news, since it means that with some buffering new TLSSocket() should possibly be made quite faster for everyone! :)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nodejs/user-feedback/issues/96#issuecomment-435574630, or mute the thread https://github.com/notifications/unsubscribe-auth/AADL4-_YW_4ASRjT1fO-aN4ED2hca0Z1ks5urWWegaJpZM4X8FHo .

dougwilson commented 6 years ago

It is this six month old issue: https://github.com/nodejs/node/issues/20263

RickBullotta commented 5 years ago

Holy WTF Batman...I (mistakenly) chose the option to install the windows build tools for Node in 10.X. This installs some SCARY sh!t that opens up big security issues, and messes with Windows Update and UAC. Remove that option. Seriously. It's bad voodoo. What was the thinking behind it, and why not at least let users/devs know the risks they're taking by installation Boxstarter and Choco?

RickBullotta commented 5 years ago

It appears that certain native modules built for 8.X won't work after an upgrade to 10.X. I've noticed it specifically with the piface module (on Ubuntu).

mhdawson commented 5 years ago

Closing in favor of https://github.com/nodejs/package-maintenance/issues/136 as we believe the package-maintenance repo is the right place for this discussion to be handled now.