nodejs / build

Better build and test infra for Node.
506 stars 166 forks source link

Dropping 32-bit builds #885

Closed gibfahn closed 5 years ago

gibfahn commented 7 years ago

This came out of discussion of https://github.com/nodejs/build/pull/809 in the last WG meeting (https://github.com/nodejs/build/issues/875, minutes at https://github.com/nodejs/build/pull/881). Discussion paraphrased below.

Basically updating CentOS 6 to gcc 4.9.3 requires devtoolset-6, which means that we can no longer do 32-bit xLinux builds.

It's difficult to know how many people are currently using x86_64 32-bit Node. The metrics show it looking pretty stagnant, but it's difficult to know how many of the downloads are from CI.

@rvagg suggested that the best way to find out would be to drop 32-bit builds in an odd-numbered release and see how many people complain, that way we have the option of adding them back in a minor version. The question was whether 9.x is too soon to do this.

Rod also said that he thought CentOS6 and Ubuntu 14 no longer had a 32-bit version, meaning that anyone relying on a 32-bit OS had already been left behind by Node 8.x. If we could confirm that this is true for most Linux and Windows versions then that would make the decision much easier.

Todo (if we decide to drop 32-bit)

ljharb commented 7 years ago

I think it’s too late to do that; there are 32-bit nvm users that will expect those binaries to be there.

I understand that documentation may allow you to justify dropping the binaries, but it’d be preferable to make a clean break at a semver major boundary.

refack commented 7 years ago

Ohh yeah, I forgot I promised we will not DDoS @ljharb

rvagg commented 6 years ago

Ah, OK .. so I wasn't properly tuned in to this at the time, kind of distracted late last year, even though I did contribute to the discussion, so I'd forgotten about it entirely.

As per #1223, we have devtoolset-6 builds working for x86 in prep for 10.0.0, https://nodejs.org/download/nightly/v10.0.0-nightly2018041142d8976dff/ is the first, and the next 10.0.0 test build will use it too.

So we can keep on shipping 32-bit Linux binaries for the duration of 10 if we want, but we could also pull it out. I'd like to stop shipping SmartOS 32-bit builds at the same time if we do. I'm fine either way but we need to get this decided this week @nodejs/build.

refack commented 6 years ago

IMHO node10 is a great opportunity to drop 1st tier 32 bit support for as many platforms as possible. I assume nvm users (via @ljharb) can live with that.

If the community pushes back we could consider restarting 32bit binary releases in a "contrib" context (seperate storage namespace, maybe even a different domain) to emphasise the "experimental" SLA. Again my assumption is that most 32bit users are either client OS users (frontend devs using npm) or embedded platform users (who possibly compile their own binaries)

ljharb commented 6 years ago

As long as it’s on a clean semver boundary, i as the nvm maintainer can live with that :-) 32bit users, i suspect, will be quite frustrated if they’re forced - on their slower, older machines - to start compiling node to use newer versions.

rvagg commented 6 years ago

https://arstechnica.com/gadgets/2018/04/apple-has-begun-notifying-macos-users-of-coming-32-bit-app-incompatibility/

Another nail in the x86 coffin. Worth noting that we (well I kind of did it unilaterally ...) dropped 32-bit builds from the "universal" macOS binary with io.js v1 and that's continued through Node v4.

I'm still of two minds about this but it's not something we can't reverse if we're presented with good reasons to do so afterward. I'm also thinking now that it partly lifts one of my objections to shipping musl binaries—that there are too many potential permutations that we would be asked for after we shipped just one. We could more easily make a stance that we're just shipping the one musl binary, for x64. It doesn't remove the potential for an explosion of arm+musl requests though ..

I'll queue up the change to ship 32-bit and get all of these build changes for 10 listed in a form that the TSC + build can easily grok and approve.

rvagg commented 6 years ago

Looks like our users have been voting with their downloads already

screenshot 2018-04-12 21 31 59

And cause it's interesting and because openssl is causing armv6 problems for us so maybe we need to consider dropping it too, here's the other archs alone (~100x smaller scale)

screenshot 2018-04-12 21 32 26
refack commented 6 years ago

And cause it's interesting and because openssl is causing armv6 problems for us so maybe we need to consider dropping it too, here's the other archs alone (~100x smaller scale)

I also had ARM6 in mind, but from a quick research it seems that there are current selling products (e.g. RasberryPi Zero family) that are based on ARM6 CPUs.

ljharb commented 6 years ago

Stats and percentages are nice, but that green line still represents human beings who may not be able to upgrade their machine.

rvagg commented 6 years ago

Yeah, that kind of fine grained empathy is nice and all but it doesn't really help in making rational decisions that account for the whole of a large and complex ecosystem.

There are currently ~1.2 times more people downloading Node 0.10 today than there are people downloading x86 builds. Should we stick 0.10 back onto LTS?

I'm still on the fence with dropping 32-bit but it seems to me that there's broad consensus from Build & TSC that it's the right time. If you'd like to make a case for keeping them around then you should start doing that work now to turn this around.

refack commented 6 years ago

IMHO the consensus is around being rational and empathetic. Dropping 32bit official support, but being responsive to community feedback.

gibfahn commented 6 years ago

Stats and percentages are nice, but that green line still represents human beings who may not be able to upgrade their machine.

If they can't upgrade their machine, do they need to upgrade to Node 10? Can't they just use Node 8 until it goes EoL? At that point their OS will probably also be EoL, so not getting the latest and greatest versions of Node shouldn't be the end of the world.

We can't support everything, so IMO the most important is to be clear and to evaluate how to keep the largest number of users happy.

If we were talking about removing old builds then I'd agree that even breaking a small number of users is still not ideal, but we're not talking about doing that.

ljharb commented 6 years ago

If we drop them, we drop them, but i think it’s important to weigh the actual cost/burden and not just arbitrarily make things harder for users. I’m not on the build team and it’s not my personal use case, so I’m not going to die on this hill :-)

refack commented 6 years ago

If we drop them, we drop them, but i think it’s important to weigh the actual cost/burden and not just arbitrarily make things harder for users.

Agreed, and IMHO that is being taken into consideration. This has been neither an arbitrary, abrupt, nor written-in-stone decision.

rvagg commented 6 years ago

Stats separating out major versions for x86 downloads would be interesting. If 8.x x86 downloads are downward in comparison to prior versions then that'd make it an easier call. Perhaps all these x86 downloads are for ancient versions of Node? I have a bit too much on my plate with this minimum compiler stuff right now but if I have time I'll see if I can pull that out.

rvagg commented 6 years ago

FYI latest nightly is the first to drop linux & smartos x86 https://nodejs.org/download/nightly/v10.0.0-nightly201804120aab8ff602/

mcollina commented 6 years ago

We should ask ourselves the question: do we want to support 32-bit for one more year on top of the next two?

SGTM, I think it's time.

Another option is to keep support for Node 10, but drop it for Node 11+. This would still mean one more year of support... but we could use this to give our users a bit more notice than 2 weeks.

rvagg commented 6 years ago
screenshot 2018-04-17 15 57 21

This clears things up a little, not quite as much as I hoped however. It's only 2017 & 2018, maybe it would be more interesting if I went back further but I got impatient enough waiting for these to generate.

The Node 8 bumps are interesting but all its doing is picking up the Node 6 numbers, so we have a persistent user group that is using 32-bit and sticking to the current LTS. The Node 0.12/0.10 (and prior) group is fading to nothing, thankfully, so we're basically left with that ~7.5k group. This is 1.25% if the ~600k downloads for 64-bit each day we're currently getting. So the case could be made that dropping 32-bit support will impact negatively just over 1% of our user-base.

Unfortunately we don't know why they are still using 32-bit, perhaps it's not for any critical reason that prevents them from using 64-bit. It's all just guesses at this stage!

As of now we're still on track to drop 32-bit for Node 10. If someone thinks that the cost of supporting 32-bit builds for the next 3 years is worth it to look after the 1.25% then you should speak up now.

ChALkeR commented 6 years ago

@rvagg Which OS does that come from?

ChALkeR commented 6 years ago

Looks to me that >90% of the 32-bit users are Windows platform users.

~15% of Windows downloads are 32-bit downloads. ~0.3% of Linux downloads are 32-bit downloads.

ChALkeR commented 6 years ago

15% of Windows looks pretty high…

Also, take a look at this: https://github.com/search?q=filename%3Aappveyor.yml+x86+node&type=Code

If that downloads the versions from nodejs.org, it might explain things.

rvagg commented 6 years ago

WHOOPS! Thanks for catching that @ChALkeR, this is all 32-bit, I intended to only look at Linux. So here's the numbers when I restrict it to just that:

screenshot 2018-04-17 20 10 11

Much easier case for dropping 32-bit there. 0.075% of downloaders are grabbing Node 8 Linux 32-bit.

Interesting about AppVeyor, I wonder if the persistent ~1% (Windows) could be explained primarily by that.

ChALkeR commented 6 years ago

@rvagg Is the intent here to drop 32-bit Linux builds in 10.x (and keep 32-bit Windows builds)? That doesn't look very clear.

rvagg commented 6 years ago

@ChALkeR yes, only Linux 32-bit, but we've also dumped SmartOS 32-bit while we're doing this since it's so miniscule. Windows 32-bit is a different story. Although the numbers above suggest it may not be too painful to do if/when we want to.

mcollina commented 6 years ago

Definitely 👍 to drop Linux 32-bit.

I'm -1 to drop Windows 32-bit fo Node 10. Consider that everybody builds on both 32-bit and 64-bit on AppVeyor because of binary addons prebuilts.

AxelTerizaki commented 6 years ago

Many times, people use 32 bit because they don't have a choice in the matter and would rather use 64 bit.

Like, in some big companies, 32 bit windows are still installed due to legacy software they can't move away from just yet.

Another question to consider is, what is the price today of maintaining 32 bit binaries? Is there something slowing node development down because of those? Just a genuine, curious question.

gibfahn commented 6 years ago

Like, in some big companies, 32 bit windows are still installed due to legacy software they can't move away from just yet.

AIUI that's why we're not dropping 32-bit Windows builds

Many times, people use 32 bit because they don't have a choice in the matter and would rather use 64 bit.

One of the things we'd really like is to hear from people who are forced to use 32-bit Linux, and who can't just use 8.x for its lifetime.

Another question to consider is, what is the price today of maintaining 32 bit binaries? Is there something slowing node development down because of those? Just a genuine, curious question.

I think I'd phrase it the other way around. Node is an open-source project, largely maintained by people in their free time. The Build working group has limited resources, and people ask us to support new things all the time. Every platform has a maintenance cost, so we should try to maximise the percentage of our users we can support. If almost no-one is using Linux 32-bit, why do the work to keep supporting it?

To be clear, if three or four people jumped in with a clear reason to keep supporting 32-bit Linux, and who were willing to put in the hours to maintain it over the life of Node 10.x, then I don't think anyone would have an issue with adding support again.

neroux commented 6 years ago

This caught me now a bit unprepared to be honest. Was this decision made because it is actually unsustainable to still provide x86 builds or rather because of fashion? What would the technical difficulties be tol continue providing x86 binaries?

The comments here which seem to be in favour of stopping x86 appear to be in regard to desktop environments (where I'd agree that x86 is somewhat on the way out). Node's habitat, the server landscape, is different though and that hardware does not necessarily come with hardware expected from current desktop machines. For example, memory can easily be only a few hundred megabytes or even less.

With Node being, and marketing itself as, a lightweight and efficient alternative to heavier runtimes, it would be nice if version 10 would not force such environments to either stick to old versions or upgrade to 64 bit which might put additional strain on its limited hardware.

refack commented 6 years ago

Hello @neroux and thank you for the feedback. The tl;dr is that it is progressively becoming harder to rent and maintain x86 hardware for testing and building node, so we can't commit to supporting it for the next 3 years (till 10.x goes EOL). We have thought of several recommended workarounds.

  1. Build your own Binary - we work hard to make sure building node from source is as easy as possible.
  2. Organize a "contrib" binary repo - If you have the time/resources/can-find-others to set-up an x86 build & distribute infra, we will be happy to coordinate.
neroux commented 6 years ago

Hello Refael,

thanks for taking the time to respond. Did I understand right, this is a definite decision and there is no chance in hell :) that x86 returns?

May I just still ask two questions for clarification

  1. Your summary refers to a hardware issue, this however is not the "vibe" I got from quickly scanning the thread (admittedly I did not read every single of the 77 comments :blush:). It rather seems to be compiler related on CentOS and for some x86-fadeout on the desktop. Also the fact that Win32 (literally 32 :)) is still available would support that.
  2. What exactly did you mean by "x86 hardware"? Any Intel-compatible machine sold today basically still is an x86 system. The difference comes from the operating system.
lucasbuchala commented 6 years ago

Hello. Sorry if this is not the right place to ask. I didn't read the full thread. I'm on a x86 machine, and definitely missed the binary for the 10.0 release. I saw that issue nodejs/node#20327 was closed saying that those binaries are no longer provided, but the release blog post in the website still says "Linux 32-bit Binary: Coming soon" and the front page, with JavaScript enabled, detects my platform as "Linux (x86)" and points me to a non-existent binary. So, I just wanted to know if there will be a binary or if I should starting considering building from source. I guess I already tried build from source in the past, and if I remember correctly, it took very long on my ancient 32-bit machine, and that's why I always appreciated the convenience of an already built binary for the x86 platform.

bnoordhuis commented 6 years ago

@neroux See https://github.com/nodejs/build/issues/885#issuecomment-381934673, the reason for dropping 32 bits Linux is that it's an insignificant fraction of all downloads.

Maintaining release binaries isn't free. We decided our efforts are better spent on where our users are. Note that you can still build from source.

neroux commented 6 years ago

Hello Ben, thanks for the response, as you wrote I could chime in here on this subject I was admittedly hoping for more of a discussion than just a plain statement :) but I guess this simply is the state of node on x86 at this point.

Though, may I ask, have these numbers really been verified? x86 might not be the major player but less than 1‰ (permille!) still does seem somewhat of a surprise. And that for an older version, version 9 is supposedly even below that figure. Could it be that a large number of x86 actually does not come from direct downloads but from package managers?

Well, assuming that I am the only one so far who raised that issue, maybe I am really the only x86 user out there :). For me unfortunately it will mean I have to migrate away from Node.js.

bnoordhuis commented 6 years ago

Could it be that a large number of x86 actually does not come from direct downloads but from package managers?

It's possible, but there's no way for us to know. But if the bug tracker is a reasonable proxy for usage, then that <1% is probably accurate because bug reports for x86 Linux are very rare.

ChALkeR commented 6 years ago

0.075% of downloaders are grabbing Node 8 Linux 32-bit

I don't think that this is a meaningful number, btw. We should either divide the numbers for:

But not 32-bit Linux platform + Node.js 8.x by any platform + any version — I have no idea what that meaningfully measures.

That said, the total number for any of those would still be around 0.1%, but perhaps a bit over it. So (@neroux), it's not «less than 1‰ (permille!)», though close to it. More like 2‰ (permille).

neroux commented 6 years ago

But if the bug tracker is a reasonable proxy for usage, then that <1% is probably accurate because bug reports for x86 Linux are very rare.

You mean the tracker on Github? I randomly opened issues from the last weeks and found it a bit difficult to tell from that alone how strong x86 still is going. Most of the time people refer to desktop platforms (which naturally will be x64).

I am not trying to argue that x86 will be still in the high double digits, I do wonder though if it wasnt maybe a bit too quick to dimiss it as being around 1‰ when it seems the numbers used are not quite accurate (based on your and @ChALkeR's response).

As I initially wrote, I could easily imagine that quite a few people still run it on 32 bit system due to its lightweightness compared to other runtimes. And if we assume a hypothetical share of 4 to 8% I wonder if supporting such a binary would be really too much hassle.

Anyhow, from the responses so far I take it that decision is quite final and I guess the coming months will show if it is a problem or not, outside of raw downloads.

Thanks for the ride anyway :)

bnoordhuis commented 6 years ago

You mean the tracker on Github?

Yes. By my reckoning we get more bug reports for Android - which isn't even a supported platform - than x86 Linux.

refack commented 6 years ago

What exactly did you mean by "x86 hardware"?

@neroux our entire build infrastructure (testing & release) runs on VMs that we get from our sponsors (see main readme). In "x86 hardware" I was referring to the shrinking availability of VM images running 32bit linux, specifically centOS which our release infra runs on, and also the lack of official 32bit support for the build toolset.

P.S. the org in general is run by volunteers (and specialty work done by employees of corporate stakeholders), so we have limited resources which we try to direct based on "educated guesses". So spesificaly for special interests we codified the Experimental support level. And so we are open for volunteers who are willing to commit to do the work needed to maintain support for this (meta) platform.

rvagg commented 6 years ago

I have some raw(ish) data for you all to play with. Please grab this and make graphs or give us stats if you think there's a case to make for a change in current policy (nothing is truly set in stone, you just have to be able to convince enough people!). I presented numbers above that I thought were meaningful but I see that there's disagreement with my choices. My methodology was to look at current versions of Node on Linux (hence the choice of Node 8) to see how much demand there is. I don't think current download numbers for unsupported versions of Node tell us a whole lot other than there are people still using older Node and are not bothering to upgrade so are probably not in the cohort that are going to impacted by a change. But the question of what to compare the numbers to is tricky. So, come up with a methodology, give us some numbers and/or graphs and make a case for why your methodology is valid and your data shows us something interesting & important.

https://gist.github.com/rvagg/38a35fe388250d95f5af4e62e5ed036a

This is Linux only, for all of 2018 so far. There's 2 header lines since we're dealing with 2 dimensions: arch + version. If you're graphing then my top tip would be to do it on a rolling average over 7 rows—weekly downloads are very lumpy, averages over 7 days smooths it out.

If you have a suggestion for a better layout or a change in data then let me know but I can't promise I have the time or awk-patience to make it happen. This data is all publicly available @ https://nodejs.org/metrics/logs/ but I understand that it's awkward to download it for yourself to run these numbers. My awk is at the bottom of the gist.

xzyfer commented 6 years ago

An additional data set that may be of interest is downloads of 32bit pre-built binaries for native addons. These are probably downloaded more frequently that Node itself i.e. CI, npm install, etc...

You can get the raw data with GitHub's GraphQL API explorer using the following query. This only works for projects that use GitHub releases for assets (rather than s3).

query { 
  repository(owner: "<org>", name: "<repo>") {
    releases(last: 10) {
      nodes {
        name,
        releaseAssets(first: 100) {
          nodes {
            name,
            downloadCount
          }
        }
      }
    }
  }
}

An example of one such repo is sass/node-sass which has ~9M monthly installs.

neroux commented 6 years ago

Thanks a lot for the numbers Rod!

I'd say these provide a pretty accurate overview of what architecture is primarily downloaded. I'd particularly like to thank you for your comment and your availability regarding tweaking the layout.

I took the liberty to grandtotal them by architecture and judging from that, x64 is - as expected - the winner with about 52 million downloads in 2018 so far. By comparison, x86 comes in at a miniscule 181,000 in the same period. However all other Linux platforms clock in at similar numbers, ARMv7 is insignificantly above the share of x86 and the others are all well below. So going by that X86 would still appear to be as relevant as ARM.

But as I mentioned earlier, I wonder how reliable the number of downloads as such a metric is in the first place, as it might be strongly biased towards desktop/development environments (which are bound to be 64 bits these days). Server environments would likely not get the download package but install from the systems's package manager.

I guess my point is, x86 should not be simply abandoned because some possibly biased numbers might indicate it is (on desktops) not the strongest contender anymore. Hence my original question if there are actual (technical) hurdles in continuing to offer x86 binaries. My understanding so far is there arent and the decision is based on the assumed market share. Here however I believe the drawn conclusions might not be fully accurate and a withdrawal from x86 might be yet a bit too early.

There'd be a Trumpish yuuuuuge :+1: for an x86 comeback from my side.

chrislea commented 6 years ago

I feel like I should point out a few bits since I was (and still am) strongly in favor of dropping 32bit Intel builds. Disclosure for those who don't know: I work with @rvagg .

There is broadly a move in the industry for Linux distros dropping 32bit support, and not just for Desktops. They are doing it in different ways, and in varying degrees, but it's happening all over. Some points to note:

Every major distro is moving away from 32bit support. There are sometimes still ways to get those distros onto 32bit hardware today (network installers, roll your own setups, etc), but the direction this is going is very clear.

As @refack noted, the build working group is entirely volunteers, so there are limited resources with which to do things. I can personally attest to how awful it was to support old distros for Node 4.x and 6.x. I actually had to make customized clang packages just to get those versions to compile on older Debian distributions. And for the record, it couldn't be done for x86, only x64. That kind of problem, especially the availability of compilers that can build 10.x or newer, is an enormous hassle to work around for the build team. There may be roughly equivalent numbers of x86 and armhf downloads today, but those architectures are trending in different directions. That time and effort can, and IMO should, be used to support forward looking systems as completely as possible.

TLDR: Making the build group spend a ton of their time trying to support Node 10.x or newer on 32bit Intel, or ancient distributions, is a bad time allocation.

My $0.02 on the "right thing to do" for the 10.x / 32bit Intel combo has been that we should make sure Node builds and passes tests on modern 32bit distros, meaning ones where we don't have to jump through any hoops to get a new enough gcc or clang to do it, and then stop there. That way people who are commercially supporting 32bit systems (for example, NAS manufacturers) can build Node for their systems with little hassle. In this case, the only people for whom the only answer is "you're going to need a newer server" is people running 32bit hardware with an ancient Linux distro, and also need Node 10.x or newer. I feel like that's quite acceptable.

neroux commented 6 years ago

There may be roughly equivalent numbers of x86 and armhf downloads today, but those architectures are trending in different directions.

That may be true, but so far the line of argument focused on the number of downloads, and going by that particular argument the popularity is similar.

an enormous hassle to work around for the build team.

Fair enough, if there are serious technical reasons for that decision I guess it eventually will be reasonable. Hence my question.

There is broadly a move in the industry for Linux distros dropping 32bit support, and not just for Desktops.

Agreed, and while I still think the focus should really shift away from desktops to servers, there is certainly a 64bit tedency there as well. Though, the current Debian version still supports 32bit and, judging from current Alpha releases of the upcoming version, so will the next version too.

Sooner or later it will be x64-only, I just believe the move might have been a bit premature. As I said earlier, Node.js aims to be a lightweight runtime which can also be used on very limited setups and forcing a 64bit system onto a machine with only a couple of hundred megabytes of memory max will reduce the already scarce resources even more.

Yes, there will be still machines with 64 MB of memory even in five or ten years, but at that point - I presume - every distribution will have moved past 32bit and these truly will be legacy systems. Right now though, they arent quite just yet :).

Just my two cents.

reconbot commented 6 years ago

I maintain node serialport, most of my users do hardware and IoT work, everything from powering vending machines and ATMs to programming chips, controlling large equipment, reading data from pacemakers and powering "smart home" equipment.

I don't currently have a position on this mater but I do have data.

My github release download stats of my precompiled binaries of the last 20 releases are such.

{"tag":"4.0.7","createdAt":"2016-12-12T04:28:52Z","ia32":39722,"other":955372,"ia32%":4}
{"tag":"5.0.0-beta3","createdAt":"2017-01-12T06:09:22Z","ia32":102,"other":1769,"ia32%":5}
{"tag":"5.0.0-beta4","createdAt":"2017-06-17T16:43:58Z","ia32":8,"other":147,"ia32%":5}
{"tag":"5.0.0-beta5","createdAt":"2017-06-20T01:21:33Z","ia32":14,"other":179,"ia32%":7}
{"tag":"5.0.0-beta6","createdAt":"2017-07-05T13:34:11Z","ia32":18,"other":142,"ia32%":11}
{"tag":"5.0.0-beta7","createdAt":"2017-07-10T19:33:12Z","ia32":205,"other":13960,"ia32%":1}
{"tag":"5.0.0-beta8","createdAt":"2017-07-17T13:28:51Z","ia32":372,"other":31839,"ia32%":1}
{"tag":"5.0.0-beta9","createdAt":"2017-07-30T00:20:29Z","ia32":19,"other":77,"ia32%":20}
{"tag":"5.0.0","createdAt":"2017-07-30T04:34:41Z","ia32":5904,"other":110970,"ia32%":5}
{"tag":"v5.1.0-beta5","createdAt":"2017-08-04T02:55:41Z","ia32":55,"other":188,"ia32%":23}
{"tag":"v6.0.0-beta1","createdAt":"2017-08-07T22:38:00Z","ia32":86,"other":576,"ia32%":13}
{"tag":"v6.0.0-beta2","createdAt":"2017-09-07T02:15:27Z","ia32":105,"other":981,"ia32%":10}
{"tag":"v6.0.0-beta3","createdAt":"2017-10-07T13:38:15Z","ia32":95,"other":178,"ia32%":35}
{"tag":"v6.0.0","createdAt":"2017-10-09T22:33:27Z","ia32":340,"other":3797,"ia32%":8}
{"tag":"v6.0.3","createdAt":"2017-10-22T16:03:05Z","ia32":164,"other":1371,"ia32%":11}
{"tag":"v6.0.4","createdAt":"2017-10-26T02:55:45Z","ia32":4130,"other":54327,"ia32%":7}
{"tag":"v6.0.5","createdAt":"2018-02-04T23:18:57Z","ia32":2014,"other":25371,"ia32%":7}
{"tag":"v6.1.0","createdAt":"2018-02-06T01:54:53Z","ia32":247,"other":1362,"ia32%":15}
{"tag":"v6.1.1","createdAt":"2018-02-28T02:18:47Z","ia32":2856,"other":35488,"ia32%":7}
{"tag":"v6.2.0","createdAt":"2018-04-18T12:18:46Z","ia32":787,"other":11287,"ia32%":7}

So I'm sill seeing 7% downloads of ia32 packages. This worried me.

However removing windows downloads

{"tag":"4.0.7","createdAt":"2016-12-12T04:28:52Z","ia32":15165,"other":790664,"ia32%":2}
{"tag":"5.0.0-beta3","createdAt":"2017-01-12T06:09:22Z","ia32":47,"other":1640,"ia32%":3}
{"tag":"5.0.0-beta4","createdAt":"2017-06-17T16:43:58Z","ia32":6,"other":147,"ia32%":4}
{"tag":"5.0.0-beta5","createdAt":"2017-06-20T01:21:33Z","ia32":9,"other":179,"ia32%":5}
{"tag":"5.0.0-beta6","createdAt":"2017-07-05T13:34:11Z","ia32":10,"other":105,"ia32%":9}
{"tag":"5.0.0-beta7","createdAt":"2017-07-10T19:33:12Z","ia32":56,"other":13225,"ia32%":0}
{"tag":"5.0.0-beta8","createdAt":"2017-07-17T13:28:51Z","ia32":82,"other":30298,"ia32%":0}
{"tag":"5.0.0-beta9","createdAt":"2017-07-30T00:20:29Z","ia32":7,"other":53,"ia32%":12}
{"tag":"5.0.0","createdAt":"2017-07-30T04:34:41Z","ia32":1609,"other":86649,"ia32%":2}
{"tag":"v5.1.0-beta5","createdAt":"2017-08-04T02:55:41Z","ia32":21,"other":111,"ia32%":16}
{"tag":"v6.0.0-beta1","createdAt":"2017-08-07T22:38:00Z","ia32":29,"other":289,"ia32%":9}
{"tag":"v6.0.0-beta2","createdAt":"2017-09-07T02:15:27Z","ia32":24,"other":816,"ia32%":3}
{"tag":"v6.0.0-beta3","createdAt":"2017-10-07T13:38:15Z","ia32":39,"other":99,"ia32%":28}
{"tag":"v6.0.0","createdAt":"2017-10-09T22:33:27Z","ia32":124,"other":2375,"ia32%":5}
{"tag":"v6.0.3","createdAt":"2017-10-22T16:03:05Z","ia32":52,"other":897,"ia32%":5}
{"tag":"v6.0.4","createdAt":"2017-10-26T02:55:45Z","ia32":1435,"other":37451,"ia32%":4}
{"tag":"v6.0.5","createdAt":"2018-02-04T23:18:57Z","ia32":691,"other":16615,"ia32%":4}
{"tag":"v6.1.0","createdAt":"2018-02-06T01:54:53Z","ia32":96,"other":840,"ia32%":10}
{"tag":"v6.1.1","createdAt":"2018-02-28T02:18:47Z","ia32":881,"other":21255,"ia32%":4}
{"tag":"v6.2.0","createdAt":"2018-04-18T12:18:46Z","ia32":182,"other":7135,"ia32%":2}

I'd say for linux it's closer to 4% or 5% of my downloads. Still a fair amount.

I also want to mention that my users are slow to upgrade, IoT and hardware devices have a long release cycle, and most of the devices will come preloaded with software and will not be represented in download counts. I want them to have the easiest time upgrading.

I'm researching current embedded hardware information to better inform my position, but the major producer of ia32 chips (intel) has dropped their IoT line completely.

@xzyfer here's a gist for my script downloading stats on a package https://gist.github.com/reconbot/caad3907a01b18fd23f24f253a1eb1aa

rvagg commented 6 years ago

Interesting data @reconbot, thanks for providing. One critical piece I'd like to see is how many of those users are on a recent version of Node. Do those versions of serialport only support modern Node? The section of the userbase I care most for this question are those that are actually missing out on Node 10 32-bit, not just general 32-bit users. If you're happy using 0.10 still then this decision likely doesn't impact you.

reconbot commented 6 years ago

I'll run those numbers, I've only supported LTS for all the versions in that data set.

reconbot commented 6 years ago

The same releases filtered for linux broken out by abi and ia32, it does drop off. Looks like 51 (an electron version?) is the most prolific usage of ia32

{
    "11": { "other": 6923, "ia32": 1406, "%": 17 },
    "14": { "other": 13616, "ia32": 182, "%": 1 },
    "46": { "other": 117631, "ia32": 6280, "%": 5 },
    "47": { "other": 20012, "ia32": 1300, "%": 6 },
    "48": { "other": 714433, "ia32": 8143, "%": 1 },
    "49": { "other": 152, "ia32": 24, "%": 14 },
    "50": { "other": 425, "ia32": 159, "%": 27 },
    "51": { "other": 65762, "ia32": 1074, "%": 2 },
    "53": { "other": 616, "ia32": 40, "%": 6 },
    "54": { "other": 6195, "ia32": 346, "%": 5 },
    "57": { "other": 57256, "ia32": 1507, "%": 3 },
    "59": { "other": 8334, "ia32": 130, "%": 2 }
}
rvagg commented 6 years ago

I don't want to derail the conversation too much but this is really fascinating data that tells us a lot about Node's IoT usage, something that I've struggled to find good sources for before. serialport binary downloads is a pretty great proxy for this information.

Here's the download breakdown by Node version of downloads:

screenshot 2018-05-07 09 32 46

The low Node 8 numbers are probably the biggest surprise here. I think it probably speaks to the slower upgrade cycle that IoT development necessitates, so we're probably just seeing that play out and are in the middle of a slow shift from 6 to 8 over the next 12 months (cross fingers). Node 7 numbers are strangely high too, not what I would expect to see.

maclover7 commented 6 years ago

Adding to build-agenda to discuss plan going forward and when/if machines will be removed from public CI

mhdawson commented 6 years ago

On the Node.js front, discussed in last build wg meeting, agreement was to leave 32 bit machines in place until 8.x goes EOL. Closing this issue, please re-open if you disagree.

Was brought up that we should check with libuv team as well. @refack agreed to reach out to them and will report back here.