search

nodejs / build

Better build and test infra for Node.
506 stars 166 forks source link

CI target architectures for libuv #14

Closed rvagg closed 9 years ago

rvagg commented 9 years ago

Continued from https://github.com/libuv/libuv/issues/12 also see #1 for additional context.

Here's a strawman proposal for architectures libuv should be tested against. They are split in to 3 classes mainly based on how difficult they will be to set up and include in the build set and how important they are to have solidly tested builds against.

Class A

Class B

Class C

The open questions, for me at least, are:

Are there any concerns here held by the libuv team? / @indutny @saghul @bnoordhuis @piscisaureus (I'm guessing here at who constitutes libuv-core btw).

tkelman commented 9 years ago

I'll second what @saghul said in https://github.com/node-forward/build/issues/1#issuecomment-51710565 about MinGW. The requirements of Julia, a downstream consumer of libuv (currently on a fork but would like to fix that), cannot be met by MSVC (even 2013; 2015 should help some but not completely). Legacy 32-bit-only mingw.org is far less important to us than the MinGW-w64 toolchain which has both i686 and x86_64 variants, but issues do still occasionally get filed at libuv when something breaks with the legacy toolchain.

rvagg commented 9 years ago

Moved "Mac OS X 10.10 (Yosemite) + XCode 6" up to class A, thanks @someoneweird for picking that up. Does it still make sense to include 10.8 in the list here?

A couple of additional items worth noting:

This list of full architectures will be reserved for use by the libuv-core team rather than for everyone submitting pull requests. The list of people who's pull requests will trigger a full-set build will be controlled by that core team and will be kept relatively small to a trusted set due to security concerns of running arbitrary code from pull requests on most of these platforms. At the moment we have a containierized build set working @ http://jenkins.node-forward.nodesource.com/ that will be used to test incoming pull requests from untrusted sources. Core members will then be able to request a full build of any pull request if it makes sense to do so (the exact mechanism of this request is yet to be decided). The containerized set simply includes the last 3 LTS releases of Ubuntu but we could expand that if necessary and possibly even include other architectures where this is easy (SmartOS/Solaris, FreeBSD). So that set is a matter for another discussion but will also come down to difficulty involved in coordinating this to make it workable.

The other item worth noting here is that we have hardware contributions from multiple companies to make this happen: DigitalOcean (Linux), Rackspace (Linux + Windows), IBM (via Softlayer) (Linux + Windows + POWER), and Voxer have also stepped up with a couple of Mac Minis in their datacenter. We also have people power provided by NodeSource (myself and @ryanstevens) and a large number of very interested parties who want to help out because they care deeply about the success of Node.js and libuv and believe this is the best place they can contribute. Over the coming months we'll be trying to make this project much more organised and give clear avenues for additional contributions by companies and community members.

jbergstroem commented 9 years ago

Does these classes also reflect usage? Will all classes have buildbots? Are fails accepted in Class C? I think a better definition of these classes would improve understanding of what the greater goal is.

Since hardware/vm is pretty cheap, I think the Linux prevalence should be diversified with BSD's (Open, Net, Dragonfly comes to mind). Illumos/SmartOS also feels like a pretty relevant inclusion. Since the build isn't limited by V8 (which would be the showstopper for most of nodejs), libuv would only improve by testing on these os'es or even additional architectures.

Also; above targets seems to be "point in time" distributions (10 lts vs 12 lts) -- a possibly more relevant mindset would be testing functionality such as disabling ipv6 or similar (ref https://github.com/libuv/libuv/issues/10). Point being that the major difference between all those Linux'es being toolchains or kernel versions.

tkelman commented 9 years ago

Oh, and I should note that I very frequently cross-compile libuv (and lots of other things) from Linux to Windows via MinGW-w64 and autotools. As far as libuv's build system goes I'd like to see that continue to work, though other communities might not do that kind of thing as often. Cross-compiling is highly relevant for ARM as well, but I haven't seen any mentions of it yet.

bnoordhuis commented 9 years ago

Does it still make sense to include 10.8 in the list here?

Data point: I still use 10.8. I may very well be an outlier, though.

I think the Linux prevalence should be diversified with BSD's (Open, Net, Dragonfly comes to mind). Illumos/SmartOS also feels like a pretty relevant inclusion.

I don't really disagree but the number of BSD users that are not FreeBSD users is practically a rounding error, and I say that as the one who did the DragonFly BSD port and spent the most time maintaining the OpenBSD port. It's okay to add such platforms to the CI matrix but test failures (and arguably build failures too) should not block tier 1 platforms.

Likewise with Illumos/SmartOS. Realistically, the only users are Joyent customers and as they pay Joyent good money, I think it's fair that responsibility for maintaining the Illumos port falls to Joyent (or their customers, if they are willing to step up.)

rmg commented 9 years ago

My opinion on platform support, without any real background in the libuv project

I think it would make sense to treat all of the listed platforms as class A and only move them to class B if/when they start failing in a way that puts a drag on productivity. There is no class C in this scheme because it's a wish list and support for those platforms belongs in an issue tracker instead.

jbergstroem commented 9 years ago

@bnoordhuis Without going into the guesttimation game for user/os adaption, my suggestions didn't assume a successful build or test suite - rather the fact that if hardware is available the effort to increase visibility is pretty low. Portability is generally a good thing and exposing shortcomings doesn't mean that it has to get highest priority. This is also why my comment started with "what does different classes mean".

rvagg commented 9 years ago

Regarding VMs being "cheap" leading to wanting greater diversity, the equation isn't as simple as throwing as many options as we can think of against this project.

What the "obvious" choices are is actually a matter for the libuv core team and I'd like them to come up with that list after taking into account the discussion here. That includes architectures, build modes etc.

Regarding cross-compiling: we can test that it's possible to do the compiling but testing the builds produced from cross-compiling is a non-trivial exercise and would involve coordination of multiple VMs. I'm personally skeptical of the value of including cross-compiling in the CI set but I'm happy to be told otherwise by the libuv team.

Regarding the classes— I grabbed them off the README which includes details about what happens when failures occur on the lower classes. However, this is just a starting point and that list is mainly focused on Node. I expect to land somewhere different with libuv and would love to hear more discussion here on how to define the classes, how to handle failure cases in the classes, etc.

jbergstroem commented 9 years ago

@rvagg After reading the README it makes more sense, thanks. If there's anything I can do to help wrt build environment or maintenance, ping me at IRC (freenode, same username).

ghostbar commented 9 years ago

FYI, Debian wheezy is soon to be oldstable. Debian is currently on "freeze" to move current testing to stable.

bnoordhuis commented 9 years ago

Closing, I think we've implemented most of what is discussed here. Testing with older compilers is maybe the exception but that hasn't been a problem so far. I'll file a separate issue for that when it becomes a problem.