First io.js Build WG and Docker sub-WG meeting

[rvagg]

The blame is on me for not organising this sooner, I've been shouldering too much of this effort on my own and would love to make space for others to help out.

This is an open meeting to anyone who feels they have something to contribute. My preference is to include those who have already stepped up with code or help with Build, Docker or other parts of io.js but I also recognise the lack of obvious ways to contribute so far may have held back additional contributors. So if you have some skills and interest in this space then you're welcome too.

Meeting via Google Hangouts, fill in your details here if you want to attend: http://doodle.com/r5cz2dq6rcpd9b5e

The Docker sub-WG is the most active group so I'd love for at least these people to be involved in this meeting:

• @ghostbar
• @wblankenship
• @pesho

Other people who have had some involvement with Build, mainly through contributing to discussions and showing an interest in the build repo, you may or may not have an interest in joining us:

• @bnoordhuis
• @rmg
• @jbergstroem
• @othiym23
• @andrewlow (I believe Andrew may not be on the V8/POWER project any more, but maybe still interested)
• @chrislea
• @Qard
• @ryanstevens

(just calling out names here to get the ball rolling, this is not an exhaustive list of people that can be involved by any means)

I'm also interested in having some libuv input since we're taking responsibility for libuv CI.

The proposed meeting dates are a couple of weeks away, mainly selfishly due to constraints on my part but also to give us time to discuss possible agenda items here.

• Adopt a charter and whatever else this document says we need to do: https://github.com/iojs/io.js/blob/v1.x/WORKING_GROUPS.md
• Discuss the relationship between "Build" and "Docker"
• I'll propose a roadmap for the activity we need to conduct for Build and we can discuss that and possibly even assign some responsibilities and map out a path. Items including:
  • Make more progress towards automatic testing of pull requests to iojs/io.js and libuv/libuv via the build-containers work (these are already running on jenkins).
  • Automatic testing of merged commits via the full CI build set
  • Reporting status to iojs/io.js and libuv/libuv. I have a nice custom badge for the READMEs in my head that would show status across the various platforms.
  • Easier test triggers for io.js and libuv collaborators so they can request containerised or full CI runs on any fork & branch/commit without having to log in to Jenkins to do it.
  • Longer-term plans to replace Jenkins with our own solution.
• Discuss the current list of platforms and how we might want to extend it and a timeframe for extending it. I know that the libuv folk are interested in a broader set of test platforms than io.js but we're not doing much to oblige that yet.
  • We also need to discuss how we might expand the list of cloud/hosting companies providing hardware. We're leaning pretty heavily on DigitalOcean and Rackspace at the moment and I wouldn't mind diversifying a little and even bringing in providers of other platforms.
• Discuss how ARM fits into the picture. I'm quite interested in ramping up our ARM testing and nightly/release builds to make io.js perfect for IOT/embedded/single-board applications. I've been considering putting out a call for hardware donations so we can get better coverage of hardware that people are actually using.

[rvagg]

Another item:

• Security, multiple concerns:
  • Infrastructure providers trust us to limit our activity to what we've said we're going to do and we need to ensure that we're responsible with that
  • Some infrastructure is provided by non-infrastructure companies and we are even more constrained in who we can give access to. The OS X machines are behind Voxer's firewall so we have to limit access to them. The ARM machines are in my office, on my network, so I need to limit access to them.
  • We are producing binaries that are widely used and must be trustworthy. We have to limit access to the machine that are producing those binaries in order to retain that trust. There is a little bit of cross-over between test and CI machines but I've been trying to keep that to a minimum.
  • We have to protect the signing keys for the Windows and OS X installers (currently owned by NodeSource)
  • We have to protect the TLS certificates for both iojs.org (owned by @indutny) and jenkins-iojs.nodesource.com (wildcard cert owned by NodeSource).
  • We have to be careful allowing arbitrary code to run on the full CI set because it's mostly not containerised (this is why the plan is to put PRs through containerised build slaves first). We need to set up structures to keep that separation in place and educate everyone who has access to escalate CI runs to the full set.

Part of the difficulty in scaling up the involvement of other people in the Build work I've been doing comes back to the points above, so I'd love help in exploring how we can expand the number of people involved while being cognisant of the security issues involved.

[bnoordhuis]

I don't really have anything to contribute to the Docker WG but I can sit in as a libuv representative if needed.

[rmg]

I'm definitely interested.

[jbergstroem]

Yeah, I'd be interested as well.

[othiym23]

I'm also interested.

[retrohacker]

Also interested. Could you expand on

Discuss the relationship between "Build" and "Docker"

Also, what requirements do we have for ARM donations?

[Qard]

I'm super interested in ARM stuff. We might be able to get some help from Online Labs.

[Fishrock123]

Longer-term plans to replace Jenkins with our own solution.

I'd be interesting in working on this in the long term.

[edouardb]

Hi, I am from Online Labs, @Qard told us that you are in need of ARM-based CI nodes. We can provide you some nodes, ping us on IRC (irc.online.net/#onlinelabs)

[rvagg]

@wblankenship:

Discuss the relationship between "Build" and "Docker"

I put this on the agenda to discuss, I have no strong opinions on it but my working model has been something like this: the io.js Docker team is really a sub-working group of the io.js Build team with some overlapping responsibilities--mainly the build-containers repo. At the same time however, the Docker team should be autonomously responsible for the docker-iojs repo and not have its activities there dictated by anyone in the broader Build team. So in some ways the Docker team is a separate WG but given the small scope of its responsibilities I can't see it making sense to have it spin up as a separate WG, leaving build-containers purely in Build's scope of responsibility. But of course this is all open to discussion which is why it'd be great to have y'all join a meeting.

Also, what requirements do we have for ARM donations?

No requirements at this stage except that I'd like to see any build assets under the control of the a subset of the Build team (what that subset is I don't know, currently just me in practice). I'll outline this in a new issue I think but the idea is this: there are lots of people using Node on ARM hardware and we can neither track that usage or keep up with it. So instead of going and actively hunting down all the hardware and operating system variants we can get our hands on to test with, lets have the community donate test hardware that we can use either as part of our CI infrastructure or in some way that informs that. The issue of BeagleBone Black is one of these, it's very popular for NodeBots but I don't have one and I'm not inclined to go out and buy one just for this because I don't see its usage as being anything near the rate of RPi for Node, but it comes with a quirky old Linux distro that makes it awkward and our current ARMv7 builds don't work on it because of glibc incompatibilities. Also consider ARMv8 support, I can't wait to have this as first-class in io.js, it'd be awesome to be able to build and distribute 64-bit binaries. But hardware is either quite expensive or comes in the form of Chromebooks at the moment (although this looks to be changing that), so it'd be great if someone in the community, or a corporate sponsor, could provide hardware if it was important to them.

@edouardb (and @Qard), thanks for joining in the conversation! I actually have an Online Labs account (rvagg@nodesource.com, I was put on to you by @wolfeidau of Ninjablocks) and had a machine connected to the io.js CI for a while from there, however, the speed was significantly slower than the Odroid XU3 I have in my office and it impacting on the total speed of CI builds. However, since that time I've also put a couple of Raspberry Pis into the mix and they are obviously a lot slower again. I've also put ccache on each of the ARM machines and this has helped a lot. Currently the ARMv7 builds made available by io.js are made on the Odroid variant of Ubuntu 14.04 which has a much newer libc than some of the ARMv7 hardware out there (like the BeagleBone Black) so it's causing some pain and since you have a Wheezy distro available then perhaps that's a better place to make release builds.

[rmg]

Has a date/time been established yet? If the Doodle is any indication this is going to be a small group. In the interest of collaboration, I added in some evening slots for myself and it looks like the result is exactly 1 time slot that works for all 4 of the people who have filled in the doodle.

[jbergstroem]

I can probably extend my schedule as well if additional participants would opt for another time.

[jbergstroem]

Do we have a date/time set yet?

[rmg]

Looks like the first couple time slots on the doodle have already gone by..

/ping @rvagg

[rvagg]

Sorry, terrible week, super-busy and unwell at the same time.

I've picked a time that was both popular and worked best for me, unfortunately we didn't have a time that worked for everyone and @wblankenship is the one missing out on the time that we missed. Thankfully I work with him @ NodeSource so I can 1:1 if need be before/after and we'll also have a recorded Hangout.

@jbergstroem @rmg @othiym23 @wblankenship: please see http://doodle.com/r5cz2dq6rcpd9b5e for the time in your timezone, it's ~11 hours from now. I don't have time right now to set up the Hangout and the rest but I'll do that in the morning after the TC meeting and post details here. Anybody else who's interested but didn't respond to the doodle can join too, I'll dump the Hangout link in here.

[retrohacker]

It turns out that my last class of the day was canceled tonight, I have to commute back from campus but should be able to make it.

[rvagg]

• Hangout for participating: https://plus.google.com/hangouts/_/hoaevent/AP36tYdM5W875vX5xRYouREWZkfLnVzw9nkYGmibICbeim4W7J3Mcw
• YouTube for public viewing live & recorded, not participating: http://www.youtube.com/watch?v=OKQi3pTF7fs
• Doc for agenda and minutes: https://docs.google.com/document/d/1vRhsYBs4Hw6vRu55h5eWTwDzS1NctxdTvMMEnCbDs14 (empty so far, will attempt to put in an agenda, help from others is welcome!)

[rvagg]

in 45 minutes from now if that wasn't clear