Redo of the build scripts

[unya]

Current build scripts are becoming, in my humble opinion, unwieldy due to heavy dependence on Makefile.

The actual support matrix is growing (we now have 3 different distros, and 3 different node versions, possibly coupled by multiple variant images and updated base system image versioning. Writing modular Makefile code doesn't count as "simple" thing, especially due to mess of shelling out for what is often trivial processing (because GNU Makefile functions are complex).

I propose that we replace both the current configure script and Makefile with Ruby-based Rakefile due to simplicity of extending it with custom functions and having easily available a proper programming language instead of mostly basic text expansion features of GNU Makefile.

[Elexy]

I also feel passing everything through a ./configure and then wrangle make through all the (growing) conditions we have to build it makes no sense anymore. Rake seems like a good option, although it adds a few dependencies. The bigger picture here is that we will run builds in an automated way from which we want a configurable amount of logging and sensible errors. Since we are a big JS house, should we look at http://jakejs.com/ ? Rake sounds like a good option as well, but why not plain old bash scripts?

Right now we run all the make stuff in CircleCI, but we should be able to run them in something like Jenkins and GoCD as well.

[conoro]

Any concerns about the Ruby dependency and version hell? Also lack of people with Ruby expertise in nearForm? PoBs (did I just invent an acronym?) sounds good to me despite the horror of the language :-)

[Elexy]

Ruby dependencies [look here], among others are a concern. I used to script everything that was beyonmd my (ba)sh in Ruby, so I'm not afraid we don't have the expertise. I know more peeps in nF with Ruby knowlegde.

Having said that, I think we should have a good look at http://jakejs.com/ We are, arguably, a Node.js authority, Jake is a mature and up to date project and we have all the tools in place to use it. https://www.sitepoint.com/replace-make-with-jake/

[unya]

Bash is a mess that can quickly grow unwieldy, and I say it as someone known to create madness-inducing zsh oneliners that make things like Map/Reduce :)

Regarding Rake - the biggest dependency is Ruby itself, which IMO is not that big of an issue but I'll defer to other's judgement. I know that I consider it much easier to get running than dealing with anything that is installed through NPM. It's at least partially because I am not a JS developer, though :)

[mcollina]

I think we should move to a better alternative.

I know Ruby decently well (even if it has been 7 years since last time I wrote something in it), and Ruby is a fairly common language for all the DevOp tasks because it's highly flexible to be some domain specific files in it. It's also significantly more commonly used than Jake (which I didn't know it existed before).

Do we need a task runner at all? Task runners are very handy if a developer have to run the individual tasks independently. However if we use a full programming language we can just write a program that runs of all if - I don't think these tasks are designed to be run independently. This creates the best setup to avoid to ship a ton of dependencies - ideally we can even ship the builder as a container.

Can we do a quick survey of the devops at nF? Tasks on this could be easily pick up by other people, and we should focus on keeping a single set of tools.

[conoro]

+1 to surveying all our DevOps guys. Is there a Poll feature in Slack that we could use on the #dev-ops channel?

[Elexy]

yes there is. but what do we put in the poll?

---

Alex Knol Senior DevOps Engineer at nearForm | +34693206868 | alex.knol@nearform.com [image: LinkedIn] https://linkedin.com/in/https://www.linkedin.com/in/elexy/[image: Twitter] https://twitter.com/https://twitter.com/E_lexy[image: skype: alex.knol] https://www.nearform.com/skype/?user=alex.knol[image: Github] https://github.com/https://github.com/Elexy

[image: nearform.com] https://www.nearform.com/Follow us on Twitter https://twitter.com/nearform, LinkedIn https://ie.linkedin.com/company/nearform, Facebook https://www.facebook.com/nearFormLtd/, YouTube https://www.youtube.com/channel/UCp2Tsbjd3P8itnBHUNHi82A

On 2 May 2018 at 17:03, conoro notifications@github.com wrote:

+1 to surveying all our DevOps guys. Is there a Poll feature in Slack that we could use on the #dev-ops channel?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/nearform/nodejs-distribution/issues/14#issuecomment-386009545, or mute the thread https://github.com/notifications/unsubscribe-auth/AAazF1qIY5xHexrhaM6JEdwBzhVdxWMZks5tucqsgaJpZM4TlEvs .

[conoro]

I think 2x polls:

1. What languages do you have expertise in for DevOps Automation:

   • Bash or other shells
   • Python
   • Ruby
   • Groovy
   • JavaScript
   • Other (please specify)

2. What languages do you recommend for DevOps Automation (in order of preference):

   • Bash or other shells
   • Python
   • Ruby
   • Groovy
   • JavaScript
   • Other (please specify)

Any other common ones people might use? Clojure, Go, CMD, Perl, Rust Evangelism Task Force, Java, Powershell, cough Tcl.

Or be more specific around shells? Zsh, fish, etc.

[Elexy]

the two languages that came out on top: python and Golang. @unya decided to stick with Make and some additions in this branch: https://github.com/nearform/nodejs-distribution/tree/flexible-builds

Not sure I like it yet. Would be a good exercise to build a go binary that we pull down in the build pipeline. Actually decoupling the build script from this repository.