Prototype CI/CD pipeline with Kontena

[joasilva]

Description

Test and document the setup of a CI/CD pipeline with Kontena.

DoD

The basic use should be running an ATTX container triggered by Jenkins/Git. Preferably, we should be able to run our tests as well.

Testing

Manual testing, peer review.

[joasilva]

Initial idea: from GitHub push, trigger running a container trough Drone via webook.

Drone is designed from the ground up to be run as containerised CD platform, with containerised artifacts and plugins. There's a ready-made official Kontena stack, so it's a natural candidate.

@jkesanie @blankdots HOWEVER, I couldn't find a Gradle plugin for Drone (cf. http://plugins.drone.io/), so how could we run our integration tests with Drone?

[joasilva]

Executing Gradle tests in Kontena seems to be unpractical for the time being (Drone plugin on Kontena, kontena-cli on Jenkins, etc.). Though this project could be tested once it develops a Kontena plugin (on their roadmap): https://github.com/qaware/gradle-cloud-deployer

But we can starting by testing Kontena+Drone CI/CD with the "push-to-master-branch" scenario.

[blankdots]

@joasilva running integration tests are run in a test container so theoretically you should have the test container that has Gradle in the stack ... this is how we are currently doing it. My assumption is that we can create a stack which runs at the end the test container and somehow gets the result ... we only need Gradle in the test container not on Drone.

[joasilva]

@blankdots Yes, that came into my mind (cf. https://hub.docker.com/_/gradle/).

[blankdots]

@joasilva we are not using that ... we have a different image, namely https://hub.docker.com/r/frekele/gradle/ ... oficial image did not work with the setup we need --- see image https://github.com/ATTX-project/platform-tests/blob/dev/attx-broker-tests/Dockerfile

[joasilva]

OK, so:

1. Git push of test.gradle triggers webhook to Drone running in Kontena
2. Drone runs pipeline: calls Gradle image (https://hub.docker.com/r/frekele/gradle/) to run test.gradle.

Studying "GitHub client id" and "GitHub secret" topics now.

[joasilva]

"GitHub client id" and "GitHub secret", figured out, managed to register Drone as OAuth app in Github (using docker-compose), trying now with Kontena.

[joasilva]

Drone running in Kontena registered with as OAuth app in Github, managed to sync repos. In progress: Drone.io pipelines. How to to build a container and run tests when a developer pushes to Github.

[joasilva]

Repo synch with Github works well in Kontena running in CSC, except that Drone fails all builds with the following message: ERROR: Error response from daemon: client is newer than server (client API version: 1.26, server API version: 1.24)

This because the stable (and beta) CoreOS relases use an old Docker version:

core@cpouta-grid-node-3 ~ $ docker version
Client:
 Version:      1.12.6
 API version:  1.24
 Go version:   go1.7.6
 Git commit:   a82d35e
 Built:        Wed Aug 16 00:03:08 2017
 OS/Arch:      linux/amd64

Server:
 Version:      1.12.6
 API version:  1.24
 Go version:   go1.7.6
 Git commit:   a82d35e
 Built:        Wed Aug 16 00:03:08 2017
 OS/Arch:      linux/amd64

Workaround: upgrade CoreOS to alpha channel, in progress.

[joasilva]

The upgrade of CoreOS nodes succeeded:

core@cpouta-grid-node-1 ~ $ cat /etc/os-release
NAME="Container Linux by CoreOS"
ID=coreos
VERSION=1548.0.0
VERSION_ID=1548.0.0
BUILD_ID=2017-09-27-0012
PRETTY_NAME="Container Linux by CoreOS 1548.0.0 (Ladybug)"
ANSI_COLOR="38;5;75"
HOME_URL="https://coreos.com/"
BUG_REPORT_URL="https://issues.coreos.com"
COREOS_BOARD="amd64-usr"

core@cpouta-grid-node-1 ~ $ docker version
Client:
 Version:      17.06.2-ce
 API version:  1.30
 Go version:   go1.8.3
 Git commit:   cec0b72
 Built:        Tue Sep  5 17:16:07 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.06.2-ce
 API version:  1.30 (minimum version 1.12)
 Go version:   go1.8.3
 Git commit:   cec0b72
 Built:        Tue Sep  5 17:16:07 2017
 OS/Arch:      linux/amd64
 Experimental: false

Running again the [test/learning Drone pipeline)(https://github.com/joasilva/hello_hapi/blob/master/.drone.yml), drone could now build and test the learning repo:

+ npm run test
npm info it worked if it ends with ok
npm info using npm@5.3.0
npm info using node@v8.6.0
npm info lifecycle hello@1.0.0~pretest: hello@1.0.0
npm info lifecycle hello@1.0.0~test: hello@1.0.0

> hello@1.0.0 test /drone/src/github.com/joasilva/hello_hapi/commit/07c052dd74b5bbd511c1af32e0cac09ba1a47896
> NODE_ENV=test node node_modules/lab/bin/lab -v -L -C -D

Basic HTTP Tests
  ✔ 1) Greets /hello/homer}  (102 ms)
  ✔ 2) Greets /hello/Homer%20Simpson}  (13 ms)
test greetings
  ✔ 3) greets with name  (1 ms)

3 tests complete
Test duration: 141 ms
No global variable leaks detected
Linting results: No issues

npm info lifecycle hello@1.0.0~posttest: hello@1.0.0
npm info ok 

Next step: try to make a pipeline for some simple Gradle build. Help from @blankdots will be appreciated.