= TDC Pipeline :page-layout: base :toc: preamble :source-language: java :icons: font :linkattrs: :sectanchors: :sectlink: :numbered: :imagesdir: img :doctype: book :tip-caption: :bulb: :note-caption: :information_source: :important-caption: :heavy_exclamation_mark: :caution-caption: :fire: :warning-caption: :warning:
image:https://travis-ci.org/rmpestano/tdc-pipeline.svg[Build Status (Travis CI), link=https://travis-ci.org/rmpestano/tdc-pipeline] image:https://coveralls.io/repos/rmpestano/tdc-pipeline/badge.png[Coverage, link=https://coveralls.io/r/rmpestano/tdc-pipeline]
Sample application for JavaEE Pipeline as code using Jenkins, Sonar and Docker presentation. Slides http://rmpestano.github.io/talks/slides/javaee-pipeline/index-en.html#/[available here^].
image:go-to-prod.png[link=https://raw.githubusercontent.com/rmpestano/tdc-pipeline/master/img/go-to-prod.png]
The pipeline code https://github.com/rmpestano/tdc-pipeline/blob/master/Jenkinsfile[can be found here^].
For pipeline DSL syntax highlight I recomment https://www.sublimetext.com/3[sublime text^] and this package: https://github.com/bjarneo/jenkinsfile-syntax.
To use it just clone the above git repository into .config/sublime-text-3/Packages/ and restart sublime.
== Running the application
IMPORTANT: You need Docker, Maven and Java installed on your machine.
All commands below must be executed in project root folder.
+
After that, a file named cars.h2.db will be created on your user home/db folder.
+
NOTE: your user must have write permissions in ~/db directory.
+
TIP: If you are on a non unix OS like Windows you must change db.path to an existing directory e.g: db.path=C:/db
If you have changed db.path in migrations command you must build the application with the same param:
+
NOTE: -v ~/db:/opt/jboss/db maps our host filesystem database (the one we created with Flyway) to docker container home dir so we can exit the container without losing data.
+
NOTE: /opt/jboss/ is the home dir (~/) in wildfly docker container.
+
. Access the application at http://localhost:8080/tdc-pipeline
== Running tests
=== Integration tests
Tests in this application use Arquillian, http://arquillian.org/guides/getting_started_rinse_and_repeat/[see this guide^] if you never heard about it.
If you want to run the test using an already running server activate wildfly-remote profile:
====
TIP: For it tests the database migration can be run only once to create tables. DataSets are prepared by http://arquillian.org/arquillian-extension-persistence/[Arquillian Persistence^].
=== Functional tests
TIP: For ft tests the database migration is required to be run every time the tests are run because Arquillian Persistence https://issues.jboss.org/browse/ARQ-1077[is not enabled in functional tests^].
=== Performance tests
Perf tests use https://gatling.io[Gatling tool^].
. <<Running the application,Deploy the application>>
By default perf tests run against http://localhost:8080/tdc-pipeline/ but you can change it by passing APP_CONTEXT system property:
====
== Running the pipeline
This application comes with a https://jenkins.io/doc/book/pipeline/syntax/[Jenkins declarative pipeline^]. Below are the steps to run the pipeline locally.
WARNING: At least 8GB of RAM is needed on the host machine, 16GB is recommended.
=== Jenkins
First thing that is needed is a Jenkins instance, http://mirrors.jenkins.io/war-stable/latest/jenkins.war[download Jenkins here^] and start it on port 8080 using:
It was tested with Jenkins 2.73.2 with following plugins installed:
====
=== Shared library
This pipeline have an https://github.com/rmpestano/tdc-pipeline/blob/master/vars/notify.groovy[example of shared library^]. Configure https://github.com/rmpestano/tdc-pipeline as a shared lib on Jenkins > Configure system > Global Pipeline Libraries like in image below:
NOTE: The name must be notify, it's the name of the function we invoke in https://github.com/rmpestano/tdc-pipeline/blob/a56134d792ff9686333820f24b1aa9e4e0a4d05e/Jenkinsfile#L156[pipeline here^].
image:shared-lib.png[link=https://raw.githubusercontent.com/rmpestano/tdc-pipeline/master/img/shared-lib.png]
More details on pipeline shared libs, https://jenkins.io/doc/book/pipeline/shared-libraries/[look here^].
=== Docker
This pipeline depends on Docker, install it according to your operating system https://docs.docker.com/engine/installation[as described here].
TIP: look for Docker CE (community edition).
TIP: To run docker without sudo https://askubuntu.com/questions/477551/how-can-i-use-docker-without-sudo[look here^].
=== Sonar
The pipeline depends on https://www.sonarqube.org/[Sonar^], you need to have a Sonar instance running on http://localhost:9000.
An easy way to start Sonar locally is just running it's docker container:
Configure Sonar on Jenkins (http://localhost:8080/configure):
image:sonar.png[link=https://raw.githubusercontent.com/rmpestano/tdc-pipeline/master/img/sonar.png]
=== Quality Gates
Quality gates can fail a pipeline if it doesn't reach the Sonar quality gates conditions, for more details https://blog.sonarsource.com/breaking-the-sonarqube-analysis-with-jenkins-pipelines/[see this post^].
Following is Jenkins configuration for quality gates:
image:jenkins-quality-gates.png[link=https://raw.githubusercontent.com/rmpestano/tdc-pipeline/master/img/jenkins-quality-gates.png]
=== Configure slack
This pipeline is integrated with https://slack.com/[slack^]. You'll need to configure your Jenkins instance to work with the https://wiki.jenkins.io/display/JENKINS/Slack+Plugin[Slack plugin^].
In http://localhost:8080/configure[Jenkins configuration^] search for Global Slack Notifier Settings and set Integration Token with SyQ9NWKGoEorB1g9h2h5xUuy and Base URL with value https://tdc-java.slack.com/services/hooks/jenkins-ci/.
The configuration above will configure jenkins to send messages to the following slack channel: https://tdc-java.slack.com/messages/C7L0N9V0B
NOTE: Use https://join.slack.com/t/tdc-java/shared_invite/enQtMjYxNDk1NDcwNzg5LWYyODdjMDQxZDgyMzNkMmUyYmQ2NzA2NWQ3YjI1NjBiYjk2YzYxNzUyYTRhMDg2MDI2N2Q3ZDVlYjRhM2U0NzE[this invitation link^] to join the channel above.
For more details on how to integrate your own slack channel and Jenkins see https://support.cloudbees.com/hc/en-us/articles/226460047-How-do-I-configure-Slack-Plugin-[this simple (5 steps) tutorial^].
=== Create the pipeline job
. Create a http://localhost:8080/view/all/newJob[new job^];
. Chose pipeline and give tdc-pipeline as name;
. On job config check Do not allow concurrent builds option on General section;
. On Build Triggers section Check pool scm and use * * * * * as value;
. Finally on Pipeline section select option Pipeline script from scm, chose GIT and use this url: http://github.com/rmpestano/tdc-pipeline;
After saving job configuration the pipeline should run on next minutes.
NOTE: There is an (manual) approval step before "going to production".
=== Jenkins on docker
There is also a https://github.com/rmpestano/docker-jenkins[jenkins docker container^] with above plugins installed, here is how to run it:
IMPORTANT: <
When running the pipeline on docker ft-tests stage will fail because it needs chrome on docker, to make it pass you need to run ft-tests with phantomjs:
====
=== After pipeline execution
The pipeline will generate a lot of evidences about the quality of the build:
. Sonar analysis along with code coverage can be viewed on http://localhost:9000/dashboard/index/com.github.rmpestano:tdc-pipeline + image::sonar-report.png[link=https://raw.githubusercontent.com/rmpestano/tdc-pipeline/master/img/sonar-report.png] . Each build VCS diff can be browsed on http://localhost:8080/job/tdc-pipeline/last-changes + image:vcs-diff.png[link=https://raw.githubusercontent.com/rmpestano/tdc-pipeline/master/img/vcs-diff.png] . Cucumber living documentation at http://localhost:8080/job/tdc-pipeline/cucumber-living-documentation + image:living-docs.png[link=https://raw.githubusercontent.com/rmpestano/tdc-pipeline/master/img/living-docs.png] . Performance tests report http://localhost:8080/job/tdc-pipeline/gatling + image:perf-reports.png[link=https://raw.githubusercontent.com/rmpestano/tdc-pipeline/master/img/perf-reports.png]
Last changes, Living docs and Gatling reports will be available at job level (http://localhost:8080/job/tdc-pipeline) if at least one pipeline succeeds. For failed pipelines you need to access build level, e.g: http://localhost:8080/job/tdc-pipeline/17/last-changes/ (where 17 is the build number).
image:pipeline-reports.png[]
====
Two docker containers will be started during the pipeline, one representing the application deployment on QA environment and another which is production:
image:containers.png[]
For QA the app should be available at http://localhost:8282/tdc-pipeline. In Prod env the application is available on port http://localhost:8181/tdc-pipeline.
=== Demo
Following is a demo video showing this pipeline: https://www.youtube.com/watch?v=xUlTyzsMPes