Author: Janine Eichler
Email: jaeichle@redhat.com
Revision: 0.4
Introduce Jenkins Pipeline
Domain Architect: Eric Lavarde
Email: elavarde@redhat.com
Consultant: Patrick C. F. Ernzer
Email: pcfe@redhat.com
Date: February to November 2016
Revision: 0.3
Satellite 6.2 is now a minimum requirement, 6.1 will not work.
Satellite 6.3, 6.4 and 6.5 have been tested.
as a general rule the latest version of Satellite is the one Patrick develops against.
Author: Nick Strugnell
Email: nstrug@redhat.com
Date: 2014-11-20
Revision: 0.1
It is extremely helpful if you read the following blog posts, that explain the concepts behind this repo, before trying to implement.
Continuous Integration for Infrastructure (CII) is a process by which the Operating System build ("build") component of a Standard Operating Environment (SOE) can be rapidly developed, tested and deployed.
A build is composed of the following components:
The CII system consists of the following components:
The architecture is shown in this YeD diagram.
The following steps should help you get started with CII.
NB I have SELinux enabled on the Jenkins server and it poses no problems.
Install a standard RHEL 7 server with a minimum of 4GB RAM, 50GB availabile in /var/lib/jenkins
and 10GB available in /var/lib/mock
if you intend to do non CI triggered mock builds (highly recommended for debugging). It's fine to use a VM for this.
verify with timedatectl
that your timezone is set correctly (for correct timestamps in Jenkins).
Register the server to RHN RHEL7 base and RHEL7 rhel-7-server-satellite-tools repos. You need the Satellite Tools repo for puppet.
Configure the server for access to the EPEL and Jenkins repos.
Install httpd
, mock
, createrepo
, git
, nc
and puppet
on the system. All but mock are available from the standard RHEL repos so should just install with yum. mock is available from EPEL.
[root@jenkins ~]# yum install httpd mock createrepo git nc puppet
Ensure that httpd
is enabled, running and reachable.
[root@jenkins ~]# systemctl enable httpd ; systemctl start httpd
[root@jenkins ~]# firewall-cmd --get-active-zones
[root@jenkins ~]# firewall-cmd --zone=public --add-service=http --permanent
[root@jenkins ~]# firewall-cmd --zone=public --add-service=https --permanent
[root@jenkins ~]# firewall-cmd --reload
[root@jenkins ~]# firewall-cmd --zone=public --list-all
Configure mock
by copying the rhel-7-x86_64.cfg or rhel-6-x86_64.cfg file to /etc/mock
on the jenkins server and setting MOCK_CONFIG for the relevant Jenkins job.
YOUROWNKEY
with your key as found in the /etc/yum.repos.d/redhat.repo
file on the Jenkins server./etc/yum.repos.d/redhat.repo
config_opts['use_nspawn'] = False
to your relevant mock config files.Install jenkins
, tomcat
and Java. If you have setup the Jenkins repo correctly you should be able to simply use yum.
[root@jenkins ~]# yum install jenkins tomcat java
Ensure that jenkins
is enabled, running and reachable.
[root@jenkins ~]# systemctl enable jenkins ; systemctl start jenkins
[root@jenkins ~]# firewall-cmd --zone=public --add-port="8080/tcp" --permanent
[root@jenkins ~]# firewall-cmd --reload
[root@jenkins ~]# firewall-cmd --zone=public --list-all
Now that Jenkins is running, browse to it's console at http://jenkinsserver:8080/
Select the 'Manage Jenkins' link, followed by 'Manage Plugins'. You will need to add the following plugins:
Select 'Configure System'
Restart Jenkins
Add the jenkins
user to the mock
group (usermod -a -G mock jenkins
). This will allow Jenkins to build RPMs.
Create /var/www/html/pub/soe-repo
, /var/www/html/pub/soe-puppet
and assign their ownership to the jenkins
user. These will be used as the upstream repositories to publish artefacts to the satellite.
/var/www/html/pub/soe-puppet-only
for the puppet only workflow and assign its ownership to the jenkins
user. It serves for the puppet only workflow.su
to the jenkins
user (su jenkins -s /bin/bash
) and use ssh-keygen
to create an ssh keypair. These will be used for authentication to both the git repository, and to the satellite server.
First of all, let's have a look at the bigger picture here. You'll create at least two jobs. One will be solely responsible for polling your SCM. When changes are detected on the SCM, the second job will be triggered. The second job will then take care of building, pushing to Satellite and running the tests. It is important to note that these steps (building the rpm's and/or puppet modules, pushing them to Satellite, running the tests etc.) will be done using a Jenkins Pipeline, so it is code written in Groovy and part of the soe-ci git repository. Documentation for the pipeline can be found here. We use a scripted pipeline, not the declarative one.
The configuration options will be explained later on.
In general you might want to have multiple of these job pairs. E.g. for EL6 and EL7, building only puppet modules or the "full" build including RPMs.
Right now a couple of job parameters are used which you need to configure:
Parameter Name | Description |
---|---|
SOE_CI_REPO_URL | the git repository url of your soe-ci project |
SOE_CI_BRANCH | the branch containing the Jenkinsfile and config file to build the RPMs and puppet modules from |
CREDENTIALS_ID_SOE_CI_IN_JENKINS | the credentials id of the user configured in Jenkisn to access the source code of the soe ci project in git |
ACME_SOE_REPO_URL | the git repository url to checkout the 'acme-soe' project from |
ACME_SOE_BRANCH | the branch to checkout of the 'acme-soe' repo. |
CREDENTIALS_ID_ACME_SOE_IN_JENKINS | analogous to CREDENTIALS_ID_SOE_CI_IN_JENKINS, can be the same, depending on your setup |
RHEL_VERSION | this param indirectly configures two things: which mock config to use (the pattern is /etc/mock/ |
REBUILD_VMS | whether or not to reinstall the test VMs before running the tests |
POWER_OFF_VMS_AFTER_BUILD | Whether or not to power off the VMs after the build. The build result (successful / failed) is not taken into consideration. |
PUPPET_ONLY | Whether or not only puppet modules should be built and pushed. This will reduce the duration of the build significantly (provided you update a CV that only contains puppet modules), however it ignores RPMs completely. |
CLEAN_WORKSPACE | Whether or not to clean the jenkins workspace before the job execution. |
VERBOSE | Whether or not to run the executed scripts in a verbose mode. Basically it decides whether run 'bash -x' or just 'bash ' |
It's simple. It's written in Groovy. If you want to test your changes first before pushing multiple commits to the repository, you can simply create a new job, which is a copy of your "second" job (remember, there are two jobs, one which pulls and one which builds), and change in the "Pipeline" section the "Defintion" from "Pipeline Script from SCM" to "Pipeline Script". Copy and paste your pipeline in the box (it's a groovy sandbox), and you're good to go. Remember this is for testing only. Once you're done, push your changes accordingly and delete the job you created for testing purposes.
script-env-vars.groovy script-env-vars-puppet-only.groovy script-env-vars-rpm.groovy
Jenkinsfile
in soe-ci.gitjenkins
user on the satellite.~jenkins/.hammer/cli.modules.d/foreman.yml
file (older Satellite versions use ~jenkins/.hammer/cli_config.yml
). More details here.jenkins
user on the Jenkins server to the jenkins
user on the satellite and ensure that jenkins
on the Jenkins server can do passwordless ssh
to the satellite.In order to create a Content View on the satellite, you need some initial content. That can be generated by Jenkins.
Now manually trigger both
This will fail, however it will create some content in the output directories by building the demo RPMs and Puppet modules. Check that these are available then do the following tasks:
script-env-vars-puppet-only.groovy
and script-env-vars-rpm.groovy
FIXME: add instructions on HC
At this point, you should be good to go. In fact Jenkins may have already kicked off a build for you when you pushed to github.
Develop your build in your checkout of acme-soe. Software that you want packaging goes in 'rpms', puppet modules in 'puppet' and BATS tests in 'tests'. You MUST update versions (in specfiles and metadata.json files) whenever you make a change, otherwise satellite6 will not pick up that you have new versions, even though Jenkins will have repackaged them.