To be fully able to run a Jenkins Continuous Integration on a GitLab repository or project, you require the following plugins:
GitLab API Plugin - Wraps GitLab Java API.
GitLab Branch Source Plugin - Contains two packages:
io.jenkins.plugins.gitlabserverconfig
- Manages server configuration and web hooks management. Ideally should reside inside another plugin with name GitLab Plugin
. In future, this package will be moved into a new plugin.
io.jenkins.plugins.gitlabbranchsource
- Adds GitLab Branch Source for Multi-branch Pipeline Jobs (including
Merge Requests) and Folder organization.
You can use any one of these ways:
Install from Jenkins Update Center. Go to Jenkins > Configure > Manage Plugins > Available and search for gitlab branch source plugin
then select Install.
Using Plugin Management Tool
java -jar plugin-management-tool.jar
-p gitlab-branch-source
-d <path-to-default-jenkins-plugins-directory>
-w <path-to-jenkins-war>
From Source:
i. Checkout out source code to your local machine:
git clone https://github.com/jenkinsci/gitlab-branch-source-plugin.git
cd gitlab-branch-source-plugin
ii. Install the plugin:
mvn clean install
or
mvn clean install -DskipTests # to skip tests
iii. Run the Plugin:
mvn hpi:run # runs a Jenkins instance at localhost:8080
or
mvn hpi:run -Djetty.port={port} # to run on your desired port number
iv. Now the *.hpi
generated can be manually installed on your Jenkins instance:
1. Select `Manage Jenkins`
2. Select `Manage Plugins`
3. Select `Advanced` tab
3. In `Upload Plugin` section, select `Choose file`
4. Select `${root_dir}/target/gitlab-branch-source.hpi`
5. Select `Upload`
6. Select `Install without restart`
Download latest release from here and manually install.
After installing the plugin on your Jenkins instance, you need configure your GitLab Server settings.
On jenkins, select Manage Jenkins
.
Select Configure System
.
Scroll down to find the GitLab
section.
Select Add GitLab Server
| Select GitLab Server
.
Now you will now see the GitLab Server Configuration options
There are 4 fields that needs to be configured:
i. Name
- Plugin automatically generates an unique server name for you. User may want to configure this field to suit their needs but should make sure it is sufficiently unique. We recommend to keep it as it is.
ii. Server URL
- Contains the URL to your GitLab Server. By default it is set to "https://gitlab.com". User can modify it to enter their GitLab Server URL e.g. https://gitlab.gnome.org/, http://gitlab.example.com:7990. etc.
iii. Credentials
- Contains a list of credentials entries that are of type GitLab Personal Access Token or any String Credentials. When no credential has been added it shows "-none-". User can add a credential by clicking "Add" button.
iv. Mange Web Hook
- If you want the plugin to setup web hook on your GitLab project(s) to get push/mr/tag/note events then check this box.
iv. Mange System Hook
- If you want the plugin to setup system hook on your GitLab project(s) to detect if a project is removed then check this box. Remember plugin can only setup system hook on your server if supplied access token has Admin
access.
v. Secret Token
- The secret token is required to authenticate the webhook payloads received from GitLab Server. Use generate secret token from Advanced options or use your own. If you are a old plugin user and did not set a secret token previously and want secret token to applied to the hooks of your existing jobs, you can add the secret token and rescan your jobs. Existing hooks with new secret token will be applied.
vi. Root URL for hooks
- By default Root URL for hooks created by this plugin is your Jenkins instance url. You can modify the root URL in by adding your custom root URL. Leave empty if you want Jenkins URL to be your custom hook url. A path is added to your hook ROOT URL /gitlab-webhook/post
for webhooks and /gitlab-systemhook/post
for system hooks.
Adding a Personal Access Token Credentials:
This is a manual setup. To automatically generate Personal Access Token see next section.
i. User is required to add a GitLab Personal Access Token
or any String Credential
type credentials entry to securely persist the token inside Jenkins.
ii. Generate a Personal Access Token
on your GitLab Server
a. Select profile dropdown menu from top-right corner
b. Select `Settings`
c. Select `Access Token` from left column
d. Enter a name | Set Scope to `api` (If admin also give `sudo` which required for systemhooks and mr comment trigger)
e. Select `Create Personal Access Token`
f. Copy the token generated
iii. Return to Jenkins | Select Add
in Credentials field | Select Jenkins
.
iv. Set Kind
to GitLab Personal Access Token.
v. Enter Token
.
vi. Enter a unique id in ID
.
vii. Enter a human readable description.
viii. Select Add
.
Testing connection:
i. Select your desired token in the Credentials
dropdown.
ii. Select Test Connection
.
iii. It should return something like Credentials verified for user {username}
.
Select Apply
(at the bottom).
GitLab Server is now setup on Jenkins.
Alternatively, users can generate a GitLab Personal Access Token within Jenkins itself and automatically add the GitLab Personal Access Token credentials to Jenkins server credentials.
Select Advanced
at the bottom of GitLab
Section.
Select Manage Additional GitLab Actions
.
Select Convert login and password to token
.
Set the GitLab Server URL
.
There are 2 options to generate token:
i. From credentials
- To select an already persisting Username Password Credentials or add an Username Password
credential to persist it.
ii. From login and password
- If this is a one time thing then you can directly enter you credentials to the text boxes and the username/password credential is not persisted.
After setting your username/password credential, select Create token credentials
.
The token creator will create a Personal Access Token in your GitLab Server for the given user with the required scope and also create a credentials for the same inside Jenkins server. You can go back to the GitLab Server Configuration to select the new credentials generated (select "-none-" first then new credentials will appear). For security reasons this token is not revealed as plain text rather returns an id
. It is a 128-bit long UUID-4 string (36 characters).
Use the following end points for web hooks and system hooks setup on your GitLab Server. The Jenkins Url
needs to be a fully qualified domain name (FQDN) so cannot be localhost
.
<jenkins_url>/gitlab-webhook/post
with push
, tag
, merge request
and note
events.
<jenkins_url>/gitlab-systemhook/post
with repository update
event.
There is an easier way to setup GitLab Server configuration on your Jenkins server. No need for messing around in the UI.
Jenkins Configuration as Code (JCasC)
or simply Configuration as Code
Plugin allows you to configure Jenkins
via a yaml
file. If you are a first time user, you can learn more about JCasC
here.
Install Configuration as Code
Plugin on your Jenkins instance.
Refer to Installing a new plugin in Jenkins.
There are multiple ways to load JCasC yaml file to configure Jenkins:
JCasC by default searches for a file with the name jenkins.yaml
in $JENKINS_ROOT
.
The JCasC looks for an environment variable CASC_JENKINS_CONFIG
which contains the path for the configuration yaml
file.
A path to a folder containing a set of config files e.g. /var/jenkins_home/casc_configs
.
A full path to a single file e.g. /var/jenkins_home/casc_configs/jenkins.yaml
.
A URL pointing to a file served on the web e.g. https://<your-domain>/jenkins.yaml
.
You can also set the configuration yaml path in the UI. Go to <your-jenkins-domain>/configuration-as-code
. Enter path or URL to jenkins.yaml
and select Apply New Configuration
.
To configure your GitLab Server in Jenkins add the following to jenkins.yaml
:
credentials:
system:
domainCredentials:
- credentials:
- gitlabPersonalAccessToken:
scope: SYSTEM
id: "i<3GitLab"
token: "glpat-XfsqZvVtAx5YCph5bq3r" # gitlab personal access token
unclassified:
gitLabServers:
servers:
- credentialsId: "i<3GitLab" # same as id specified for gitlab personal access token credentials
manageWebHooks: true
manageSystemHooks: true # access token should have admin access to set system hooks
name: "gitlab-3214"
serverUrl: "https://gitlab.com"
hooksRootUrl: ""
secretToken: ""
See handling secrets section in JCasC documentation for better security.
GitLab Branch Source Plugin allows you to create 2 type of jobs:
Multibranch Pipeline Jobs
- For single project.Folder Organization
- For multiple projects inside a owner (user/group/subgroup).The Multibranch Pipeline job type enables you to implement different Jenkinsfiles for different branches of the same project. In a Multibranch Pipeline job, Jenkins automatically discovers, manages and executes Pipelines for Branches/Merge Requests/Tags which contain a Jenkinsfile
in source control. This eliminates the need for manual Pipeline creation and management.
To create a Multibranch Pipeline Job
:
Select New Item
on Jenkins home page.
Enter a name for your job, select Multibranch Pipeline
| select Ok
.
In Branch Sources
sections, select Add source
| select GitLab Project
.
Now you need to configure your jobs.
i. Select Server
configured in the initial server setup.
ii. [Optional] Add Checkout Credentials
(SSHPrivateKey or Username/Password) if there is any private projects that will be built by the plugin.
iii. Add path to the owner where the project you want to build exists. If user, enter username
. If group, enter group name
. If subgroup, enter subgroup path with namespace
.
iv. Based on the owner provided. All the projects are discovered in the path and added to the Projects
listbox. You can now choose the project you want to build.
v. Behaviours
(a.k.a. SCM Traits) allow different configurations option to your build. More about it in the SCM Trait APIs section.
Now you can go ahead and save the job.
For more info see this.
After saving, a new web hook is created in your GitLab Server if a GitLab Access Token
is specified in the server configuration. Then the branch indexing starts based on what options behaviours you selected. As the indexing proceeds new jobs are started and queued for each branches with a Jenkinsfile
in their root directory.
The Job results are notified to the GitLab Server as Pipeline Status for the HEAD commit of each branches built. The build for forked MR cannot be notified to GitLab Server as GitLab doesn't provide Pipeline status for Merge Requests from forks for security concerns. See this.
We have a workaround for this. Jenkins will build the MRs from forked projects if the MR author is a trusted owner i.e. has Developer
/Maintainer
/Owner
access level. More about it in the SCM Trait APIs section.
As the web hook is now setup on your Jenkins CI by the GitLab server. Any push-events or merge-request events or tag events trigger the concerned build in Jenkins.
Folders Organization enable Jenkins to monitor an entire GitLab User
/Group
/Subgroup
and automatically create new Multibranch Pipelines for projects which contain branches/merge requests/tags containing a Jenkinsfile
. In our plugin this type of job is called GitLab Group
.
To create a GitLab Group Job
:
Select New Item
on Jenkins home page.
Enter a name for your job.
Select Organization Folder
as the job type and press the Ok
button.
In Configuration
, under Projects
, select GitLab Group
as the Repository Sources
.
Now you need to configure your jobs.
i. Select Server
configured in the initial server setup.
ii. [Optional] Add Checkout Credentials
(SSHPrivateKey or Username/Password) only if there are any private projects required to be built.
iii. Add path to the owner whose projects you want to build. If user, enter username
. If group, enter group name
. If subgroup, enter subgroup path with namespace
.
v. Behaviours
(a.k.a. SCM Traits) are allow different configuration option to your build. More about it in the SCM Trait APIs section.
The indexing in this group job type only needs to discover one branch withJenkinsfile
and thus it only shows the partial indexing log. You need to visit individual projects to see their full indexing.
The following behaviours apply to both Multibranch Pipeline Jobs
and Folder Organization
(unless otherwise stated).
Discover branches
- To discover branches.
Only Branches that are not also filed as MRs
- If you are discovering origin merge requests, it may not make sense to discover the same changes both as a merge request and as a branch.Only Branches that are filed as MRs
- This option exists to preserve legacy behaviour when upgrading from older versions of the plugin. NOTE: If you have an actual use case for this option please file a merge request against this text.All Branches
- Ignores whether the branch is also filed as a merge request and instead discovers all branches on the origin project.Discover merge requests from origin
- To discover merge requests made from origin branches.
Merging the merge request merged with current target revision
- Discover each merge request once with the discovered revision corresponding to the result of merging with the current revision of the target branch.The current merge request revision
- Discover each merge request once with the discovered revision corresponding to the merge request head revision without merging.Both current mr revision and the mr merged with current target revision
- Discover each merge request twice. The first discovered revision corresponds to the result of merging with the current revision of the target branch in each scan. The second parallel discovered revision corresponds to the merge request head revision without merging.Discover merge requests from forks
- To discover merge requests made from forked project branches.
Strategy:
Merging the merge request merged with current target revision
- Discover each merge request once with the discovered revision corresponding to the result of merging with the current revision of the target branch.The current merge request revision
- Discover each merge request once with the discovered revision corresponding to the merge request head revision without merging.Both current mr revision and the mr merged with current target revision
- Discover each merge request twice. The first discovered revision corresponds to the result of merging with the current revision of the target branch in each scan. The second parallel discovered revision corresponds to the merge request head revision without merging.Trust
Members
- Discover MRs from Forked Projects whose author is a member of the origin project.Trusted Members
- [Recommended] Discover MRs from Forked Projects whose author is has Developer/Maintainer/Owner accesslevel in the origin project.Everyone
- Discover MRs from Forked Projects filed by anybody. For security reasons you should never use this option. It may be used to reveal your Pipeline secrets environment variables.Nobody
- Discover no MRs from Forked Projects at all. Equivalent to removing the trait altogether.If Members
or Trusted Members
is selected, then plugin will build the target branch of MRs from non/untrusted members.
These traits can be selected by selecting Add
in the Behaviours
section.
Tag discovery
- Discover tags in the project. To automatically build tags install basic-branch-build-plugin
.
Discover group/subgroup projects
- Discover subgroup projects inside a group/subgroup. Only applicable to GitLab Group
Job type whose owner is a Group
/Subgroup
but not User
.
Discover shared projects
- Discover projects that are shared with the configured owner group from another group. Up until version 684 of the plugin this used to be the default behavior but is now a separate trait that is not added by default due to potential security concerns.
Log build status as comment on GitLab
- Enable logging build status as comment on GitLab. A comment is logged on the commit or merge request once the build is completed. You can decide if you want to log success builds or not. You can also use sudo user to comment the build status as commment e.g. jenkinsadmin
or something similar.
Trigger build on merge request comment
- Enable trigger a rebuild of a merge request by comment with your desired comment body (default: jenkins rebuild
). The job can only be triggered by trusted members of the project i.e. users with Developer/Maintainer/Owner accesslevel (also includes inherited from ancestor groups). By default only trusted members of project can trigger MR.
You may want to disable this option because trusted members do not include members inherited from shared group (there is no way to get it from GitLabApi as of GitLab 13.0.0). If disabled, MR comment trigger can be done by any user having access to your project.
Disable GitLab project avatar
- Disable avatars of GitLab project(s). It is not possible to fetch avatars when API has no token authentication or project is private. So you may use this option as a workaround. We will fix this issue in a later release.
Project Naming Strategy
- Choose whether you want project name
, the full project path (with namespace)
, contextual project path (partial namespace
, or simple project path (no namespace)
as job names of each project. Due to legacy reasons, we have project path (with namespace)
as default naming scheme. Note if a job is already created and the naming strategy is changed it will cause projects and build logs to be destroyed.
Filter by name (with regex)
- Filter the type of items you want to discover in your project based on the regular expression specified. For example, to discover only master
branch, develop
branch and all Merge Requests add (master|develop|MR-.*)
.
Filter by name (with wildcards)
- Filter the type of items you want to discover in your project based on the wildcards specified. For example, to discover only master
branch, develop
branch and all Merge Requests add development master MR-*
.
Skip pipeline status notifications
- Disable notifying GitLab server about the pipeline status.
Override hook management modes
- Override default hook management mode of web hook and system hook. ITEM
credentials for webhook is currently not supported.
Checkout over SSH
- [Not Recommended] Use this mode to checkout over SSH. Use Checkout Credentials
instead.
Webhook Listener Conditions
- Set conditions based on the webhook content on when a build should be triggered.
By default Multibranch Jobs have the following environment variables (provided by Branch API Plugin):
Branch - BRANCH_NAME
Merge Request - BRANCH_NAME
, CHANGE_ID
, CHANGE_TARGET
, CHANGE_BRANCH
, CHANGE_FORK
, CHANGE_URL
, CHANGE_AUTHOR
, CHANGE_TITLE
. CHANGE_AUTHOR_DISPLAY_NAME
Tag - BRANCH_NAME
, TAG_NAME
, TAG_TIMESTAMP
, TAG_DATE
, TAG_UNIXTIME
This plugin adds a few more environment variables to Builds (WorkflowRun
type only) which is the payload received as WebHook) See https://docs.gitlab.com/ee/user/project/integrations/webhooks.html#events.
A few points to note:
If no response is recorded for any field in the Web Hook Payload, it returns an empty String. To add more variables see
package io.jenkins.plugins.gitlabbranchsource.Cause
.
GITLAB_OBJECT_KIND
- This environment variable should be used to check the event type before accessing the environment variables. Possible values arenone
,push
,tag_push
andmerge_request
.Any variables ending with
#
indicates the index of the list of the payload starting from 1.
Environment Variables are available from Push Event, Tag Push Event and Merge Request Event.
See https://docs.gitlab.com/ee/user/project/integrations/webhooks.html#push-events
Note:
Jenkins by default refrains from automatically building Tags on push (See reason). You need to install Branch Build Strategy Plugin to solve this.
See Guide: https://github.com/jenkinsci/basic-branch-build-strategies-plugin/blob/master/docs/user.adoc
Do remember if you are using Basic Branch Build for tag builds you also need to add strategies for branch and pull request (change request) else they would not be automatically built (See GIF below).
See https://docs.gitlab.com/ee/user/project/integrations/webhooks.html#tag-events
See https://docs.gitlab.com/ee/user/project/integrations/webhooks.html#merge-request-events
To create a Job DSL seed job see this tutorial.
Here is a sample seed job script for folder organization job:
organizationFolder('GitLab Organization Folder') {
description("GitLab org folder created with Job DSL")
displayName('My Project')
// "Projects"
organizations {
gitLabSCMNavigator {
projectOwner("baymac")
credentialsId("i<3GitLab")
serverName("gitlab-3214")
// "Traits" ("Behaviours" in the GUI) that are "declarative-compatible"
traits {
subGroupProjectDiscoveryTrait() // discover projects inside subgroups
gitLabBranchDiscovery {
strategyId(3) // discover all branches
}
originMergeRequestDiscoveryTrait {
strategyId(1) // discover MRs and merge them with target branch
}
gitLabTagDiscovery() // discover tags
}
}
}
// "Traits" ("Behaviours" in the GUI) that are NOT "declarative-compatible"
// For some 'traits, we need to configure this stuff by hand until JobDSL handles it
// https://issues.jenkins.io/browse/JENKINS-45504
configure {
def traits = it / navigators / 'io.jenkins.plugins.gitlabbranchsource.GitLabSCMNavigator' / traits
traits << 'io.jenkins.plugins.gitlabbranchsource.ForkMergeRequestDiscoveryTrait' {
strategyId(2)
trust(class: 'io.jenkins.plugins.gitlabbranchsource.ForkMergeRequestDiscoveryTrait$TrustPermission')
}
}
// "Project Recognizers"
projectFactories {
workflowMultiBranchProjectFactory {
scriptPath 'Jenkinsfile'
}
}
// "Orphaned Item Strategy"
orphanedItemStrategy {
discardOldItems {
daysToKeep(10)
numToKeep(5)
}
}
// "Scan Organization Folder Triggers" : 1 day
// We need to configure this stuff by hand because JobDSL only allow 'periodic(int min)' for now
triggers {
periodicFolderTrigger {
interval('1d')
}
}
}
To see all the APIs supported by Job DSL you can visit the following link:
http://localhost:8080/jenkins/plugin/job-dsl/api-viewer/index.html#path/organizationFolder-organizations-gitLabSCMNavigator-traits
You can also use JCasC to directly create job from a Job DSL seed job. Here's an example of the yaml config:
jobs:
- script: >
organizationFolder('GitLab Organization Folder') {
description("GitLab org folder configured with JCasC")
displayName('My Project')
// "Projects"
organizations {
gitLabSCMNavigator {
projectOwner("baymac")
credentialsId("i<3GitLab")
serverName("gitlab-3214")
// "Traits" ("Behaviours" in the GUI) that are "declarative-compatible"
traits {
subGroupProjectDiscoveryTrait() // discover projects inside subgroups
gitLabBranchDiscovery {
strategyId(3) // discover all branches
}
originMergeRequestDiscoveryTrait {
strategyId(1) // discover MRs and merge them with target branch
}
gitLabTagDiscovery() // discover tags
}
}
}
// "Traits" ("Behaviours" in the GUI) that are NOT "declarative-compatible"
// For some 'traits, we need to configure this stuff by hand until JobDSL handles it
// https://issues.jenkins.io/browse/JENKINS-45504
configure { node ->
def traits = node / navigators / 'io.jenkins.plugins.gitlabbranchsource.GitLabSCMNavigator' / traits
traits << 'io.jenkins.plugins.gitlabbranchsource.ForkMergeRequestDiscoveryTrait' {
strategyId('2')
trust(class: 'io.jenkins.plugins.gitlabbranchsource.ForkMergeRequestDiscoveryTrait$TrustPermission')
}
}
// "Project Recognizers"
projectFactories {
workflowMultiBranchProjectFactory {
scriptPath 'Jenkinsfile'
}
}
// "Orphaned Item Strategy"
orphanedItemStrategy {
discardOldItems {
daysToKeep(-1)
numToKeep(-1)
}
}
// "Scan Organization Folder Triggers" : 1 day
// We need to configure this stuff by hand because JobDSL only allow 'periodic(int min)' for now
configure { node ->
node / triggers / 'com.cloudbees.hudson.plugins.folder.computed.PeriodicFolderTrigger' {
spec('H H * * *')
interval(86400000)
}
}
}
You can also use file or url to load the script, see this.
This project uses Jenkins JIRA to track issues. You can file issues under gitlab-branch-source-plugin
component.
Send your mail in the Developer Mailing list.
Join our Gitter channel.
This plugin was created by the Google Summer of Code (GSoC) Team for Multibranch Pipeline Support for GitLab.
Current Maintainers: