search

cloudogu / ces-build-lib

Jenkins pipeline shared library adding features for Maven, Gradle, Docker, SonarQube, Git and others
GNU Affero General Public License v3.0
75 stars 49 forks source link
ces docker git gradle java jenkins jenkins-pipeline maven sonarqube

readme

ces-build-lib

Jenkins Pipeline Shared library, that contains additional features for Git, Maven, etc. in an object-oriented manner as well as some additional pipeline steps.

Table of contents

Usage

Syntax completion

You can get syntax completion in your Jenkinsfile when using the ces-build-lib, by adding it as dependency to your project.

You can get the source.jar from JitPack.

With Maven this can be done like so:

Current version is .
For further details and options refer to the JitPack website.

This is confirmed to work with IntelliJ IDEA.

Maven

Maven from local Jenkins tool

Run maven from a local tool installation on Jenkins.

See MavenLocal

def mvnHome = tool 'M3'
def javaHome = tool 'OpenJDK-8'
Maven mvn = new MavenLocal(this, mvnHome, javaHome)

stage('Build') {
    mvn 'clean install'
}

Maven Wrapper

Run maven using a Maven Wrapper from the local repository.

With local JDK tool

Similar to MavenLocal you can use the Maven Wrapper with a JDK from a local tool installation on Jenkins:

def javaHome = tool 'OpenJDK-8'
Maven mvn = new MavenWrapper(this, javaHome)

stage('Build') {
    mvn 'clean install'
}

With the JDK provided by the build agent

It is also possible to not specify a JDK tool and use the Java Runtime on the Build agent's PATH. However, experience tells us that this is absolutely non-deterministic and will result in unpredictable behavior.
So: Better set an explicit Java tool to be used or use MavenWrapperInDocker.

Maven mvn = new MavenWrapper(this)

stage('Build') {
    mvn 'clean install'
}

Maven in Docker

Run maven in a docker container. This can be helpful, when

Plain Maven In Docker

The builds are run inside the official maven containers from Dockerhub

See MavenInDocker

Maven mvn = new MavenInDocker(this, "3.5.0-jdk-8")

stage('Build') {
    mvn 'clean install'
}

Maven Wrapper In Docker

It's also possible to use the MavenWrapper in a Docker Container. Here, the Docker container is responsible for providing the JDK.

See MavenWrapperInDocker

Maven mvn = MavenWrapperInDocker(this, 'adoptopenjdk/openjdk11:jdk-11.0.10_9-alpine')

stage('Build') {
    mvn 'clean install'
}

Since Oracle's announcement of shorter free JDK support, plenty of JDK images have appeared on public container image registries, where adoptopenjdk is just one option. The choice is yours.

Advanced Maven in Docker features

The following features apply to plain Maven as well as Maven Wrapper in Docker.

Maven starts new containers

If you run Docker from your maven build, because you use the docker-maven-plugin for example, you can connect the docker socket through to your docker in maven like so:

stage('Unit Test') {
    // The UI module build runs inside a docker container, so pass the docker host to the maven container
    mvn.enableDockerHost = true

    mvn docker:start 

    // Don't expose docker host more than necessary
    mvn.enableDockerHost = false
}

There are some security-related concerns about this. See Docker.

Local repo

Maven in Docker

If you would like to use Jenkin's local maven repo (or more accurate the one of the build executor, typically at /home/jenkins/.m2) instead of a maven repo per job (within each workspace), you can use the following options:

Maven mvn = new MavenInDocker(this, "3.5.0-jdk-8")
mvn.useLocalRepoFromJenkins = true

This speed speeds up the first build and uses less memory. However, concurrent builds of multi module projects building the same version (e.g. a SNAPSHOT), might overwrite their dependencies, causing non-deterministic build failures.

Set image and credentials

It is possible to set credentials for a registry login by setting a credentialsId and custom image with registry prefix.

Maven mvn = new MavenInDocker(this, "3.5.0-jdk-8") // uses image: maven:3.5.0-jdk-8 from DockerHub
Maven mvn1 = new MavenInDocker(this, "mirror.gcr.io/maven:latest") // uses image: maven:latest from Google
Maven mvn2 = new MavenInDocker(this, "3.5.0-jdk-8" , credentialsId) // loads the username and password credentials from jenkins
Maven without Docker

The default is the default maven behavior /home/jenkins/.m2 is used. If you want to use a separate maven repo per Workspace (e.g. to avoid concurrent builds overwriting dependencies of multi module projects building the same version (e.g. a SNAPSHOT) the following will work:

mvn.additionalArgs += " -Dmaven.repo.local=${env.WORKSPACE}/.m2"

Lazy evaluation / execute more steps inside container

If you need to execute more steps inside the maven container you can pass a closure to your maven instance that is lazily evaluated within the container. The String value returned are the maven arguments. 

Maven mvn = new MavenInDocker(this, "3.5.0-jdk-8"),
echo "Outside Maven Container! ${new Docker(this).findIp()}"
mvn {
    echo "Insinde Maven Container! ${new Docker(this).findIp()}"
    'clean package -DskipTests'
}

Mirrors

You can define maven mirrors as follows:

Maven.useMirrors([name: 'maven-proxy', mirrorOf: 'central', url: 'https://maven.example.org'],
                 [name: 'google-maven', mirrorOf: 'central', url: 'https://maven-central.storage.googleapis.com/maven2/'],
)

Repository Credentials

If you specified one or more <repository> in your pom.xml that requires authentication, you can pass these credentials to your ces-build-lib Maven instance like so:

mvn.useRepositoryCredentials([id: 'ces', credentialsId: 'nexusSystemUserCredential'],
                             [id: 'another', credentialsId: 'nexusSystemUserCredential'])

Note that the id must match the one specified in your pom.xml and the credentials ID must belong to a username and password credential defined in Jenkins.

Deploying to Nexus repository

Deploying artifacts

ces-build-lib makes deploying to nexus repositories easy, even when it includes signing of the artifacts and usage of the nexus staging plugin (as necessary for Maven Central or other Nexus repository pro instances).

Simple deployment

The most simple use case is to deploy to a nexus repo (not Maven Central):

Simple Example: 

# <distributionManagement> in pom.xml (preferred)
mvn.useRepositoryCredentials([id: 'ces', credentialsId: 'nexusSystemUserCredential'])
# Alternative: Distribution management via Jenkins (deprecated)
mvn.useRepositoryCredentials([id: 'ces', url: 'https://ecosystem.cloudogu.com/nexus', credentialsId: 'nexusSystemUserCredential', type: 'Nexus2'])

# Deploy
mvn.deployToNexusRepository()

Note that if the pom.xml's version contains -SNAPSHOT, the artifacts are automatically deployed to the snapshot repository (e.g. on oss.sonatype.org). Otherwise, the artifacts are deployed to the release repository (e.g. on oss.sonatype.org).

Signing artifacts (e.g. Maven Central)

If you want to sign the artifacts before deploying, just set the credentials for signing before deploying, using Maven.setSignatureCredentials() passing the secret key as ASC file (as jenkins secret file credential) and the passphrase (as jenkins secret text credential). An ASC file can be exported via gpg --export-secret-keys -a ABCDEFGH > secretkey.asc. See Working with PGP Signatures

Deploying with staging (e.g. Maven Central)

Another option is to use the nexus-staging-maven-plugin instead of the default maven-deploy-plugin. This is useful if you deploy to a Nexus repository pro, such as Maven Central.

Just use the Maven.deployToNexusRepositoryWithStaging() instead of Maven.deployToNexusRepository().

When deploying to Maven Central, make sure that your pom.xml adheres to the requirements by Maven Central, as stated here.

Note that as of nexus-staging-maven-plugin version 1.6.8, it does seem to read the distribution repositories from pom.xml only.

That is, you need to specify them in your pom.xml, they cannot be passed by the ces-build-lib. So for example for maven central you need to add the following:

<distributionManagement>
    <snapshotRepository>
        <id>ossrh</id>
        <url>https://oss.sonatype.org/content/repositories/snapshots</url>
    </snapshotRepository>
    <repository>
        <id>ossrh</id>
        <url>https://oss.sonatype.org/service/local/staging/deploy/maven2/</url>
    </repository>
</distributionManagement>

In addition you either have to pass an url to useRepositoryCredentials() or specify the nexus-staging-maven plugin in your pom.xml:

  <plugin>
    <groupId>org.sonatype.plugins</groupId>
    <artifactId>nexus-staging-maven-plugin</artifactId>
    <!-- ... -->
    <configuration>
      <serverId>ossrh</serverId>
      <nexusUrl>https://oss.sonatype.org/</nexusUrl>
    </configuration>
  </plugin>

Either way, the repository ID (here: ossrh) and the base nexus URL (here: https://oss.sonatype.org) in distributionManagement and nexus-staging-maven plugin must conform to each other.

Summing up, here is an example for deploying to Maven Central:

// url is optional, if described in nexus-staging-maven-plugin in pom.xml 
mvn.useRepositoryCredentials([id: 'ossrh', url: 'https://oss.sonatype.org', credentialsId: 'mavenCentral-UsernameAndAcccessTokenCredential', type: 'Nexus2'])
mvn.setSignatureCredentials('mavenCentral-secretKey-asc-file','mavenCentral-secretKey-Passphrase')
mvn.deployToNexusRepositoryWithStaging()

Note that the staging of releases might well take 10 minutes. After that, the artifacts are in the release repository, which is later (feels like nightly) synced to Maven Central.

For an example see cloudogu/command-bus.

Deploying sites

Similar to deploying artifacts as described above, we can also easily deploy a Maven site to a "raw" maven repository.

Note that the site plugin does not provide options to specify the target repository via the command line. That is, it has to be configured in the pom.xml like so:

<distributionManagement>
    <site>
        <id>ces</id>
        <name>site repository cloudogu ecosystem</name>
        <url>dav:https://your.domain/nexus/repository/Site-repo/${project.groupId}/${project.artifactId}/${project.version}/</url>
    </site>
</distributionManagement>

Where Site-repo is the name of the raw repository that must exist in Nexus to succeed.

Then, you can deploy the site as follows:

mvn.useRepositoryCredentials([id: 'ces', credentialsId: 'nexusSystemUserCredential'])
mvn.deploySiteToNexus()

Where

For an example see cloudogu/continuous-delivery-slides-example

Passing additional arguments

Another option for deployToNexusRepositoryWithStaging() and deployToNexusRepository() is to pass additional maven arguments to the deployment like so: mvn.deployToNexusRepositoryWithStaging('-X') (enables debug output).

Maven Utilities

Available from both local Maven and Maven in Docker.

See Maven

Gradle

Gradle Wrapper in Docker

It's also possible to use a GradleWrapper in a Docker Container. Here, the Docker container is responsible for providing the JDK.

See GradleWrapperInDocker

Example:

String gradleDockerImage = 'openjdk:11.0.10-jdk'
Gradle gradlew = new GradleWrapperInDocker(this, gradleDockerImage)

stage('Build') {
    gradlew "clean build"
}

Since Oracle's announcement of shorter free JDK support, plenty of JDK images have appeared on public container image registries, where adoptopenjdk is just one option. The choice is yours.

See Maven in Docker for passing credentials to the registry.

Git

An extension to the git step, that provides an API for some commonly used git commands and utilities. Mostly, this is a convenient wrapper around using the sh 'git ...' calls.

Example: 

Git git = new Git(this)

stage('Checkout') {
  git 'https://your.repo'
  /* Don't remove folders starting in "." like .m2 (maven), .npm, .cache, .local (bower), etc. */
  git.clean('".*/"')
}

Credentials

You can optionally pass usernamePassword (i.e. a String containing the ID that refers to the Jenkins credentials) to Git during construction. These are then used for cloning and pushing.

Note that the username and passwort are processed by a shell. Special characters in username or password might cause errors like Unterminated quoted string. So it's best to use a long password that only contains letters and numbers for now.

Git annonymousGit = new Git(this)
Git gitWithCreds = new Git(this, 'ourCredentials')

annonymousGit 'https://your.repo'
gitWithCreds 'https://your.repo' // Implicitly passed credentials

Git Utilities

Read Only

Changes to local repository

Note that most changing operations offer parameters to specify an author. Theses parameters are optional. If not set the author of the last commit will be used as author and committer. You can specify a different committer by setting the following fields:

It is recommended to set a different committer, so it's obvious those commits were done by Jenkins in the name of the author. This behaviour is implemented by GitHub for example when committing via the Web UI.

Changes to remote repository

Docker

The Docker class provides the default methods of the global docker variable provided by docker plugin:

Docker methods provided by the docker plugin

Additional features provided by the Docker class

The Docker class provides additional convenience features:

Example from Jenkinsfile:

 Docker docker = new Docker(this)
 def dockerContainer = docker.build("image/name:1.0").run()
 waitUntil {
     sleep(time: 10, unit: 'SECONDS')
     return docker.isRunning(dockerContainer)
 }
 echo docker.findIp(dockerContainer)
 echo docker.findEnv(dockerContainer)

Docker.Image methods provided by the docker plugin

Additional features provided by the Docker.Image class

Examples:

Docker Container that uses its own docker client:

new Docker(this).image('docker') // contains the docker client binary
    .mountJenkinsUser()
    .mountDockerSocket()
    .inside() {
        sh 'whoami' // Would fail without mountJenkinsUser = true
        sh 'id' // Would fail without mountJenkinsUser = true

        // Start a "sibling" container and wait for it to return
        sh 'docker run hello-world' // Would fail without mountDockerSocket = true 

    }

Docker container that does not have its own docker client

new Docker(this).image('kkarczmarczyk/node-yarn:8.0-wheezy')
    .mountJenkinsUser()
    .mountDockerSocket()
    .installDockerClient('17.12.1')
    .inside() {
        // Start a "sibling" container and wait for it to return
        sh 'docker run hello-world' // Would fail without mountDockerSocket = true & installDockerClient()
    }

Dockerfile

The Dockerfile class provides functions to lint Dockerfiles. For example:

stage('Lint') {
    Dockerfile dockerfile = new Dockerfile(this)
    dockerfile.lint() // Lint with default configuration
    dockerfile.lintWithConfig() // Use your own hadolint configuration with a .hadolint.yaml configuration file
}

The tool hadolint is used for linting. It has a lot of configuration parameters which can be set by creating a .hadolint.yaml file in your working directory. See https://github.com/hadolint/hadolint#configure

SonarQube

When analyzing code with SonarQube there are a couple of challenges that are solved using ces-build-lib's SonarQube class:

Constructors

In general, you can analyse with or without the SonarQube Plugin for Jenkins:

With the SonarQube instance you can now analyze your code. When using the plugin (i.e. sonarQubeEnv) you can also wait for the quality gate status, that is computed by SonarQube asynchronously. Note that this does not work for token and usernamePassword.

A complete example

stage('Statical Code Analysis') {
  def sonarQube = new SonarQube(this, [sonarQubeEnv: 'sonarQubeServerSetupInJenkins'])

  sonarQube.analyzeWith(new MavenInDocker(this, "3.5.0-jdk-8"))
  sonarQube.timeoutInMinutes = 4

  if (!sonarQube.waitForQualityGateWebhookToBeCalled()) {
    unstable("Pipeline unstable due to SonarQube quality gate failure")
  }
}

Note that

Branches

By default, the SonarQube legacy logic, of creating one project per branch in a Jenkins Multibranch Pipeline project.

A more convenient alternative is the paid-version-only Branch Plugin or the sonarqube-community-branch-plugin, which has similar features but is difficult to install, not supported officially and does not allow for migration to the official branch plugin later on.

You can enable either branch plugins like so:

sonarQube.isUsingBranchPlugin = true
sonarQube.analyzeWith(mvn)

The branch plugin is using master as integration branch, if you want use a different branch as master you have to use the integrationBranch parameter e.g.:

def sonarQube = new SonarQube(this, [sonarQubeEnv: 'sonarQubeServerSetupInJenkins', integrationBranch: 'develop'])
sonarQube.isUsingBranchPlugin = true
sonarQube.analyzeWith(mvn)

Note that using the branch plugin requires a first analysis without branches.

You can do this on Jenkins or locally.

On Jenkins, you can achieve this by setting the following for the first run:

sonarQube.isIgnoringBranches = true
sonarQube.analyzeWith(mvn)

Recommendation: Use Jenkins' replay feature for this. Then commit the Jenkinsfile with isUsingBranchPlugin.

An alternative is running the first analysis locally, e.g. with maven mvn clean install sonar:sonar -Dsonar.host.url=https://sonarcloud.io -Dsonar.organization=YOUR-ORG -Dsonar.login=YOUR-TOKEN

SonarCloud

SonarCloud is a public SonarQube instance that has some extra features, such as PullRequest decoration for GitHub, BitBucket, etc. ces-build-lib encapsulates the setup in SonarCloud class. It works just like SonarQube, i.e. you can create it using sonarQubeEnv, token, etc. and it provides the analyzeWith() and waitForQualityGateWebhookToBeCalled() methods.
The only difference: You either have to pass your organization ID using the sonarOrganization: 'YOUR_ID' parameter during construction, or configure it under https://yourJenkinsInstance/configure as "Additional analysis properties" (hit the "Advanced..." button to get there): sonar.organization=YOUR_ID.

Example using SonarCloud:

  def sonarQube = new SonarCloud(this, [sonarQubeEnv: 'sonarcloud.io', sonarOrganization: 'YOUR_ID'])

  sonarQube.analyzeWith(new MavenInDocker(this, "3.5.0-jdk-8"))

  if (!sonarQube.waitForQualityGateWebhookToBeCalled()) {
    unstable("Pipeline unstable due to SonarCloud quality gate failure")
  }

Just like for ordinary SonarQube, you have to setup a webhook in SonarCloud for waitForQualityGateWebhookToBeCalled() to work (see above).

If you want SonarCloud to decorate your Pull Requests, you will have to

See also Pull Request analysis.

Note that SonarCloud uses the Branch Plugin, so the first analysis has to be done differently, as described in Branches.

Pull Requests in SonarQube

As described above, SonarCloud can annotate PullRequests using the SonarCloud Application for GitHub. It is no longer possible to do this from a regular community edition SonarQube, as the GitHub Plugin for SonarQube is deprecated.

So a PR build is treated just like any other. That is,

The Jenkins GitHub Plugin sets BRANCH_NAME to the PR Name, e.g. PR-42.

Changelog

Provides the functionality to read changes of a specific version in a changelog that is based on the changelog format on https://keepachangelog.com/.

Note: The changelog will automatically be formatted. Characters like ", ', \ will be removed. A \n will be replaced with \\n. This is done to make it possible to pass this string to a json struct as a value.

Example: 

Changelog changelog = new Changelog(this)

stage('Changelog') {
  String changes = changelog.getChangesForVersion('v1.0.0')
  // ...
}

changelogFileName

You can optionally pass the path to the changelog file if it is located somewhere else than in the root path or if the file name is not CHANGELOG.md.

Example: 

Changelog changelog = new Changelog(this, 'myNewChangelog.md')

stage('Changelog') {
  String changes = changelog.getChangesForVersion('v1.0.0')
  // ...
}

GitHub

Provides the functionality to do changes on a github repository such as creating a new release.

Example: 

Git git = new Git(this)
GitHub github = new GitHub(this, git)

stage('Github') {
  github.createRelease('v1.1.1', 'Changes for version v1.1.1')
}

GitFlow

A wrapper class around the Git class to simplify the use of the git flow branching model.

Example: 

Git git = new Git(this)
git.committerName = 'jenkins'
git.committerEmail = 'jenkins@your.org'
GitFlow gitflow = new GitFlow(this, git)

stage('Gitflow') {
  if (gitflow.isReleaseBranch()){
    gitflow.finishRelease(git.getSimpleBranchName())
  }
}

SCM-Manager

Provides the functionality to handle pull requests on a SCMManager repository.

You need to pass usernamePassword (i.e. a String containing the ID that refers to the Jenkins credentials) to SCMManager during construction. These are then used for handling the pull requests.

SCMManager scmm = new SCMManager(this, 'ourCredentials')

Set the repository url through the repositoryUrl property like so:

SCMManager scmm = new SCMManager(this, 'https://hostname/scm', 'ourCredentials')

Pull Requests

Each method requires a repository parameter, a String containing namespace and name, e.g. cloudogu/ces-build-lib.

Example:

def scmm = new SCMManager(this, 'https://your.ecosystem.com/scm', scmManagerCredentials)

def pullRequestId = scmm.createPullRequest('cloudogu/ces-build-lib', 'feature/abc', 'develop', 'My title', 'My description')
pullRequestId = scmm.searchPullRequestIdByTitle('cloudogu/ces-build-lib', 'My title')
scmm.updatePullRequest('cloudogu/ces-build-lib', pullRequestId, 'My new title', 'My new description')
scmm.addComment('cloudogu/ces-build-lib', pullRequestId, 'A comment')

HttpClient

HttpClient provides a simple curl frontend for groovy.

Example:

HttpClient http = new HttpClient(scriptMock, 'myCredentialID')

// Simplest example
echo http.get('http://url')

// POSTing data
def dataJson = JsonOutput.toJson([
    comment: comment
])
def response = http.post('http://url/comments"', 'application/json', dataJson)

if (response.status == '201' && response.content-type == 'application/json') {
    def json = readJSON text: response.body
    echo json.count
}

K3d

K3d provides functions to set up and administer a local k3s cluster in Docker.

Example:

K3d k3d = new K3d(this, env.WORKSPACE, env.PATH)

try {
    stage('Set up k3d cluster') {
        k3d.startK3d()
    }

    stage('Do something with your cluster') {
        k3d.kubectl("get nodes")
    }
    stage('Apply your Helm chart') {
        k3d.helm("install path/to/your/chart")
    }

    stage('build and push development artefact') {
        String myCurrentArtefactVersion = "yourTag-1.2.3-dev"
        imageName = k3d.buildAndPushToLocalRegistry("your/image", myCurrentArtefactVersion)
        // your image name may look like this: k3d-citest-123456/your/image:yourTag-1.2.3-dev
        // the image name can be applied to your cluster as usual, f. i. with k3d.kubectl() with a customized K8s resource 
    }

    stage('configure components'){
        // add additional components
        k3d.configureComponents(["k8s-minio"             : ["version": "latest", "helmRepositoryNamespace": "k8s"],
                                 "k8s-loki"              : ["version": "latest", "helmRepositoryNamespace": "k8s"],
                                 "k8s-promtail"          : ["version": "latest", "helmRepositoryNamespace": "k8s"],
                                 "k8s-blueprint-operator": null, // null values will delete components from the config
        ])
    }

    stage('execute k8s-ces-setup') {
        k3d.setup('0.20.0')
    }

    stage('install resources and wait for them') {
        imageName = "registry.cloudogu.com/official/my-dogu-name:1.0.0"
        k3d.installDogu("my-dogu-name", imageName, myDoguResourceYamlFile)

        k3d.waitForDeploymentRollout("my-dogu-name", 300, 5)
    }

    stage('install a dependent dogu by applying a dogu resource') {
        k3d.applyDoguResource("my-dependency", "nyNamespace", "10.0.0-1")
        k3d.waitForDeploymentRollout("my-dependency", 300, 5)
    }

} catch (Exception ignored) {
    // in case of a failed build collect dogus, resources and pod logs and archive them as log file on the build.
    k3d.collectAndArchiveLogs()
    throw e
} finally {
    stage('Remove k3d cluster') {
        k3d.deleteK3d()
    }
}

DoguRegistry

DoguRegistry provides functions to easily push dogus and k8s components to a configured registry.

Example:

DoguRegistry registry = new DoguRegistry(this)

// push dogu
registry.pushDogu()

// push k8s component
registry.pushK8sYaml("pathToMyK8sYaml.yaml", "k8s-dogu-operator", "mynamespace", "0.9.0")

Bats

Bats provides functions to easily execute existing bats tests for a project.

Example:

Docker docker = new Docker(this)

stage('Bats Tests') {
    Bats bats = new Bats(this, docker)
    bats.checkAndExecuteTests()
}

Makefile

Makefile provides function regarding the Makefile from the current directory.

Example:

    Makefile makefile = new Makefile(this)
    String currentVersion = makefile.getVersion()

Markdown

Markdown provides function regarding the Markdown Files from the projects docs directory 

    Markdown markdown = new Markdown(this)
    markdown.check()

markdown.check executes the function defined in Markdown running a container with the latest https://github.com/tcort/markdown-link-check image and verifies that the links in the defined project directory are alive

Additionally, the markdown link checker can be used with a specific version (default: stable).

    Markdown markdown = new Markdown(this, "3.11.0")
    markdown.check()

DockerLint (Deprecated)

Use Dockerfile.lint() instead of lintDockerfile()! See Dockerfile

lintDockerfile() // uses Dockerfile as default; optional parameter

See lintDockerFile

ShellCheck

shellCheck() // search for all .sh files in folder and runs shellcheck
shellCheck(fileList) // fileList="a.sh b.sh" execute shellcheck on a custom list

See shellCheck

Steps

mailIfStatusChanged

Provides the functionality of the Jenkins Post-build Action "E-mail Notification" known from freestyle projects.

catchError {
 // Stages and steps
}
mailIfStatusChanged('a@b.cd,123@xy.z')

See mailIfStatusChanged

isPullRequest

Returns true if the current build is a pull request (when the CHANGE_IDenvironment variable is set) Tested with GitHub.

stage('SomethingToSkipWhenInPR') {
    if (!isPullRequest()) {
      // ...
    }

}

findEmailRecipients

Determines the email recipients: For branches that are considered unstable (all except for 'master' and 'develop') only the Git author is returned (if present). Otherwise, the default recipients (passed as parameter) and git author are returned.

catchError {
 // Stages and steps
}
mailIfStatusChanged(findEmailRecipients('a@b.cd,123@xy.z'))

The example writes state changes email to 'a@b.cd,123@xy.z' + git author for stable branches and only to git author for unstable branches.

findHostName

Returns the hostname of the current Jenkins instance. For example, if running on http(s)://server:port/jenkins, server is returned.

isBuildSuccessful

Returns true if the build is successful, i.e. not failed or unstable (yet).

findVulnerabilitiesWithTrivy

Returns a list of vulnerabilities or an empty list if there are no vulnerabilities for the given severity.

findVulnerabilitiesWithTrivy(trivyConfig as Map)

trivyConfig = [ 
    imageName: 'alpine:3.17.2', 
    severity: [ 'HIGH, CRITICAL' ], 
    trivyVersion: '0.41.0',
    additionalFlags: '--ignore-unfixed'
]

Here the only mandatory field is imageName. If no imageName was passed the function returns an empty list.

Simple examples

node {
    stage('Scan Vulns') {
        def vulns = findVulnerabilitiesWithTrivy(imageName: 'alpine:3.17.2')
        if (vulns.size() > 0) {
            archiveArtifacts artifacts: '.trivy/trivyOutput.json'
            unstable "Found  ${vulns.size()} vulnerabilities in image. See vulns.json"
        }
    }
}

Ignore / allowlist

If you want to ignore / allow certain vulnerabilities please use a .trivyignore file Provide the file in your repo / directory where you run your job e.g.:

.gitignore
Jenkinsfile
.trivyignore

Offical documentation

# Accept the risk
CVE-2018-14618

# Accept the risk until 2023-01-01
CVE-2019-14697 exp:2023-01-01

# No impact in our settings
CVE-2019-1543

# Ignore misconfigurations
AVD-DS-0002

# Ignore secrets
generic-unwanted-rule
aws-account-id

If there are vulnerabilities the output looks as follows.

{
  "SchemaVersion": 2,
  "ArtifactName": "alpine:3.17.2",
  "ArtifactType": "container_image",
  "Metadata": {
    "OS": {
      "Family": "alpine",
      "Name": "3.17.2"
    },
    "ImageID": "sha256:b2aa39c304c27b96c1fef0c06bee651ac9241d49c4fe34381cab8453f9a89c7d",
    "DiffIDs": [
      "sha256:7cd52847ad775a5ddc4b58326cf884beee34544296402c6292ed76474c686d39"
    ],
    "RepoTags": [
      "alpine:3.17.2"
    ],
    "RepoDigests": [
      "alpine@sha256:ff6bdca1701f3a8a67e328815ff2346b0e4067d32ec36b7992c1fdc001dc8517"
    ],
    "ImageConfig": {
      "architecture": "amd64",
      "container": "4ad3f57821a165b2174de22a9710123f0d35e5884dca772295c6ebe85f74fe57",
      "created": "2023-02-11T04:46:42.558343068Z",
      "docker_version": "20.10.12",
      "history": [
        {
          "created": "2023-02-11T04:46:42.449083344Z",
          "created_by": "/bin/sh -c #(nop) ADD file:40887ab7c06977737e63c215c9bd297c0c74de8d12d16ebdf1c3d40ac392f62d in / "
        },
        {
          "created": "2023-02-11T04:46:42.558343068Z",
          "created_by": "/bin/sh -c #(nop)  CMD [\"/bin/sh\"]",
          "empty_layer": true
        }
      ],
      "os": "linux",
      "rootfs": {
        "type": "layers",
        "diff_ids": [
          "sha256:7cd52847ad775a5ddc4b58326cf884beee34544296402c6292ed76474c686d39"
        ]
      },
      "config": {
        "Cmd": [
          "/bin/sh"
        ],
        "Env": [
          "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
        ],
        "Image": "sha256:ba2beca50019d79fb31b12c08f3786c5a0621017a3e95a72f2f8b832f894a427"
      }
    }
  },
  "Results": [
    {
      "Target": "alpine:3.17.2 (alpine 3.17.2)",
      "Class": "os-pkgs",
      "Type": "alpine",
      "Vulnerabilities": [
        {
          "VulnerabilityID": "CVE-2023-0464",
          "PkgID": "libcrypto3@3.0.8-r0",
          "PkgName": "libcrypto3",
          "InstalledVersion": "3.0.8-r0",
          "FixedVersion": "3.0.8-r1",
          "Layer": {
            "DiffID": "sha256:7cd52847ad775a5ddc4b58326cf884beee34544296402c6292ed76474c686d39"
          },
          "SeveritySource": "nvd",
          "PrimaryURL": "https://avd.aquasec.com/nvd/cve-2023-0464",
          "DataSource": {
            "ID": "alpine",
            "Name": "Alpine Secdb",
            "URL": "https://secdb.alpinelinux.org/"
          },
          "Title": "Denial of service by excessive resource usage in verifying X509 policy constraints",
          "Description": "A security vulnerability has been identified in all supported versions of OpenSSL related to the verification of X.509 certificate chains that include policy constraints. Attackers may be able to exploit this vulnerability by creating a malicious certificate chain that triggers exponential use of computational resources, leading to a denial-of-service (DoS) attack on affected systems. Policy processing is disabled by default but can be enabled by passing the `-policy' argument to the command line utilities or by calling the `X509_VERIFY_PARAM_set1_policies()' function.",
          "Severity": "HIGH",
          "CweIDs": [
            "CWE-295"
          ],
          "CVSS": {
            "nvd": {
              "V3Vector": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H",
              "V3Score": 7.5
            },
            "redhat": {
              "V3Vector": "CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:N/I:N/A:H",
              "V3Score": 5.9
            }
          },
          "References": [
            "https://access.redhat.com/security/cve/CVE-2023-0464",
            "https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-0464",
            "https://git.openssl.org/gitweb/?p=openssl.git;a=commitdiff;h=2017771e2db3e2b96f89bbe8766c3209f6a99545",
            "https://git.openssl.org/gitweb/?p=openssl.git;a=commitdiff;h=2dcd4f1e3115f38cefa43e3efbe9b801c27e642e",
            "https://git.openssl.org/gitweb/?p=openssl.git;a=commitdiff;h=879f7080d7e141f415c79eaa3a8ac4a3dad0348b",
            "https://git.openssl.org/gitweb/?p=openssl.git;a=commitdiff;h=959c59c7a0164117e7f8366466a32bb1f8d77ff1",
            "https://nvd.nist.gov/vuln/detail/CVE-2023-0464",
            "https://ubuntu.com/security/notices/USN-6039-1",
            "https://www.cve.org/CVERecord?id=CVE-2023-0464",
            "https://www.openssl.org/news/secadv/20230322.txt"
          ],
          "PublishedDate": "2023-03-22T17:15:00Z",
          "LastModifiedDate": "2023-03-29T19:37:00Z"
        }
      ]
    }
  ]
}

Examples

What is the Cloudogu EcoSystem?

The Cloudogu EcoSystem is an open platform, which lets you choose how and where your team creates great software. Each service or tool is delivered as a Dogu, a Docker container. Each Dogu can easily be integrated in your environment just by pulling it from our registry.

We have a growing number of ready-to-use Dogus, e.g. SCM-Manager, Jenkins, Nexus Repository, SonarQube, Redmine and many more. Every Dogu can be tailored to your specific needs. Take advantage of a central authentication service, a dynamic navigation, that lets you easily switch between the web UIs and a smart configuration magic, which automatically detects and responds to dependencies between Dogus.

The Cloudogu EcoSystem is open source and it runs either on-premises or in the cloud. The Cloudogu EcoSystem is developed by Cloudogu GmbH under AGPL-3.0-only.

License

Copyright © 2020 - present Cloudogu GmbH This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, version 3. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details. You should have received a copy of the GNU Affero General Public License along with this program. If not, see https://www.gnu.org/licenses/. See LICENSE for details.

MADE WITH :heart: FOR DEV ADDICTS. Legal notice / Imprint