search

jenkinsci / klocwork-plugin

Jenkins Klocwork Plugin
https://plugins.jenkins.io/klocwork/
MIT License
20 stars 21 forks source link

Running klocwork scan on docker agent #69

Open grzegorz-kotarski opened 2 years ago

grzegorz-kotarski commented 2 years ago

Jenkins and plugins versions report

Environment ```text Result Jenkins: 2.303.1 OS: Linux - 5.15.0-43-generic --- Parameterized-Remote-Trigger:3.1.5.1 PrioritySorter:4.0.0 ace-editor:1.1 allure-jenkins-plugin:2.29.0 ant:1.11 antisamy-markup-formatter:2.1 apache-httpcomponents-client-4-api:4.5.13-1.0 authentication-tokens:1.4 badge:1.8 blueocean:1.25.0 blueocean-autofavorite:1.2.4 blueocean-bitbucket-pipeline:1.25.0 blueocean-commons:1.25.0 blueocean-config:1.25.0 blueocean-core-js:1.25.0 blueocean-dashboard:1.25.0 blueocean-display-url:2.4.1 blueocean-events:1.25.0 blueocean-git-pipeline:1.25.0 blueocean-github-pipeline:1.25.0 blueocean-i18n:1.25.0 blueocean-jira:1.25.0 blueocean-jwt:1.25.0 blueocean-personalization:1.25.0 blueocean-pipeline-api-impl:1.25.0 blueocean-pipeline-editor:1.25.0 blueocean-pipeline-scm-api:1.25.0 blueocean-rest:1.25.0 blueocean-rest-impl:1.25.0 blueocean-web:1.25.0 bootstrap4-api:4.6.0-3 bootstrap5-api:5.1.1-1 bouncycastle-api:2.25 branch-api:2.7.0 build-pipeline-plugin:1.5.8 build-timeout:1.20 built-on-column:1.1 caffeine-api:2.9.2-29.v717aac953ff3 checks-api:1.7.2 cloudbees-bitbucket-branch-source:2.9.11 cloudbees-folder:6.16 command-launcher:1.6 conditional-buildstep:1.4.1 config-autorefresh-plugin:1.0 copyartifact:1.46.2 credentials:2.6.1 credentials-binding:1.27 cvs:2.19 display-url-api:2.3.5 docker-build-publish:1.3.3 docker-commons:1.17 docker-java-api:3.1.5.2 docker-plugin:1.2.3 docker-workflow:1.26 durable-task:1.39 echarts-api:5.1.2-11 email-ext:2.83 emailext-template:1.2 envinject:2.4.0 envinject-api:1.8 external-monitor-job:1.7 favorite:2.3.3 font-awesome-api:5.15.4-1 generic-webhook-trigger:1.77 gerrit-code-review:0.4.7 gerrit-trigger:2.35.2 gerrit-verify-status-reporter:0.0.3 ghprb:1.42.2 git:4.8.2 git-client:3.10.0 git-server:1.10 github:1.34.1 github-api:1.133 github-branch-source:2.11.3 github-pullrequest:0.3.0 gradle:1.37.1 groovy-postbuild:2.5 handlebars:3.0.8 handy-uri-templates-2-api:2.1.8-1.0 htmlpublisher:1.25 jackson2-api:2.12.4 javadoc:1.6 jaxb:2.3.0.1 jdk-tool:1.5 jenkins-design-language:1.25.0 jenkins-multijob-plugin:1.36 jira:3.6 jjwt-api:0.11.2-9.c8b45b8bb173 job-dsl:1.78.1 job-import-plugin:3.4 jobConfigHistory:2.28.1 jquery:1.12.4-1 jquery-detached:1.2.1 jquery-ui:1.0.2 jquery3-api:3.6.0-2 jsch:0.1.55.2 junit:1.53 klocwork:2022.2 ldap:2.7 lockable-resources:2.11 mailer:1.34 mapdb-api:1.0.9.0 matrix-auth:2.6.8 matrix-project:1.19 maven-plugin:3.13 mercurial:2.15 momentjs:1.1.1 okhttp-api:3.14.9 pam-auth:1.6 parameterized-trigger:2.41 pipeline-build-step:2.15 pipeline-github-lib:1.0 pipeline-githubnotify-step:1.0.5 pipeline-graph-analysis:1.11 pipeline-input-step:2.12 pipeline-milestone-step:1.3.2 pipeline-model-api:1.9.3 pipeline-model-declarative-agent:1.1.1 pipeline-model-definition:1.9.3 pipeline-model-extensions:1.9.3 pipeline-rest-api:2.19 pipeline-stage-step:2.5 pipeline-stage-tags-metadata:1.9.3 pipeline-stage-view:2.19 pipeline-utility-steps:2.10.0 plain-credentials:1.7 plugin-util-api:2.5.0 popper-api:1.16.1-2 popper2-api:2.10.1-1 powershell:1.6 protecode-sc:0.18.2 pubsub-light:1.16 python:1.3 resource-disposer:0.16 role-strategy:3.2.0 run-condition:1.5 scm-api:2.6.5 script-security:1.78 snakeyaml-api:1.29.1 sse-gateway:1.24 ssh-credentials:1.19 ssh-slaves:1.33.0 sshd:3.1.0 structs:1.23 subversion:2.14.5 throttle-concurrents:2.5 timestamper:1.13 token-macro:266.v44a80cf277fd trilead-api:1.0.13 variant:1.4 windows-slaves:1.8 workflow-aggregator:2.6 workflow-api:2.46 workflow-basic-steps:2.24 workflow-cps:2633.v6baeedc13805 workflow-cps-global-lib:2.21 workflow-cps-global-lib-http:1.13.0 workflow-durable-task-step:2.40 workflow-job:2.42 workflow-multibranch:2.26 workflow-scm-step:2.13 workflow-step-api:2.24 workflow-support:3.8 ws-cleanup:0.39 ```

What Operating System are you using (both controller, and any agents involved in the problem)?

Ubuntu-based containers.

Reproduction steps

  1. Unpack klocwork installation to /tmp/klocwork/latest

  2. Add klocwork installation named latest with path /tmp/klocwork/latest

  3. Create pipeline with docker agent, e.g.:

    agent { docker { image '' reuseNode true label 'some_label' } } stage('Klocwork') { environment { KW_LTOKEN = '/tmp/ltoken' KW_PROJECT='' } stages { // call kwauth & kwdeploy here stage('Initialize') { steps { klocworkWrapper(installConfig: 'latest', ltoken: "${KW_LTOKEN }", serverConfig: '', serverProject: "${KW_PROJECT}") { klocworkBuildSpecGeneration([ additionalOpts: '', buildCommand: './build.sh', // NOTE: put proper build command/script here ignoreErrors: false, output: 'kwinject.out', tool: "kwinject"]) klocworkIntegrationStep1([ buildSpec: 'kwinject.out', disableKwdeploy: false, duplicateFrom: '', ignoreCompileErrors: true, importConfig: '', incrementalAnalysis: false, tablesDir: 'kwtables']) // rest of pipeline

  4. Run the pipeline

Expected Results

kwinject working properly.

Actual Results

[Pipeline] { [Pipeline] klocworkBuildSpecGeneration [Klocwork BuildSpecBuilder] - Starting Klocwork Build Specification Generation Step [my-pipeline] $ docker exec --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** --env **** cf5ba0f7bed568a7702aa710ea139cfcedae9b61234a425eef93848bd0e14505 /bin/sh -c "kwinject --version" /bin/sh: 1: kwinject: not found Return code: 127

Anything else?

It looks like kwinject cannot be found.

If I put full path to the kwinject using tool argument to klocworkBuildSpecGeneration then it can be found, but the next step - executing kwbuildproject fails, and it is not possible to provide full path to kwbuildproject.

For me it is strange the command is being run with docker command - because the pipeline is running in docker container already - see beginning of the build log:

[Pipeline] Start of Pipeline
[Pipeline] getContext
[Pipeline] node
Running on my-node in /home/jenkins/workspace/my-pipeline
[Pipeline] {
[Pipeline] isUnix
[Pipeline] sh
+ docker inspect -f . <MY_IMAGE>
.
[Pipeline] withDockerContainer
my-node does not seem to be running inside a container
$ docker run -t -d -u 1002:120 -v /home/jenkins/.ssh:/home/jenkins/.ssh -w /home/jenkins/workspace/my-pipeline -v /home/jenkins/workspace/`my-pipeline:/home/jenkins/workspace/my-pipeline:rw,z -v /home/jenkins/workspace/my-pipeline_tmp:/home/jenkins/workspace/my-pipeline_tmp:rw,z -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** <MY_IMAGE> cat
$ docker top cf5ba0f7bed568a7702aa710ea139cfcedae9b61234a425eef93848bd0e14505 -eo pid,comm
[Pipeline] { // rest of build log

Perhaps this is the problem (?)

michael-baron commented 2 years ago

Hi,

The plugin requires the path for the location of the Klocwork tools to be set. For a pipeline job you can add:

PATH = "/tmp/klocwork/latest/bin:$PATH"

to the environment section of your pipeline. This should resolve the issue with the location of the tools being unknown.

Best Regards,

Michael

grzegorz-kotarski commented 2 years ago

Hi,

The plugin requires the path for the location of the Klocwork tools to be set. For a pipeline job you can add:

PATH = "/tmp/klocwork/latest/bin:$PATH"

to the environment section of your pipeline. This should resolve the issue with the location of the tools being unknown.

Best Regards,

Michael

Hi Michael,

It does not work, it looks like it is related to https://issues.jenkins.io/browse/JENKINS-48082

See example pipeline code:

stage('Docker') {
        agent {
          docker {
            image '<MY_IMAGE>'
          }
        }
        steps {
            echo 'Docker:'
            withEnv(['PATH+WHATEVER=/tmp/klocwork']) {
                echo "env.PATH=${env.PATH}"
                sh 'echo sh PATH = $PATH'
                sh '''#!/usr/bin/env bash
                echo bash PATH = $PATH
                '''
            }
        }
    }
    stage('Master') {
        agent {
          label 'master'
        }
        steps {
            echo 'Master:'
            withEnv(['PATH+WHATEVER=/tmp/klocwork']) {
                echo "env.PATH=${env.PATH}"
                sh 'echo sh PATH = $PATH'
                sh '''#!/usr/bin/env bash
                echo bash PATH = $PATH
                '''
            }
        }
    }

The result of its execution is:

$ docker run -t -d -u 1002:120 -v /home/jenkins/.ssh:/home/jenkins/.ssh -w /home/jenkins/workspace/my-pipeline -v /home/jenkins/workspace/my-pipeline:/home/jenkins/workspace/my-pipeline:rw,z -v /home/jenkins/workspace/my-pipeline_tmp:/home/jenkins/workspace/my-pipeline_tmp:rw,z -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** <MY_IMAGE> cat
$ docker top ae03c529a78bd9a947bb948bd5e9c428343997de60305d4b741f722d07712345 -eo pid,comm
[Pipeline] {
[Pipeline] echo
Docker:
[Pipeline] withEnv
[Pipeline] {
[Pipeline] echo
env.PATH=/tmp/klocwork:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
[Pipeline] sh
+ echo sh PATH = /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
sh PATH = /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
[Pipeline] sh
bash PATH = /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
$ docker stop --time=1 ae03c529a78bd9a947bb948bd5e9c428343997de60305d4b741f722d07712345
$ docker rm -f ae03c529a78bd9a947bb948bd5e9c428343997de60305d4b741f722d07712345
[Pipeline] // withDockerContainer
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Master)
[Pipeline] node
Running on Jenkins in /var/jenkins_home/workspace/my-node
[Pipeline] {
[Pipeline] echo
Master:
[Pipeline] withEnv
[Pipeline] {
[Pipeline] echo
env.PATH=/tmp/klocwork:/opt/java/openjdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
[Pipeline] sh
+ echo sh PATH = /tmp/klocwork:/opt/java/openjdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
sh PATH = /tmp/klocwork:/opt/java/openjdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
[Pipeline] sh
bash PATH = /tmp/klocwork:/opt/java/openjdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

As you can see for docker the env. variable has not changed.

Best Regards, Greg

grzegorz-kotarski commented 2 years ago

An "ugly" workaround would be to set the PATH variable using docker args: 

docker {
    image '<MY_IMAGE>'
    args '-e PATH=$PATH:/tmp/klocwork/latest/bin'