Plugin hosting request for buildpulse-flaky-tests-plugin

[siddhantdange]

Repository URL

https://github.com/buildpulse/buildpulse-flaky-tests-plugin

New Repository Name

buildpulse-flaky-tests-plugin

Description

Our Jenkins plugin allows one to easily send test results to BuildPulse for CI analytics and flaky test detection.

GitHub users to have commit permission

@siddhantdange

Jenkins project users to have release permission

sidbuildpulse

Issue tracker

GitHub issues

[github-actions[bot]]

Hello from your friendly Jenkins Hosting Checker

It appears you have some issues with your hosting request. Please see the list below and correct all issues marked Required. Your hosting request will not be approved until these issues are corrected. Issues marked with Warning or Info are just recommendations and will not stall the hosting process.

• :no_entry: Required: The following usernames in 'Jenkins project users to have release permission' need to log into Jira: sidbuildpulse (reports are re-synced hourly, wait to re-check for a bit after logging in)
• :no_entry: Required: The following usernames in 'Jenkins project users to have release permission' need to log into Artifactory: sidbuildpulse (reports are re-synced hourly, wait to re-check for a bit after logging in)

You can re-trigger a check by editing your hosting request or by commenting /hosting re-check

[siddhantdange]

/hosting re-check

[github-actions[bot]]

Hello from your friendly Jenkins Hosting Checker

It looks like you have everything in order for your hosting request. A member of the Jenkins hosting team will check over things that I am not able to check(code review, README content, etc) and process the request as quickly as possible. Thank you for your patience.

Hosting team members can host this request with /hosting host

[jenkins-cert-app]

Security audit, information and commands

The security team is auditing all the hosting requests, to ensure a better security by default.

This message informs you that a Jenkins Security Scan was triggered on your repository. It takes ~10 minutes to complete.

Commands

The bot will parse all comments, and it will check if any line start with a command. Security team only:

• /audit-ok => the audit is complete, the hosting can continue :tada:.
• /audit-skip => the audit is not necessary, the hosting can continue :tada:.
• /audit-findings => the audit reveals some issues that require corrections :pencil2:.

Anyone:

• /request-security-scan => the findings from the Jenkins Security Scan were corrected, this command will re-scan your repository :mag:.
• /audit-review => the findings from the audit were corrected, this command will ping the security team to review the findings :eyes:. It's only applicable when the previous audit required changes.

Only one command can be requested per comment.

(automatically generated message, version: 1.28.6)

[jenkins-cert-app]

:x: Jenkins Security Scan failed. The Security team was notified about this.

[Wadeck]

Hey @siddhantdange I think the Security scan is failing because you do not include in the pom.xml the pluginRepository, leading to the failure to find maven-hpi-plugin:3.13 (available in the Jenkins repo)

    <pluginRepositories>
        <pluginRepository>
            <id>repo.jenkins-ci.org</id>
            <url>https://repo.jenkins-ci.org/public/</url>
        </pluginRepository>
    </pluginRepositories>

[siddhantdange]

/request-security-scan

[jenkins-cert-app]

The Jenkins Security Scan discovered 4 finding(s) :mag:.

Please follow the instructions below for every identified issues:

• Implement the recommended fix to address the issue.
• If you think it's a false positive, suppress the warning directly within the code.
• Alternative, you write an explanation here about why you think it's irrelevant. That will require a manual review, leading to a slower process.

After addressing the findings through one of the above methods:

• If all modifications have been made to the code, please initiate a new security scan by triggering the /request-security-scan command.
• If there are any unresolved findings (those not corrected or suppressed), request a review from the Jenkins security team by using the /audit-review command.

---

Stapler: Missing POST/RequirePOST annotation

You can find detailed information about this finding here.

BuildPulseCommand.java#60

``` Potential CSRF vulnerability: If BuildPulseCommand#downloadAndGetCommand connects to user-specified URLs, modifies state, or is expensive to run, it should be annotated with @POST or @RequirePOST ```

Stapler: Missing permission check

You can find detailed information about this finding here.

BuildPulseCommand.java#60

``` Potential missing permission check in BuildPulseCommand#downloadAndGetCommand ```

Jenkins: Plaintext password storage

You can find detailed information about this finding here.

BuildPulseStep.java#28

``` Field should be reviewed whether it stores a password and is serialized to disk: key ```

BuildPulseCommand.java#20

``` Field should be reviewed whether it stores a password and is serialized to disk: key ```

[siddhantdange]

/request-security-scan

[jenkins-cert-app]

The Jenkins Security Scan discovered 3 finding(s) :mag:.

Please follow the instructions below for every identified issues:

• Implement the recommended fix to address the issue.
• If you think it's a false positive, suppress the warning directly within the code.
• Alternative, you write an explanation here about why you think it's irrelevant. That will require a manual review, leading to a slower process.

After addressing the findings through one of the above methods:

• If all modifications have been made to the code, please initiate a new security scan by triggering the /request-security-scan command.
• If there are any unresolved findings (those not corrected or suppressed), request a review from the Jenkins security team by using the /audit-review command.

---

Stapler: Missing POST/RequirePOST annotation

You can find detailed information about this finding here.

BuildPulseCommand.java#63

``` Potential CSRF vulnerability: If BuildPulseCommand#downloadAndGetCommand connects to user-specified URLs, modifies state, or is expensive to run, it should be annotated with @POST or @RequirePOST ```

Stapler: Missing permission check

You can find detailed information about this finding here.

BuildPulseCommand.java#63

``` Potential missing permission check in BuildPulseCommand#downloadAndGetCommand ```

Jenkins: Plaintext password storage

You can find detailed information about this finding here.

BuildPulseStep.java#28

``` Field should be reviewed whether it stores a password and is serialized to disk: keyId ```

[siddhantdange]

/request-security-scan

[jenkins-cert-app]

The Jenkins Security Scan did not find anything dangerous with your plugin, congratulations! :tada:

---

:bulb: The Security team recommends that you are setting up the scan in your repository by following our guide.

[siddhantdange]

@NotMyFault is there any further action needed?

[NotMyFault]

I'll take a look later the week

[siddhantdange]

Hi @NotMyFault - just wanted to check back in here!

[NotMyFault]

I'll take a look when I get a shot to, currently I don't

[NotMyFault]

I took a brief look over your hosting request and have some feedback for you:

• Create a .github folder with the following files and directories: https://github.com/jenkinsci/archetypes/tree/master/common-files/.github
• Create a .mvn folder with the following files: https://github.com/jenkinsci/archetypes/tree/master/common-files/.mvn
• Add a Jenkinsfile with the following content: https://github.com/jenkinsci/archetypes/blob/master/common-files/Jenkinsfile
• https://github.com/buildpulse/buildpulse-jenkins-plugin/blob/7026f859a8eaa8051cf1922f5c04f75fb2e5bf50/pom.xml#L10 is obsolete and should be inlined into index.jelly
• https://github.com/buildpulse/buildpulse-jenkins-plugin/blob/7026f859a8eaa8051cf1922f5c04f75fb2e5bf50/pom.xml#L14 should be at least 2.440.3
• https://github.com/buildpulse/buildpulse-jenkins-plugin/blob/7026f859a8eaa8051cf1922f5c04f75fb2e5bf50/pom.xml#L15-L20is obsolete and can be removed
• https://github.com/buildpulse/buildpulse-jenkins-plugin/blob/7026f859a8eaa8051cf1922f5c04f75fb2e5bf50/pom.xml#L11 should point to jenkinsci
• https://github.com/buildpulse/buildpulse-jenkins-plugin/blob/7026f859a8eaa8051cf1922f5c04f75fb2e5bf50/pom.xml#L25 same like above
• https://github.com/buildpulse/buildpulse-jenkins-plugin/blob/7026f859a8eaa8051cf1922f5c04f75fb2e5bf50/pom.xml#L32 should be <tag>${scmTag}</tag>
• https://github.com/buildpulse/buildpulse-jenkins-plugin/blob/7026f859a8eaa8051cf1922f5c04f75fb2e5bf50/pom.xml#L39-L40 should match 2.440.3, and have the corresponding bom version
• Remove all other dependencies declared, your plugin doesn't depend upon. This looks like bad copy/paste
• https://github.com/buildpulse/buildpulse-jenkins-plugin/blob/7026f859a8eaa8051cf1922f5c04f75fb2e5bf50/pom.xml#L132-L153 is obsolete and can be removed

[mawinter69]

• Please put your classes in a separate package specific to your plugin and not directly in io.jenkins.plugins, e.g. io.jenkins.plugins.buildpulse
• you try to work with remote locations but fail to execute the things remotely. Consider that normally the code you write is executed on the controller and not on the agent. In https://github.com/buildpulse/buildpulse-jenkins-plugin/blob/7026f859a8eaa8051cf1922f5c04f75fb2e5bf50/src/main/java/io/jenkins/plugins/BuildPulseStep.java#L90 you get the workspace but then onyl work with the String that represents the path on the agent. You will need to work always with FilePath and use FilePath.createLauncher() if you want to execute external processes, or you can use FilePath.act() when you want to download a file and write it to the workspace. At the moment everything will fail miserably when the step is running on an agent.
• https://github.com/buildpulse/buildpulse-jenkins-plugin/blob/7026f859a8eaa8051cf1922f5c04f75fb2e5bf50/src/main/java/io/jenkins/plugins/BuildPulseCommand.java#L145 you need to make sure that this code is executed on the agent and not the controller, otherwise you will get the wrong filename when the controller is linux and the agent is mac or windows
• You're suppressing the warning about plaintext storage. And it clearly looks as if https://github.com/buildpulse/buildpulse-jenkins-plugin/blob/7026f859a8eaa8051cf1922f5c04f75fb2e5bf50/src/main/java/io/jenkins/plugins/BuildPulseStep.java#L45 is a secret access key. It might be persisted somewhere in the file system when the pipeline step is executed

[siddhantdange]

Hi @NotMyFault, we've pushed an update that addresses many of these points, do you mind checking the pom.xml again? Also, many of these dependencies we have are for our tests (updated pom.xml to reflect that).

@mawinter69

• Question - for testing, we've spun up a Jenkins locally and ran the plugin within a Jenkinsfile. It seems to work just fine, but I believe you're alluding that the plugin is actually running on the Jenkins master and not agent? I've added a sample Jenkinsfile to the end of this.
• We do not persist this key, and only use it as a part of invoking the uploader binary.

Test Jenkinsfile:

pipeline {
    agent {
        docker {
            image 'ruby:3.0.5' // Specify the Ruby container image you want to use
            args '-u root' // Optionally, specify additional Docker container run arguments
        }
    }

    environment {
        GIT_REPO_URL = 'my-repo-url' // Replace with your repository URL
        GIT_BRANCH = 'main' // Replace with your branch name if needed
    }

    stages {
        stage('Clone Repository') {
            steps {
                script {
                    echo 'Cloning the repository...'
                    scmVars = git branch: "${env.GIT_BRANCH}", url: "${env.GIT_REPO_URL}", credentialsId: 'github'
                    env.GIT_COMMIT = scmVars.GIT_COMMIT
                }
            }
        }

        stage('Setup') {
            steps {
                script {
                    sh "gem install bundler -v '2.3.26'"
                    sh 'bundle install --jobs=4 --retry=3 --path vendor/bundle'
                    sh 'bin/rspec'
                }
            }
        }
    }

    post {
        always {
            buildpulseStep accountId: '', repositoryId: '', junitXMLPaths: '', key: '', secret: '', commitSHA: env.GIT_COMMIT, branch: env.GIT_BRANCH, tags: '', quota: '', coveragePaths: ''
        }
    }
}

[mawinter69]

To test with a real agent install the Mock agent plugin, create an agent and run the pipeline on that agent. I'm not familiar with docker agents in Jenkins but afaik such docker containers run on the controller host and the docker plugin creates a local workspace on the controller and maps this at the same place in the docker container. So the path on the agent and the controller are identical and your plugin finds the file.

You might not persist the key yourself, but Jenkins is writing xml files with step parameters to the file system

[siddhantdange]

@mawinter69 I tested this pipeline with the Jenkinsfile above + my plugin, running on Mock Agent, and it seems to work fine.

From the logs:

Running on [agent](http://localhost:8080/computer/agent/) in /var/jenkins_home/mock-agents/agent/workspace/test-pipeline

Happy to paste full logs if you want to see that.

[mawinter69]

I think it only works because the mock-agent runs on the same physical machine with the same user. So you should setup a real agent that is running on a different machine (or on the same machine as a different user that has no access to /var/jenkins_home)

[siddhantdange]

@mawinter69 Made the necessary changes, tested it on another agent on another node. Mind taking another look?

[mawinter69]

It looks good now from the point that execution on agents works without problems I think. This leaves 2 open points:

• Please put your classes in a separate package specific to your plugin and not directly in io.jenkins.plugins, e.g. io.jenkins.plugins.buildpulse
• you store the secret in the BuildStep object. As mentioned this bears the risk that the secret is stored in plain text in xml files that Jenkins creates for each pipeline run. You already use a credential to be able to globally manage the password which is good. Instead of converting the credentials into a String in the constructor you should only store the credentialId in the BuildPuleStep and move retrieval of the password from the credential to the run method of the Execution (before line 120)

[mawinter69]

There might be one issue with downloading the binary file: https://github.com/buildpulse/buildpulse-jenkins-plugin/blob/d2e6c4cdf257ffdac2b6efe888c15c1e4bac347b/src/main/java/io/jenkins/plugins/BuildPulseCommand.java#L172 is executed on the controller so when the controller is Linux you will always get the unix binary even when the agent is on windows. With FilePath.toComputer() -> Computer.getSystemProperties() you can determine the remote OS.

https://github.com/buildpulse/buildpulse-jenkins-plugin/blob/d2e6c4cdf257ffdac2b6efe888c15c1e4bac347b/src/main/java/io/jenkins/plugins/BuildPulseStep.java#L132 Looks suspicious, why hard coded git url pointing to something not existing?

[siddhantdange]

@mawinter69 Addressed these comments and updated repository. The hardcoded url is for a required argument to the binary - the binary infers repository url from env vars for other CI providers, however, this does not apply here so we've filled in a dummy value (added comment for clarity).

[mawinter69]

Some more things

• the pom.xml has no parent: It should have something like:

  <parent>
  <groupId>org.jenkins-ci.plugins</groupId>
  <artifactId>plugin</artifactId>
  <version>4.86</version>
  <relativePath/>
  </parent>

  The missing pom is the reason why you have to define the dependencies in lines 55-72 and 122-139. You can just remove them. This also allows you to remove the lines 160-182 and 27-28

• https://github.com/buildpulse/buildpulse-jenkins-plugin/blob/b65c7234731072e025c5cb0ac2941e930d746495/pom.xml#L45 the artifactid should match the jenkins version so it should be 2.440.x
• create a file src/main/resources/index.jelly with a description what the plugin does. This is what is shown in the Jenkins UI in the available plugins page
• You might want to add a config.jelly for your BuildStep. With this file you allow your customers to use the snippet generator to generate the correct syntax for the step. Also add corresponding help files for each things that can be configured. This can be of great help when you have the help inline and don't need to first visit e.g. the plugin page or some other place.
• The CODEOWNERS file should be * @jenkinsci/buildpulse-flaky-tests-plugin-developers

[siddhantdange]

@mawinter69 Updated once more - will add the config.jelly as a part of a future update. By the way, I appreciate all the time you're spending here!

Also looks like when clicking on a plugin 'Available Plugins', it redirects me to the plugin documentation (plugin README)

[mawinter69]

A few more things:

• the CODEOWNERS file is .github/CODEOWNERS, so delete the one in the project root and update the one in .github
• the urls in the pom.xml are wrong. You requested above buildpulse-flaky-tests-plugin so the urls in the pom should use that, what most people do is to write https://github.com/jenkinsci/${project.artifactId}-plugin , affected lines: 10, 30, 33 - 35
• The version in line 90 is probably not needed as you get the version via the bom

[siddhantdange]

@mawinter69 updated CODEOWNERS, pom.xml, and renamed the repository to buildpulse-flaky-tests-plugin

[mawinter69]

The urls in line 33-35 are missing a / after the org

[siddhantdange]

@NotMyFault @mawinter69 updated

[github-actions[bot]]

Hello from your friendly Jenkins Hosting Checker

It looks like you have everything in order for your hosting request. A member of the Jenkins hosting team will check over things that I am not able to check(code review, README content, etc) and process the request as quickly as possible. Thank you for your patience.

Hosting team members can host this request with /hosting host

[siddhantdange]

@mawinter69 is this approved?

[NotMyFault]

/request-security-scan

[jenkins-cert-app]

The Jenkins Security Scan discovered 2 finding(s) :mag:.

Please follow the instructions below for every identified issues:

• Implement the recommended fix to address the issue.
• If you think it's a false positive, suppress the warning directly within the code.
• Alternative, you write an explanation here about why you think it's irrelevant. That will require a manual review, leading to a slower process.

After addressing the findings through one of the above methods:

• If all modifications have been made to the code, please initiate a new security scan by triggering the /request-security-scan command.
• If there are any unresolved findings (those not corrected or suppressed), request a review from the Jenkins security team by using the /audit-review command.

---

Jenkins: Plaintext password storage

You can find detailed information about this finding here.

BuildPulseStep.java#42

``` Field should be reviewed whether it stores a password and is serialized to disk: key ```

BuildPulseCommand.java#24

``` Field should be reviewed whether it stores a password and is serialized to disk: keyId ```

[siddhantdange]

/request-security-scan

[jenkins-cert-app]

The Jenkins Security Scan discovered 1 finding(s) :mag:.

Please follow the instructions below for every identified issues:

• Implement the recommended fix to address the issue.
• If you think it's a false positive, suppress the warning directly within the code.
• Alternative, you write an explanation here about why you think it's irrelevant. That will require a manual review, leading to a slower process.

After addressing the findings through one of the above methods:

• If all modifications have been made to the code, please initiate a new security scan by triggering the /request-security-scan command.
• If there are any unresolved findings (those not corrected or suppressed), request a review from the Jenkins security team by using the /audit-review command.

---

Jenkins: Plaintext password storage

You can find detailed information about this finding here.

BuildPulseCommand.java#24

``` Field should be reviewed whether it stores a password and is serialized to disk: keyId ```

[siddhantdange]

/request-security-scan

[jenkins-cert-app]

The Jenkins Security Scan did not find anything dangerous with your plugin, congratulations! :tada:

---

:bulb: The Security team recommends that you are setting up the scan in your repository by following our guide.

[siddhantdange]

@NotMyFault keyId is never serialized to disk

[siddhantdange]

@NotMyFault @mawinter69 Just wanted to bump this

[timja]

Hi apologies for the delay.

---

1. function name should not include step: https://github.com/buildpulse/buildpulse-flaky-tests-plugin/blob/main/src/main/java/io/jenkins/plugins/buildpulse/BuildPulseStep.java#L195
2. its not recommended to include developers in your pom.xml as from experience this is never maintained, information from the plugin site will come from this repository in respect to maintainers, https://github.com/buildpulse/buildpulse-flaky-tests-plugin/blob/main/pom.xml#L18-L24
3. your plugin doesn't look to be properly setup in respect to version, you should have revision and changelist properties defined, see https://github.com/jenkinsci/archetypes/blob/master/empty-plugin/src/main/resources/archetype-resources/pom.xml#L35-L36

[mawinter69]

• Please do not use System.out.println here https://github.com/buildpulse/buildpulse-flaky-tests-plugin/blob/6046891a9fbcc70e164a7a01cf009cb9b50a401c/src/main/java/io/jenkins/plugins/buildpulse/BuildPulseStep.java#L190