Jenkins associate input parameter with git ref

[saada]

Search before asking

• [X] I had searched in the issues and found no similar feature requirement.

Use case

The current Jenkins connection assumes that the repo clone by the Jenkins job is the same as the business logic repo that we want Dota metrics from.

In this use case, we want to support separated Jenkins repositories that services leverage to build their app.

Let’s assume we have two git repos: “webapp” and “jenkinsfiles”. If I create a devlake project for the webapp, the GitHub connection in that project points to repo1 (webapp) while the Jenkins connection points to repo2 (jenkinsfiles). When Jenkins runs a new deployment job, it will clone a git SHA from repo2 (jenkinsfiles) not repo1 (webapp). The only link Jenkins knows about back to repo1 is a Jenkins input parameter called “TAG” that gets passed to the job. TAG is a git ref in repo1.

Note: putting the GitHub and Jenkins connections in same project is not enough today. So a deployment is performed with a "devops-setting commit" in repo 2(jenkinsfiles) and a "business-code commit" in both repo 1(webapp), and we should use the latter commit to measure DORA.

If we could specify in the Jenkins association which input parameter defines the git tag/sha, then we would get correct metrics.

Description

If we could specify in the Jenkins association which input parameter defines the git tag/sha, then we would get correct metrics

Related issues

No response

Are you willing to submit a PR?

• [X] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[d4x1]

@saada Could you show me how to pass tags to jenkins' job and where the tag parameter will appear?

[saada]

Given a jenkinsfile like this

pipeline {
    agent any
    parameters {
        string(name: 'RELEASE_TAG', defaultValue: '', description: 'Enter git ref for web app repo')
        choice(name: 'CLUSTER', choices: ['qa', 'prod'], description: 'Pick cluster')
    }
    stages {
        stage('deploy to cluster') {
            steps {
                echo "Deploying ${params.RELEASE_TAG} TO ${params.CLUSTER}"

               // ...
            }
        }
    }
}

The parameter in this case is called "RELEASE_TAG". But in another job it could be called TAG or GIT_REF.

The result should look something like this image but with the specified parameters above

[d4x1]

@saada Thank you for your reply, it's not diffcult to support this feature.

My questions are: when should we use repo2(jenkins file)'s commit sha, when shoud we use repo1(parameter in jenkins file)'s commit sha ? and how to config it(when ? where ?) ?

Do you have any ideas?

[saada]

Ideally it would be configured via a new feature exposed here in the Jenkins settings.

If there's a new field in here called Jenkins parameter to use when calculating DORA metrics: where I can enter in there parameter names such as TAG or GIT_REF or GIT_SHA or REPO_TAG. It's important to support various git references here. It might be a tag, sha, branch, etc.

By default we can assume the jenkins sha wiill match the github connection sha. Unless this field is populated, then we override that default with the custom parameter value instead.

[d4x1]

@saada User can custom parameter name doubtly. But is there is a git ref/tag/branch, we cannot fetch the commit sha behind the ref/tag/branch, and the commit_sha field in cicd_pipelint_commits will be v0.x.x or release-xxx(not the commit sha).

Is this acceptable and expectale?

[saada]

@d4x1 that would work. Thank you. Might want to clarify in the UI as well that SHAs won't work.

[d4x1]

@d4x1 that would work. Thank you. Might want to clarify in the UI as well that SHAs won't work.

Got it, it will be completed next week. (I am busy with my personal matters)

[saada]

Thank you so much, no rush! Best of luck

[Startrekzky]

Hi @saada , we need more time to re-think the solution. The previous solution we came up with could not solve your problem. Sorry for the inconvenience.

[klesh]

Hi, @saada , I'm proposing a solution for your use case, would you like to see if it can solve your problem or if anything is missing? Thanks.

Sorry, I just realized that this has nothing to do with the PR, but, we may solve your problem by using the webhook in a similar manner:

pipeline {
    agent any
    parameters {
        string(name: 'RELEASE_TAG', defaultValue: '', description: 'Enter git ref for web app repo')
        choice(name: 'CLUSTER', choices: ['qa', 'prod'], description: 'Pick cluster')
    }
    stages {
        stage('deploy to cluster') {
            steps {
                echo "Deploying ${params.RELEASE_TAG} TO ${params.CLUSTER}"
                sh 'somehow get the commit sha based on the release tag and then do curl http://<devlake domain>/plugins/webhook/1/deployments -X 'POST' -d ...'
            }
        }
    }
}

[d4x1]

I leave this comment mainly to record why I don't code to completed this feature directly. Devlake can fetch build parameters, and regard some of them (such as GIT_SHA) as commit_sha in cicd_pipelint_commits, but this can lead to DORA metrics be calculated incorrectly after an diccussion with @klesh. So I think the best way is to use webhook so far.

[saada]

Webhook solution is fine by me. Can we document a tutorial on how to implement this solution? Thank you

[d4x1]

I think @Startrekzky is working on this.

[github-actions[bot]]

This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.

[github-actions[bot]]

This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.

[github-actions[bot]]

This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.

[github-actions[bot]]

This issue has been closed because it has been inactive for a long time. You can reopen it if you encounter the similar problem in the future.

[github-actions[bot]]

This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.

[github-actions[bot]]

This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.

[github-actions[bot]]

This issue has been closed because it has been inactive for a long time. You can reopen it if you encounter the similar problem in the future.