Jenkins Blue Ocean CI/CD pipeline - similar to GitGraph

[HariSekhon]

Proposal

Jenkins CI/CD pipeline graph - similar to what GitGraph currently does except:

1. different icons
2. Jenkins parallel stages need to be vertically aligned, whereas GitGraph has commits on different branches before/after each other

Use Cases

Demonstrating popular CI/CD workflows and strategies - Jenkins Blue Ocean UI pipeline diagram is currently the gold standard for visualizing this and most widely use - so I want it to look like this.

I would like to be able to create Blue Ocean diagrams in code without having to create the actual pipelines in Jenkins and then take screenshots. That is so last decade.

Please let this become another cool Diagram-as-Code type 🙏🏼

Screenshots

pipeline in progress, some steps failed but ignored eg. code scan

pipeline in progress, some steps skipped:

pipelines completed, some steps failed, other steps skipped:

no changes detected, so Approval not asked for an Apply skipped too:

pipeline Approval stage not approved or timed out, so terraform Apply stage skipped:

More screenshots can be found here along with the Jenkinsfile and Groovy Shared Library code behind all these pipelines:

https://github.com/HariSekhon/Jenkins

More screenshots can be found in my Diagrams-as-Code repo which uses MermaidJS among other diagram languages:

https://github.com/HariSekhon/Diagrams-as-Code

https://github.com/HariSekhon/Diagrams-as-Code/tree/master/screenshots

Code Sample

Doesn't match exactly to the above screenshots, but just to give a flavour of how to handle different pipeline stages.

Any stages without a state should be success or not run yet if they are proceeded a stage set to running that are not parallel siblings:

JenkinsGraph
    stage id: "Setup"
    parallel id: "Downloads"
        stage id: "Download ArgoCD"
        stage id: "Download Grype"
        stage id: "Download Kustomize"
        stage id: "Download Trivy"
    endparallel  # or could use the shorter 'merge' keyword?
    parallel id: "Code Scanning" 
        stage id: "Grype" state: "failed"
        stage id: "Trivy" state: "failed"
    endparallel
    stage id: "GCP Auth"
    stage id: "Wait for GCP CloudBuild" 
    stage id: "Add GCR Docker Image Tags"
    stage id: "GCP Docker Auth"
    parallel id: "Container Scanning"
        stage id: "Grype" state: "failed"
        stage id: "Trivy" state: "failed"
    endparallel
    stage id: "ArgoCD Deploy" state: "aborted"
    pipeline state: "failed"  # could probably be set automatically based on prior stages, or allow explicit override as sometimes stage failures are ignored and pipeline still set to `success`, such as for code scanning tools which often show errors and are ignored

The same pipeline in progress part way through, where some of the nodes are hollow not executed yet and the currently executing ones are blue glowing orbs:

JenkinsGraph
    stage id: "Setup"
    parallel id: "Downloads"
        stage id: "Download ArgoCD"
        stage id: "Download Grype"
        stage id: "Download Kustomize"
        stage id: "Download Trivy"
    endparallel  # or could use the shorter 'merge' keyword?
    parallel id: "Code Scanning" 
        stage id: "Grype" state: "running"
        stage id: "Trivy" state: "running"
    endparallel
    stage id: "GCP Auth"
    stage id: "Wait for GCP CloudBuild" 
    stage id: "Add GCR Docker Image Tags"
    stage id: "GCP Docker Auth"
    parallel id: "Container Scanning"
        stage id: "Grype" 
        stage id: "Trivy" 
    endparallel
    stage id: "ArgoCD Deploy"

[sidharthv96]

Interested folks can start off by modifying the gitGraph diagram.

Some notes on the syntax.

1. Would need a more generic name, which doesn't include "Jenkins".
2. Can avoid endparallel and use intendation like mindmap does to differentiate stages.
3. stage seems redundant, what if we avoid stage id: completely, and just use the labels?

CIPipeline
    "Setup"
    parallel "Downloads"
        "Download ArgoCD"
        "Download Grype"
        "Download Kustomize"
        "Download Trivy"
    parallel "Code Scanning" 
        "Grype" state: "running"
        "Trivy" state: "running"
    "GCP Auth"
    "Wait for GCP CloudBuild" 
    "Add GCR Docker Image Tags"
    "GCP Docker Auth"
    parallel "Container Scanning"
        "Grype" state: "failed"
        "Trivy" state: "failed"
    "ArgoCD Deploy" state: "aborted"
    pipeline state: "failed" 

Or even more concise

CIPipeline
    Setup
    Downloads
        "Download ArgoCD ❯ unicode"
        Download Grype
        Download Kustomize
        Download Trivy
    "Code Scanning ⚠"
        Grype
        Trivy
    GCP Auth
    Wait for GCP CloudBuild 
    Add GCR Docker Image Tags
    GCP Docker Auth
    Container Scanning
        Grype state: failed
        Trivy state: failed
    ArgoCD Deploy state: aborted
    pipeline state: failed 

[HariSekhon]

I've updated the original post with some real-world screenshots showing some real variations including:

• pipeline success (green)
• pipeline in progress (blue)
• pipeline aborted (grey)
• stages linear left to right
• stages parallel stage (stacked vertically via a fork and merge)
• stage success (green)
• stage failure (red)
• stage skipped (hollow - and notice the lines swerve around it unless in a parallel block)
• stage in progress (blue flashing)
• stage circled is just the one selected by your mouse to read the output further down off screen

[kamazee]

When trying to use existing gitGraph for visualizing the required state of a Jenkins pipeline, I noticed more differences that should be kept in mind:

1. Jenkins' visualization creates forks between stages; however gitGraph forks on commits.
2. I didn't find how to do an "octopus merge" in mermaid in order to merge few branches at once; while in Git it might be not that widespread, in CI pipelines it's every here and there.

Compare this image from the initial post in this issue:

and this (I tried to reproduce few steps):

%% gitGraph
%%     branch " "
%%     commit id: "Build bingo-app-server artifact"
%%     commit id: "Build bingo-app-regulator artifact"
%%     commit id: "Deploy configuration"
%%     branch "  "
%%     checkout " "
%%     commit id: "Run migrations"
%%     checkout "  "
%%     commit id: "Check/deploy kafka topics"
%%     checkout " "
%%     merge "  "

%%{init: {'gitGraph': {'showBranches': false}} }%%
gitGraph
    commit id: "Start"
    commit id: "Setup"
    branch l1
    branch l2
    branch l3
    branch l4
    branch l5
    checkout main
    commit id: "Download ArgoCD"
    checkout l1
    commit id: "Dowload Clairctl"
    checkout l2
    commit id: "Download Grype"
    checkout l3
    commit id: "Download Kustomize"
    checkout l4
    commit id: "Download Trivy"
    checkout l5
    commit id: "Install NodeJS"
    checkout main
    merge l1
    merge l2
    merge l3
    merge l4
    merge l5
    commit id: "Code Scanning"
    commit id: "GCP Auth"
    branch l6
    commit id: "GCP Docker Auth"
    commit id: "GCP CloudBuild"
    checkout main
    merge l6
    commit id: "Wait for GCR Docker Images"
    branch l7
    commit id: "Docker Pull"
    checkout main
    commit id: "Add GCR Docker Image Tags"
    merge l7
    branch l8
    commit id: "Container Image Scanning"
    checkout main
    merge l8
    commit id: "ArgoCD Deploy"
    commit id: "End"

A rendered image for the reference, in case `gitGraph` rendering changes over time