CI/CD Evaluation

[cvarjao]

Criteria

• Easy / Fast / Repeatable (1 - 10)
• Location (On-Premises vs SaaS)
• CI, CD, or both
• Extendable
• Cloud Native / HA
• OPs friendly
• Demo
• Enterprise Visibility (e..g: Stats, Dashboard)
• License
• Price Model

Use Cases

• GoCD - Shea
• Argo - Shea
• GitHub Actions - Shea
• BCDK/Pipeline-CLI (OnPremises) - Clecio
• AzureDevOps - Jason Leach
• ShipIt - Alec
• Samson - Alec
• concourse-ci - Leo Lou
• CircleCI - Alec

10 Minutes Demo/Show

Due Date: July 2nd

[stewartshea]

Title: GitHub Actions & OpenShift

Criteria

Easy / Fast / Repeatable (1 - 10)

• Development Team: 8/10
  • Simple UI -
  • Easy to start, no infrastructure to deploy
  • Ultimately requires custom images to handle developer requirements
  • Lots of flexibility on trigger options
  • Fully capable of a codified deployment

Location (On-Premises vs SaaS)

• SaaS
• Currently still in Beta

CI, CD, or both

• Whatever you want it to be.....
• Seems to lean more on the CI side of things
• Doesn't appear to have any CD type primitives

Extendable

• Container based extensions
• Closed source, so modifications to the existing code is not possible

Cloud Native / HA

• Not really kubernetes native, but is

OPs friendly

• Nothing to set up in OpenShift aside from a service account
• Builds can happen outside of OpenShift
• Likely requires the building and management of various agent containers

Demo

Enterprise Visibility (e..g: Stats, Dashboard)

• Tied to each repository and it appears to lack any multi-project visibility

License

• Commercial? Currently has no cost.

Price Model

• Still in Beta with many limitations
• Limitations
  • Each workflow can run for a maximum of 58 minutes including queuing and execution.
  • Each workflow can run a maximum of 100 actions
  • A repository can run a maximum of 2 workflows concurrently
  • A task performed within an action cannot trigger other actions.

Resources

• https://help.github.com/en/articles/about-github-actions

• https://scotch.io/bar-talk/introducing-github-actions

• https://blog.kontena.io/deploying-to-kubernetes-from-github-actions/

• https://developer.github.com/actions/

• https://developer.github.com/actions/creating-github-actions/accessing-the-runtime-environment/

• Sample Pipeline

  
  workflow "Build and Deploy to OpenShift" {
  on = "push"
  resolves = ["Rollout Deployment"]
  }

action "Build Image" { uses = "actions/docker/tag@86ff551d26008267bb89ac11198ba7f1d807b699" runs = "docker build -t beer-selector ." }

action "Tag image" { uses = "actions/docker/cli@86ff551d26008267bb89ac11198ba7f1d807b699" runs = "docker tag beer-selector docker-registry.lab.pathfinder.gov.bc.ca/shea-argo/beer-selector:latest" needs = ["Build Image"] }

action "Auth to OpenShift registry" { uses = "actions/docker/login@master" needs = ["Tag image"] secrets = ["DOCKER_PASSWORD", "DOCKER_USERNAME", "DOCKER_REGISTRY_URL"] }

action "Push image" { uses = "actions/docker/cli@86ff551d26008267bb89ac11198ba7f1d807b699" runs = "docker push docker-registry.lab.pathfinder.gov.bc.ca/shea-argo/beer-selector:latest" needs = ["Auth to OpenShift registry"] }

action "OpenShift Login" { uses = "stewartshea/jenkins2-with-docker@master" secrets = ["DOCKER_PASSWORD", "OPENSHIFT_URL"]
needs = ["Push image"]

}

action "Rollout Deployment" { uses = "stewartshea/jenkins2-with-docker@master" secrets = ["DOCKER_PASSWORD", "OPENSHIFT_URL"] runs = "oc --config=/github/home/workspace/kube rollout latest beer-selector" needs = ["OpenShift Login"]

}

[stewartshea]

Title: GoCD and OpenShift

Criteria

Easy / Fast / Repeatable (1 - 10)

• Platform Services: 6/10

  • Requires some initial upfront effort to create a reusable template and integrate with OpenShift OAuth
  • Requires maintenance of the image and custom images that will be used

• Development Team: 8/10

  • Easily deployed from template; ephemeral and persistent options available
  • Less plugins / complexity to manage
  • Ultimately requires custom images to handle developer requirements
  • Seems to poll repos rather than use webhooks, but can extend this
  • Fully capable of a codified deployment

Location (On-Premises vs SaaS)

• On-Premises

CI, CD, or both

• Closer to CI than CD
• Similar to Jenkins in many of the functional ways

Extendable

• Uses custom docker images for extensions
• Plugins can be added easily as jar files (either in init containers, in the entrypoint, or on )

Cloud Native / HA

• Not really cloud native, seems to have a master server and database
• Master / Slave architecture for HA
• No native kubernetes integration aside from the elasic agent config which works similar to Jenkins + OpenShift Pods but requires too many privileges. Using agents with auto-registration works just as well.

OPs friendly

• Could run on less resources than Jenkins but ultimately this still has Java in the base and isn't considered light. Probably lighter than Jenkins.
• Requires upkeep from the platform-services team in terms of templates

Demo

Enterprise Visibility (e..g: Stats, Dashboard)

• Can be run as a single tool or per-team tool
• Does have an analytics plugin (not tested or shown in the demo)

License

• Apache-2.0

Price Model

• FOSS

Resources

• GoCD Documentation
• Deployment on OpenShift Blog
• Sample pipeline

  format_version: 3
  pipelines:
  test:
  group: defaultGroup
  label_template: ${COUNT}
  lock_behavior: none
  display_order: -1
  materials:
    git:
      git: https://github.com/BCDevOps/platform-services
      shallow_clone: true
      auto_update: true
      branch: status-page
  stages:
  - uninstall-dev:
      fetch_materials: true
      keep_artifacts: false
      clean_workspace: false
      approval:
        type: success
      jobs:
        cleanDev:
          # elastic_profile_id: standard-agent
          timeout: 0
          tasks:            
          - exec:
              command: /bin/bash
              arguments: 
                - -c 
                - apps/statuspage/.gocd/uninstall.sh
              run_if: passed
  - install-dev:
      fetch_materials: true
      keep_artifacts: false
      clean_workspace: false
      approval:
        type: success
      jobs:
        deployDev:
          # elastic_profile_id: standard-agent
          timeout: 0
          tasks:            
          - exec:
              command: /bin/bash
              arguments: 
                - -c 
                - apps/statuspage/.gocd/install.sh
              run_if: passed
  - configure-dev:
      fetch_materials: true
      keep_artifacts: false
      clean_workspace: false
      approval:
        type: success
      jobs:
        deployDev:
          # elastic_profile_id: standard-agent
          timeout: 0
          tasks:            
          - exec:
              command: /bin/bash
              arguments: 
                - -c 
                - apps/statuspage/.gocd/configure.sh
              run_if: passed

[stewartshea]

Title: Argo Project & OpenShift

Criteria

Easy / Fast / Repeatable (1 - 10)

• 10/10 for installation
• 10/10 for "Project sync setup"
• Likely a 5/10 for developing an appropriate multi-tenant operational model

Location (On-Premises vs SaaS)

• On-Premises

CI, CD, or both

• CD only
• Argo workflows or some other tool like Jenkins is required for CI

Extendable

• Argo workflow follows the same pattern as "bring your own container" model for CI steps. In this way it is very extendable
• It is fully Open Source and accepts pull requests

Cloud Native / HA

• Possibly better to say "kube-native"
• Uses CRDs and GitOps style approach to sync manifests from a repo into a running kubernetes based environment

OPs friendly

• Very lightweight in terms of resources
• Must be run as a "platform service" and cannot be deployed in the traditional multi-tenant model. Requires cluster-wide view access and as a result must be run by a central team.
  • "The rules of the argocd-manager-role role can be modified such that it only has create, update, patch, delete privileges to a limited set of namespaces, groups, kinds. However get, list, watch privileges are required at the cluster-scope for Argo CD to function."

Demo

Enterprise Visibility (e..g: Stats, Dashboard)

• UI is very minimal; cluster wide reports likely need to be pulled from Custom Resource data embedded in each cluster (which shouldn't be difficult) and visualized with 3rd party tools

License

• Apache License 2.0

Price Model

• FOSS

Resources

• https://argoproj.github.io/argo-cd/
• https://medium.com/@doronsegal/workflow-using-argo-kubernetes-6b45ef3f1614
• https://argoproj.github.io/docs/argo/demo.html
• https://github.com/argoproj/argo/tree/master/examples

[jleach]

Title: Azure DevOps & OpenShift

Criteria

Easy / Fast / Repeatable (1 - 10)

• 8/10
• SaaS so there is nothing to setup unless you want your own agent;
• Agents run in MS's infra structure so its fast and repeatable.

Location (On-Premises vs SaaS)

• SaaS

CI, CD, or both

• Both
• Use Pipelines for CI
• Use releases for managed or autumated CD.

Extendable

• Build a custom agent and run on-prem with any tool chain you like.

Cloud Native / HA

• N/A

OPs friendly

• Run on MS' infrastructure for quick / fast builds; idea for doing verification and validation.
• Run on-prem when its time to build an image and release it.
• Because its SaaS there is nothing for OPs to do.

Demo

Presentation video here.

Enterprise Visibility (eg: Stats, Dashboard)

• Minimal.
• DevOps teams could just add a step to report metrics to whatever we have to collect them.

License

• N/A (SaaS)

Price Model

• FOSS;
• As needed could be billed to our enterprise account.

Resources

• Azure DevOps

[wenzowski]

Title: GitHub Actions & OpenShift

Criteria

Easy / Fast / Repeatable (1 - 10)

• Development Team: 9/10

  • Simple UI -
  • Easy to start
  • Rails (buit via s2i) is easy to maintain

Location (On-Premises vs SaaS)

• On-Premises

CI, CD, or both

• CD only
• Reads CI and other status checks from Github

Extendable

• The "shipit-engine" gem is added to a custom Rails app
• Upstream objects can be redefined/monkeypatched in downstream app to support business-specific requirements
• Shopify takes pull requests on upstream for generic bug fixes and features

Cloud Native / HA

• Required two backing stores: Postgres (HA via Patroni) and Redis (HA via ?)

OPs friendly

• Builds can happen outside of OpenShift or inside as long as they report Github Status
• Deploys happen from inside OpenShift

Demo

• https://cas-shipit.pathfinder.gov.bc.ca/

Enterprise Visibility (e..g: Stats, Dashboard)

• Deployment definitions based on a (repo, branch, environment) triple
• Dashboard shows state and logs of all deploys
• One dashboard per tenant
• API enables enterprise aggregation

License

• MIT License; Copyright (c) 2015 Shopify

Price Model

• Free, as in beer.

Resources

• https://github.com/bcgov/cas-shipit

[ll911]

Title: concourse-ci and K8s/OpenShift

Criteria

Easy / Fast / Repeatable (1 - 10)

• Platform Services: ?/10

  • works great with Helm and K8s, not so much on openshift 3.11 when CRD is disabled,
  • CRD is required, maybe works better with openshift 4.2+

• Development Team: 10/10

  • Configuration As Code
  • Cool Visualization
  • CI Under Source Control
  • Reproducible, Debuggable Builds
  • Rapid Local Iteration
  • Bring Your Own Integrations

Location (On-Premises vs SaaS)

• On-Premises

CI, CD, or both

• both, Built on the simple mechanics of resources, tasks, and jobs

Extendable

• single binary
• easy to bring your own into your pipeline. Resource types are implemented as container images containing scripts - using the docker-image resource type, they can be fetched from a Docker registry.

Cloud Native / HA

• Possibly better to say "kube-native"
• Uses CRDs and Pipeline style approach
• running kubernetes based environment

OPs friendly

• lighter than Jenkins.
• Requires upkeep from the platform-services team in terms of templates

Demo

Enterprise Visibility (e..g: Stats, Dashboard)

• Can be run as a single tool or per-team tool

License

• Apache-2.0

Price Model

• FOSS

Resources

• concourse-ci Documentation
• hello world deployment
• Sample pipeline