Greenfield Template - DevOps Quickstart

Overview

The nr-epd-digital-services is a fully functional set of pipeline workflows and a starter application stack intended to help Agile DevOps teams hit the ground running. Currently supports OpenShift with plans for AWS (Amazon Web Services). Pipelines are run using GitHub Actions.

Features:

• Pull Request-based pipeline
• Sandboxed development deployments
• Gated production deployments
• Container publishing (ghcr.io) and importing (OpenShift)
• Security, vulnerability, infrastructure and container scan tools
• Automatic dependency patching with Pull Requests
• Enforced code reviews and pipeline checks
• Templates and setup documentation
• Starter TypeScript application stack

This project is in active development. Please visit our issues page to view or request features.

Workflow 1 of 3: PR Open

Workflow 2 of 3: PR Close

Workflow 3 of 3: Main Merge

Deployments

Out-of-the-box, sandboxed, pull request-based development deployments allowing for multiple developers to work on and see their features at once.

Deployment to production is gatekept using GitHub environments, requiring sign off from code maintainers.

Deployment to test, staging or pre-prod (pick a name!) is currently planned to be transitory, allowing access to gatekept data, but only stopping before production deployment if a failure occurs. (work in progress)

Successful deployments are linked in Pull Request comments.

Builds

Builds are handled by Docker Actions and published to the GitHub Container Registry (ghcr.io). This allows for publicly accessible builds that can be consumed by OpenShift, Amazon Web Services or any other container service.

Testing

Unit tests are run in jest, but other test frameworks can always be installed. SonarCube should be configured to pick up sarif files and provide coverage reports.

Code Quality

Code quality is reporting are performed by:

• SonarCloud
• CodeQL

Sonar reports are provided as Pull Request comments.

Code Coverage

Code coverage is generated by any included tests. Results are provided as Pull Request comments.

Security Scanning

Dependency, container and vulnerability scanning is performed by:

• Trivy
• Snyk

Penetration Testing

Penetration testing and reporting is performed by:

• OWASP ZAP

Dependency Management

Dependency scanning, patching and updating by PR is performed by:

• Snyk

Higher-Level Environments

Higher-level environments come after DEV deployments and are usually called any of TEST, STAGING, PRE-PROD or PROD. Since data and token access is more frequently sensitive access must be controlled by only allowing access from a merge to the main branch.

Higher-Level Environment Gatekeeping

Optionally, higher-level deployments can be prevented until manually approved.

Workflows

1: Pull Request Opened/Modified

This workflow is triggered when a Pull Request to the main branch is created or modified. Each development deployment is separate, using its own stack. This avoids collisions between development environments and provides isolation for testing and experimentation. Pipeline steps are enforced, preventing merge of failing code.

The workflow, located here, includes:

• Pull Request-based ephemeral, sandboxed environments
• Docker/Podman container building
• Build caching to save time and bandwidth
• GitHub Container Registry image publishing
• RedHat OpenShift deployment, with other options under consideration
• Jest JavaScript testing enforced in-pipeline
• SonarCloud static analysis test coverage reporting

Triggers are used to determine whether images need to be built or previous ones consumed. Partial or full skips, like when limited to documentation, are shown below.

When a PR is merged, the message comment below is added. Despite showing on this pull request it is actually handled by the next pipeline.

2: Pull Request Close Pipeline

The workflow, located here, fires when a pull request is closed.

• ghcr.io cleanup of dev images over 14 days-old
• OpenShift dev artifact removal

When a pull request is merged to main, one additional job is run. This promotes the new images to the TEST environment.

• Image promotion to higher-level environments

If this closure was triggered by a merge to the main branch it will trigger the following workflow.

3: Pull Request Main Merge Pipeline

The workflow, located here, includes:

• GitHub CodeQL semantic code analysis and vulerability scanning
• OWASP ZAP Zed Attack Proxy web app penetration testing
• SonarCloud static analysis for continuous code quality and security scanning
• Snyk vulnerability scanning and PR-based dependency patching
• Tryvy repository and base image scanning
• Higher-level deployments (e.g. TEST, STAGING, PRE-PROD, PROD)
• Publishing of production images to the GitHub Container Registry (ghcr.io)

Starter Application

The starter stack includes a frontend, backend and postgres database. The frontend and backend are buld with NestJS. They currently do very little, but provide placeholders for more functional products. See the backend and frontend folders for source, including Dockerfiles.

Features:

• TypeScript strong-typing for JavaScript
• NestJS frontend and backend
• ESLint linting enforced on code staging (currently disabled)
• Postgres database

Local development can be supported using Docker Compose. Please be aware that Podman and Podman Compose work as drop-in replacements for the Docker counterparts.

docker-compose up -d

Getting Started

Initial setup is intended to take four hours or less. This depends greatly on intended complexity, features selected/excluded and outside cooperation.

Please read our setup guide for more information.

Example APIs, UIs and Metabase/Oracle Templates

Templates for APIs, UIs and Metabase/Oracle can be used to kickstart or extend projects. Please visit our collaborators' NR Architecture Templates repository for more information.