CD proposal

[ir4y]

Set up automatic deployment. Now the deployment process is manual, so once we apply any changes to the source code we have to ask @KyleOps to perform the system rollout. After this task is completed the continuous delivery pipeline should be set up.

Below I provide my vision of a step-by-step guide on how to accomplish the goal and responsible people

• [x] Setup docker image publishing to GitHub registry https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry https://docs.github.com/en/packages/managing-github-packages-using-github-actions-workflows/publishing-and-installing-a-package-with-github-actions @projkov
• ~~Remove ingress configuration from https://github.com/hl7au/au-fhir-core-inferno/tree/master/infra/aws-impl Ingress is used by other services like SMILE CDR and ontoserver so it should be managed on the higher level of infrastructure confirmation~~
• [x] Create an AWS service account with access to @KyleOps
• terraform s3 bucket https://github.com/hl7au/au-fhir-core-inferno/blob/master/infra/aws-impl/provider.tf#L29C3-L34C4
• inferno namespace of EKS fhir-k8s-dev cluster
• [x] Add AWS service account key and secret to the repository secret @brettesler-ext
• [x] Add Github Action step to apply the configuration on each push to the main branch @ir4y

[jgsuess]

I think there is not enough information in this ticket for me to understand what is actually intended in terms of eventual deployment.

However, I see that this is a deployment topic, and this is generally poorly served by Github, as it has limited integration of registries, in particular for Helm and Terraform and very little support for observability or GitOps. For this reason, I suggest that we develop the automated deployment in Gitlab, which provides these features and allows us to concentrate assets in a single location and increase the ability to track from issue to deployment and monitor.

It will also simplify credential management and by that increase safety, as Gitlab uses tokens scoped to the lifetime of the pipeline, reducing leaks. I am raising this last point as we should build the security properties for business-as-usual into deployment at this time. They tend to be challenging to add later.

Note that disruption would be small, as this will not affect current product deliverables and associated pipelines, which arrive as OCI containers. All CI/CD infrastructure for these products will remain as is.

Here are the features in Gitlab I am referring to, that I believe are not available in Github:

• https://docs.gitlab.com/ee/user/packages/container_registry/
• https://docs.gitlab.com/ee/user/packages/helm_repository/
• https://docs.gitlab.com/ee/user/infrastructure/iac/
• https://docs.gitlab.com/ee/integration/vault.html
• https://docs.gitlab.com/ee/user/clusters/agent/gitops.html
• https://docs.gitlab.com/ee/user/clusters/create/

I have experience in applying these features.

[ir4y]

Hi @jgsuess thank you for your feedback. All your ideas make perfect sense. To be honest I also prefer Gitlab and use it a lot for other projects. However, I am not sure how we can leverage it for the current project since it is under sparked and hl7au and driven by the community. Do you envision creating a fork of this repo on gitlab.com? If we go this way, @brettesler-ext could you please clarify the process, do we need community approval for this kind of action? Another question is how we are going to maintain the Terraform project for the AWS cloud infrastructure including Ontoserver and SmileCDR configuration. Should it be open-source and publicly available to all SPARKED participants? I tend to think, that it should be. Happy to continue this conversation letter today on the call.

[jgsuess]

I would like to avoid making changes to the source code locations as they currently are. They are sanctified and organised. I think my approach would be to use a downstream repository and add the respective implementation files for DevOps there. This means that the local repository would remain unchanged, and no DevOps artifacts would enter here. Alternatively, and preferred, I would have this repository publish binary artifacts and consume them on the Gitlab side. This has the advantage that versioning and packaging are required, providing a clean process boundary.

[ir4y]

@KyleOps thank you for implementing the action for k8s deployment. Now we have a CI/CD pipeline up and running.

Once the issue with public access to GitHub package https://github.com/hl7au/au-fhir-core-inferno/pkgs/container/au-fhir-core-inferno is resolved I would like to add the following list of enhancements:

• [ ] Adjust CI to follow https://github.com/hl7au/au-fhir-core-inferno?tab=readme-ov-file#release-management. The pipeline should be triggered when a new GitHub release is created

  on:
  release:
  types: [created]

• [ ] The release tag should be passed to Terraform to replace the default tag in the module variable https://github.com/hl7au/au-fhir-core-inferno/blob/master/infra/modules/au_fhir_inferno/input.tf#L17