Health-Informatics-UoN / Carrot-Mapper

Carrot: Convenient And Reusable Rapid Omop Transformer.
https://carrot4omop.ac.uk
MIT License
12 stars 3 forks source link

Combine deployment workflows into environments #645

Closed AndyRae closed 3 months ago

AndyRae commented 4 months ago

Changes

Combines the existing deployment workflows from devel/test/prod, into Github environments. Allowing to deploy to all of them from the "master" branch, staged workflows, and then removing the existing Gitflow workflow. Ultimately it will enable us to deploy much quicker, and safer.

Includes:

Closes #424 Closes #639

Depends on #643 just for ease.

Deployment

Using one workflow for deployment enables resource dependencies per environment - as in, the webapp will not deploy if the functions fail. And each environment depends on the previous one succeeding.

As there are 3 existing container registries, but we only want to push the docker image once and deploy it 3 times, we now only use 1 registry. The workflow now uploads to the current ccomreg.azurecr.io registry, with a latest/github.sha tag, then updates the tag on the dev/test/prod app service accordingly. I can't see the registries in the Azure resources, so if they need to be moved to the resource group now is the chance, or can use one of the other 2 registries also if that is preferred.

Tests

We are testing for the current environment (Python 3.8, Django 3.1), and the desired one (Python 3.11, Django 4.2) in #640

Some Django tests needed fixing, and 3 are marked to skip as they fail to existing bugs, or rely on a more complete OMOP db existing (might look at this at some point). We do setup the OMOP.Concept table in a Pytest fixture to enable the db to be created. The tests removed for TestScanReportActiveConceptFilterViewSet were outdated, that API endpoint is only for use by the AZ_FUNCTION_USER, this test is kept, but won't pass until the bug is fixed.

Changes Needed

Some environment variables need setting up in the Github settings (I don't have access): Add three new Environments:

In each environment, setting:

These already exist in the Github secrets, just with their environment appended instead.

Once this branch is merged to devel, and then into master. We'll then delete the devel/test branches, and set master as the default branch again.

Checks

Important: please complete these before merging.

github-actions[bot] commented 3 months ago
Tests Skipped Failures Errors Time
25 1 :zzz: 0 :x: 0 :fire: 1.523s :stopwatch: