GSTT-CSC / MLOps

Framework for building ML apps
GNU General Public License v3.0
9 stars 5 forks source link

142 automate application release process #145

Closed laurencejackson closed 1 year ago

laurencejackson commented 1 year ago

Draft PR for adding release feature

Releases are defined by yml files that specify an artifact source, artifact destination, and builder. See examples in tests/release/data

e.g.

source:
  mlflow:
    artifacts:
      - models:/test-model/1
destination:
  local:
builder:
  - docker build -t ghcr.io/gstt-csc/mlops:dev -f tests/data/Dockerfile tests/data
  - docker push ghcr.io/gstt-csc/mlops:dev
  - gh workflow run test_release.yml --ref $(git rev-parse --abbrev-ref HEAD) -f devBaseImage=ghcr.io/gstt-csc/mlops -f version=$(awk -F "=" '/version/ {print $2}' tests/data/_version.py | sed 's/'\''//g')

and running

mlops release <path to that yml file>

Will download the artifacts defined in the mlflow source and keep them in a local destination to be passed to the builder - currenty a list of shall commands. In this case the docker application will be packaged and pushed to github with the dev tag, the test_release.yml github workflow will then pick this up and run tests and documentation building steps.

github-actions[bot] commented 1 year ago

Coverage

Coverage Report
FileStmtsMissCoverMissing
mlops
   Experiment.py1503080%44–49, 70, 77–78, 95, 97, 102, 172–186, 205, 207–208, 256–257, 265–268, 272–273, 280–281
   ProjectFile.py23196%41
   cli.py541769%19–22, 36, 58–64, 84–86, 94–97, 105–106, 111
mlops/data/tools
   tools.py1059014%18–26, 29, 32–67, 70–116, 129–202
mlops/data/transforms
   LoadImageXNATd.py544124%27–32, 54–116
mlops/release
   Release.py33779%24, 28, 33–38
mlops/release/destinations
   ReleaseDestination.py9189%18
   SharepointDestination.py6267%12–13
   ZenodoDestination.py331942%21–25, 29–35, 43–60
mlops/release/sources
   MLFlowSource.py13654%13–14, 21–27
   ReleaseSource.py14286%14, 23
mlops/utils
   Config.py17170%1–24
TOTAL54923358% 

Tests Skipped Failures Errors Time
19 2 :zzz: 0 :x: 0 :fire: 4m 6s :stopwatch: