search

outerbounds / metaflow-tools

Tools and utilities for operating Metaflow in production
48 stars 43 forks source link

Refresh argo installation in Azure and GCP templates. #21

Open jackie-ob opened 1 year ago

jackie-ob commented 1 year ago

The Azure TF templates currently use this command to install Argo Workflows (old official quick start):

kubectl apply -n argo -f https://raw.githubusercontent.com/argoproj/argo-workflows/master/manifests/quick-start-postgres.yaml

Unfortunately, we are pulling the latest master version of this file, PLUS the content of the YAML file also refers to all kinds of :latest images which also float forwards all the time.

Argo Workflows org appears to have updated their quickstart instructions recently - it's a different set of steps now.

https://argoproj.github.io/argo-workflows/quick-start/#install-argo-workflows

In particular, it is now always pre-pinned. I think we need to migrate our Azure (and GCP) TF templates to invoke THAT instead. Line here needs changing. I will post an GH issue for this for now, it will take some testing to validate it all works fine.

This was investigated in response to non-functional argo installations based on the current Azure templates. Community slack thread here: https://outerbounds-community.slack.com/archives/C02116BBNTU/p1669973765585769

bbrandt commented 1 year ago

Handled in Azure with #31.

@jackie-ob Can you update this Issue to only be for updating the GCP templates? I did not modify GCP as I have no way to test these. We really need some automation here to know when these scripts start failing.