zenml-io / mlstacks

A series of Terraform based recipes to provision popular MLOps stacks on the cloud.
https://mlstacks.zenml.io/
Apache License 2.0
247 stars 32 forks source link

Stack YAMLs created by manual `terraform apply` use old ZenML version #45

Closed fa9r closed 1 year ago

fa9r commented 1 year ago

I manually deployed the Vertex recipes using terraform apply with "zenml-version" : "0.35.1" set in my values.tfvars.json. I love that this automatically created a vertex_stack_2023-03-13T12_01.yaml for me, however, when trying to import it, I got the following error:

Error: Cannot import stacks from other ZenML versions. The stack was created using ZenML version 0.12.0, you have version 0.35.1 installed. You can retry using the `--ignore-version-mismatch` flag. However, be aware that this might fail or lead to other unexpected behavior.

This only happened for the manual terraform deployment, using zenml stack recipe deploy ... worked as expected.

Slightly related to this: the default ZenML version in values.tfvars.json is set to "zenml-version" : "0.13.0" for the Vertex recipe. Maybe those should be set to the newest ZenML version everywhere as well.

wjayesh commented 1 year ago

For the manual case of deploying recipes using terraform apply, passing the variables can be done either through the -var flag for single variables or using the -var-file for when you have multiple variables. So in your case, you should use the latter to make the recipe use the right version.

When using the zenml CLI, the file values.tfvars.json is automatically read and used for deployment and the zenml version is set to the current version for the user.

I stopped updating the versions in the values.tfvars.json file because any manual deployments would involve the user already interacting directly with the file and therefore can assign the right version to it before passing the to the command. However, I think we can keep it to the latest, as good practice.