FlorianPydde
commented
2 years ago Hi Jonathan. Apologies for the (very) late reply.
It's a very pertinent question and, if I understand it correctly, my answer is "it depends".
For teams starting with MLOps, I'd say that the training is part of the inner loop (DEV env) and only the model artefact and scoring script are moved to the outer loop (TEST, pred-PROD, etc).
For more advance teams, you might want to consider auto-retraining functionalities also pushed to PROD, which would enable model retraining based on some event trigger (data drift. model drift, etc).
Does that make sense ?
Hi,
I've used this for demoing to my customer and I think it would great to show how the azure-pipelines can be used to deploy to higher environments using the recommended approach of "compile once promote everywhere" off of the main branch.
As I am new to ML Ops, I'm not sure the recommended approach for deploying to higher environments
Should the training be part of the "compile once" continuous integration/build phase https://github.com/microsoft/dstoolkit-mlops-base/blob/322f45171b91646eae8b4d276c371d3685d7c54f/azure-pipelines/PIPELINE-1-modeling.yml#L76
and these pieces https://github.com/microsoft/dstoolkit-mlops-base/blob/322f45171b91646eae8b4d276c371d3685d7c54f/azure-pipelines/PIPELINE-1-modeling.yml#L104 be part of the continuous deployment/promote everywhere
The high level of what I'm trying to understand is how the batch inference and training pipeline should fit into this flow