search

Azure / mlops-v2

Azure MLOps (v2) solution accelerators. Enterprise ready templates to deploy your machine learning models on the Azure Platform.
https://learn.microsoft.com/en-us/azure/machine-learning/concept-model-management-and-deployment
MIT License
480 stars 225 forks source link

[repo] <Fix versions of all libraries > #16

Closed james-tn closed 1 year ago

james-tn commented 2 years ago

Why?

If you don't fix the version, if there's an upgrade from the library the pipeline may break. Currently the install CLI action does not fix the version which means that Devops Agent will pick up the latest Azure ML CLI version while the job/pipeline definition still follow old version. This may break the pipeline.

How?

Anything else?

cynthiajiangatl commented 2 years ago

Is the following error caused by the cli version issue? The "Run Azure ML Pipeline" step was never completed successfully.

2022-05-16T18:54:53.5569884Z ERROR: Met error <class 'marshmallow.exceptions.ValidationError'>:Validation for PipelineJobSchema failed: 2022-05-16T18:54:53.5571134Z 2022-05-16T18:54:53.5571645Z { 2022-05-16T18:54:53.5572090Z "jobs": { 2022-05-16T18:54:53.5572782Z "prep-job": { 2022-05-16T18:54:53.5573302Z "key": [ 2022-05-16T18:54:53.5574419Z "Pipeline node name should be a valid python identifier(upper or lower letter, number, underscore and start with a letter or underscore). Currently got prep-job." 2022-05-16T18:54:53.5575270Z ] 2022-05-16T18:54:53.5575711Z }, 2022-05-16T18:54:53.5576318Z "train-job": { 2022-05-16T18:54:53.5576837Z "key": [ 2022-05-16T18:54:53.5577924Z "Pipeline node name should be a valid python identifier(upper or lower letter, number, underscore and start with a letter or underscore). Currently got train-job." 2022-05-16T18:54:53.5578763Z ] 2022-05-16T18:54:53.5579186Z }, 2022-05-16T18:54:53.5580129Z "evaluate-job": { 2022-05-16T18:54:53.5580713Z "key": [ 2022-05-16T18:54:53.5581832Z "Pipeline node name should be a valid python identifier(upper or lower letter, number, underscore and start with a letter or underscore). Currently got evaluate-job." 2022-05-16T18:54:53.5582670Z ] 2022-05-16T18:54:53.5583100Z }, 2022-05-16T18:54:53.5583731Z "register-job": { 2022-05-16T18:54:53.5584238Z "key": [ 2022-05-16T18:54:53.5585335Z "Pipeline node name should be a valid python identifier(upper or lower letter, number, underscore and start with a letter or underscore). Currently got register-job." 2022-05-16T18:54:53.5586160Z ] 2022-05-16T18:54:53.5586597Z } 2022-05-16T18:54:53.5587015Z } 2022-05-16T18:54:53.5587430Z }

james-tn commented 2 years ago

That's exactly it. The dev team needs to follow this basic practice. If you downgrade to cli version 2.2.1 this will also fix

lokijota commented 2 years ago

@cynthiajiangatl It's a simple fix, btw -- just edit I believe pipeline.yml and replace the "-" in the job steps names with "_" (remember also the references to steps), and it'll work. I also had this problem last week.

setuc commented 1 year ago

This has been fixed in the current version. Closing the issue for now.