microsoft / dstoolkit-mlops-base

Support ML teams to accelerate their model deployment to production leveraging Azure
MIT License
89 stars 39 forks source link

Endpoint Model Profiler #32

Open FlorianPydde opened 2 years ago

FlorianPydde commented 2 years ago

AML provides a model profiling functionality which enables team to assess their deployment services (memory consumption, latency, etc): https://docs.microsoft.com/en-us/azure/machine-learning/how-to-deploy-profile-model?pivots=py-sdk

During model deployment to AKS (TEST/PROD), it may be useful to create the model profile and upload it to the default blob storage, for simplicity. This file can used to trigger action based specifics metrics or display it on a operation dashboard