Monitoring a prompt-flow model in production with system-assigned instance

[fraddddd]

Hello! Following this guide, I created an endpoint for my prompt flow, by using a System Assigned Identity, then I tested the enpoint and used it in production. Now, I want to explore the monitoring functionality as discribed in page "Model monitoring for generative AI applications (preview)". but it is failing because the data assets are empty. I see now that probably I should have used a User-assigned identity.

Can you confirm that it is possible to use the "Monitoring" functionality only with a User-assigned identity? If so, can you please specify it in this page? As far as I see here, the only difference between the two types of identities in usage is that the User-assigned identity is recommended if you need to create multiple endpoints (that was not my use case).

If it should be possible to monitor the model with a system-assigned identity, can you please help me to figure out the possible reasons for which the data assets of input and output are created but are empty?

Thanks in advance!

---

Document Details

⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

• ID: 96e1e856-69a8-8496-651c-d0c460992250
• Version Independent ID: 156e113f-5d44-7b90-d1ce-c8cce907964a
• Content: Deploy a flow as a managed online endpoint for real-time inference (preview) - Azure Machine Learning
• Content Source: articles/machine-learning/prompt-flow/how-to-deploy-for-real-time-inference.md
• Service: machine-learning
• Sub-service: core
• GitHub Login: @likebupt
• Microsoft Alias: keli19

[ManoharLakkoju-MSFT]

@fraddddd Thanks for your feedback! We will investigate and update as appropriate.

[Naveenommi-MSFT]

@likebupt Could you please review add comments on this, update as appropriate.

[Naveenommi-MSFT]

@fraddddd Thank you for bringing this to our attention. I've delegated this to content author @likebupt, who will review it and offer their insightful opinions.

[imatiach-msft]

"Can you confirm that it is possible to use the "Monitoring" functionality only with a User-assigned identity?" Hi @fraddddd , sorry for the late response. I think it should be possible to use a system-assigned identity. Could you please send the output logs to us so we can take a look? Please send to email on my github profile.

[likebupt]

Hi @fraddddd , Thanks for your feedback. My understanding is that your endpoint works well, but you set up monitoring failed due to the dataset is empty. I think this might be related to the model data collector issue. Could you please share your workspace region, endpoint name and deployment name so that we can investigate? Thanks.