Closed needuv closed 1 month ago
I believe the identity object is pickle-able.
Could you share a repro if it is not the case?
Hi @needuv. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.
Hi @needuv, we're sending this friendly reminder because we haven't heard back from you in 7 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!
Hi @xiangyan99, we just tested with the latest azure-identity
and confirmed we are not seeing the errors with pickling that we saw before with multiprocess
. Will close this issue
Is your feature request related to a problem? Please describe.
The
azure-ai-evaluation
SDK exposes acredential
field in its evaluators (example: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/evaluators/_content_safety/_violence.py#L14) to be used when calling the Azure AI RAI service. Right now, this credential is optional, and at execution time we chooseDefaultAzureCredential
if the user did not pass anything in. However, we need to make thiscredential
parameter required in the evaluators since choosing a default credential is not safe SDK behavior.If the evaluator is being run on a single input OR when it is used in the batch
evaluate
API with no credential being passed in, the customer will not see any errors. However, if the user passes a credential to the evaluator and uses it in theevaluate
function, it will not work sinceevaluate
uses multiprocessing to do batch runs of multiple evaluators. Multiprocessing requires the evaluator to be pickle-able, but the credentials inazure-identity
are not pickle-able, which breaks this scenario.Describe the solution you'd like The credentials in
azure-identity
should be pickle-able so that they can be used in evaluators when doing batch evaluation usingevaluate
.