Azure / azure-sdk-for-python

This repository is for active development of the Azure SDK for Python. For consumers of the SDK we recommend visiting our public developer docs at https://docs.microsoft.com/python/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-python.
MIT License
4.37k stars 2.71k forks source link

Not able to invoke batch endpoint with input parameter of type mlflow_model #32761

Open AtleH opened 8 months ago

AtleH commented 8 months ago

Describe the bug Invoking a batch endpoint and providing a reference to a model in a registry as input doesn't seem to work.

Invoking a batch endpoint by calling

ml_client = MLClient(...)
ml_client.batch_endpoints.invoke(endpoint="endpoint-name", inputs=model_input)

and providing an input parameter like this:

model_input = {
   "model": Input(type=AssetTypes.MLFLOW_MODEL, path=("azureml://registries/name1/models/name2/versions/1"))
}

fails during with this message:

azure.ai.ml.exceptions.ValidationException: Failed to extract version when parsing asset //registries/name1/models/name2/versions/1 of type data as arm id. Version must be provided.

The problem seems to arise during validation of the input, i.e. before it hits any infrastructure.

To Reproduce Steps to reproduce the behavior:

  1. Run the following code:
    
    from azure.ai.ml import Input, MLClient
    from azure.ai.ml.constants import AssetTypes
    from azure.identity import DefaultAzureCredential

client = MLClient(DefaultAzureCredential(), subscription_id="*****") model_input = { "model": Input( type=AssetTypes.MLFLOW_MODEL, path=("azureml://registries/name1/models/name2/versions/1") ) }

client.batch_endpoints.invoke("endpoint", inputs=model_input)

Running the above produces this output:

$ python invoke.py Traceback (most recent call last): File "/Users/ATLEH/testing/aion-registry-bug/invoke.py", line 14, in client.batch_endpoints.invoke("endpoint", inputs=model_input) File "/Users/ATLEH/testing/aion-registry-bug/.venv/lib/python3.11/site-packages/azure/core/tracing/decorator.py", line 78, in wrapper_use_tracer return func(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/Users/ATLEH/testing/aion-registry-bug/.venv/lib/python3.11/site-packages/azure/ai/ml/_telemetry/activity.py", line 275, in wrapper return f(args, **kwargs) ^^^^^^^^^^^^^^^^^^ File "/Users/ATLEH/testing/aion-registry-bug/.venv/lib/python3.11/site-packages/azure/ai/ml/operations/_batch_endpoint_operations.py", line 327, in invoke self._resolve_input(input_data, os.getcwd()) File "/Users/ATLEH/testing/aion-registry-bug/.venv/lib/python3.11/site-packages/azure/ai/ml/operations/_batch_endpoint_operations.py", line 517, in _resolve_input raise e File "/Users/ATLEH/testing/aion-registry-bug/.venv/lib/python3.11/site-packages/azure/ai/ml/operations/_batch_endpoint_operations.py", line 506, in _resolve_input entry.path = orchestrator.get_asset_arm_id(entry.path, asset_type) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/ATLEH/testing/aion-registry-bug/.venv/lib/python3.11/site-packages/azure/ai/ml/operations/_operation_orchestrator.py", line 187, in get_asset_arm_id raise ValidationException( azure.ai.ml.exceptions.ValidationException: Failed to extract version when parsing asset //registries/name1/models/name2/versions/1 of type data as arm id. Version must be provided.


**Expected behavior**
The batch endpoint should be invoked with the given parameters.

**Screenshots**

**Additional context**
Problem also reported for the az cli: https://github.com/Azure/azure-cli/issues/27685
github-actions[bot] commented 8 months ago

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @azureml-github @Azure/azure-ml-sdk.

santiagxf commented 8 months ago

Hi @AtleH. Thanks for reaching out and taking the time to report the issue. We appreciate the feedback! We confirm this was a bug in our CLI/SDK and a fix has been issued. It should be included in the next release coming next week. I will update the thread once published.

AtleH commented 7 months ago

Tested with version 1.12.1, but still producing the same error.

BradAF commented 5 months ago

This is also an issue with endpoints as well:

azure.ai.ml.exceptions.ValidationException: Failed to extract version when parsing asset //locations//workspaces//environments//versions/2 of type environments as arm id. Version must be provided.

Switching to /locations//workspaces//environments/:2 gets past this validation error, but then mangles the environment ARM id by appending azureml://locations//workspaces//environments/:2 to an ARM id:

azure.ai.ml.exceptions.ValidationException: { "result": "Failed", "errors": [ { "message": "Environment schema data cannot be a string; In order to specify an existing environments, please provide the correct registry path prefixed with 'azureml://':\n; id for environment is invalid", "path": "environment", "value": "azureml:/subscriptions//resourceGroups//providers/Microsoft.MachineLearningServices/workspaces//environments///locations//workspaces//environments//versions/1"

santiagxf commented 5 months ago

Thanks for reporting this issue. We are analyzing the issue and will provide an update shortly.

jeroenboeye commented 2 months ago

Any update on this?