Setting kwarg job_name upon ml_client.batch_endpoints.invoke() does nothing

[MihailV1989]

• Package Name: azure-ai-ml
• Package Version: 1.11.0
• Operating System: Linux/Windows
• Python Version: 3.9

Describe the bug Setting the kwarg "job_name" upon calling ml_client.batch_endpoints.invoke() does nothing.

To Reproduce Steps to reproduce the behavior:

1. Create a batch endpoint with a PipelineComponentBatchDeployment as described here: https://learn.microsoft.com/en-us/azure/machine-learning/how-to-use-batch-training-pipeline?view=azureml-api-2&tabs=python
2. Invoke the endpoint like this:

job = ml_client.batch_endpoints.invoke(
                                       endpoint_name="",
                                       deployment_name="",
                                       inputs={},
                                       job_name="test_name"
)

The kwarg "job_name" is not documented: https://learn.microsoft.com/en-us/python/api/azure-ai-ml/azure.ai.ml.operations.batchendpointoperations?view=azure-python#azure-ai-ml-operations-batchendpointoperations-invoke

But taking a look in to the source code azure/ai/ml/operations/_batch_endpoint_operations.py you can see that there is really such a kwarg, see the code snippet below.

The Azure CLI has also equivalent argument: https://learn.microsoft.com/en-us/cli/azure/ml/batch-endpoint?view=azure-cli-latest#az-ml-batch-endpoint-invoke-optional-parameters

Expected behavior The created batch job should have a display Name "test_name" instead of a random name like "tidy_dress_xjcnyqw1".

Additional context Setting the kwarg "params_override" like this: params_override = [{EndpointYamlFields.BATCH_JOB_NAME: "test_name"}]

also does not achieve the desired result.

So there is no way to invoke a batch endpoint and get a job posted with a custom name. A workaround would be to set the description, for example after invoking the endpoint like this:

posted_job = ml_client.jobs.get(job.name)
ml_client.jobs.create_or_update(posted_job, description=f"training-{test_name}")

But this doesn't work as well.

    @distributed_trace
    @monitor_with_activity(logger, "BatchEndpoint.Invoke", ActivityType.PUBLICAPI)
    def invoke(
        self,
        endpoint_name: str,
        *,
        deployment_name: Optional[str] = None,
        inputs: Optional[Dict[str, Input]] = None,
        **kwargs,
    ) -> BatchJob:
        """Invokes the batch endpoint with the provided payload.

        :param endpoint_name: The endpoint name.
        :type endpoint_name: str
        :keyword deployment_name: (Optional) The name of a specific deployment to invoke. This is optional.
            By default requests are routed to any of the deployments according to the traffic rules.
        :paramtype deployment_name: str
        :keyword inputs: (Optional) A dictionary of existing data asset, public uri file or folder
            to use with the deployment
        :paramtype inputs: Dict[str, Input]
        :raises ~azure.ai.ml.exceptions.ValidationException: Raised if deployment cannot be successfully validated.
            Details will be provided in the error message.
        :raises ~azure.ai.ml.exceptions.AssetException: Raised if BatchEndpoint assets
            (e.g. Data, Code, Model, Environment) cannot be successfully validated.
            Details will be provided in the error message.
        :raises ~azure.ai.ml.exceptions.ModelException: Raised if BatchEndpoint model cannot be successfully validated.
            Details will be provided in the error message.
        :raises ~azure.ai.ml.exceptions.EmptyDirectoryError: Raised if local path provided points to an empty directory.
        :return: The invoked batch deployment job.
        :rtype: ~azure.ai.ml.entities.BatchJob

        .. admonition:: Example:

            .. literalinclude:: ../samples/ml_samples_misc.py
                :start-after: [START batch_endpoint_operations_invoke]
                :end-before: [END batch_endpoint_operations_invoke]
                :language: python
                :dedent: 8
                :caption: Invoke endpoint example.
        """
        outputs = kwargs.get("outputs", None)
        job_name = kwargs.get("job_name", None)
        params_override = kwargs.get("params_override", None) or []

[github-actions[bot]]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @azureml-github @Azure/azure-ml-sdk.

[shaheen-syed]

Any update on this issue?