[@azure/arm-machinelearning] Unable to create Job due to bad validation

[zoubicek]

• Package Name: arm-machinelearning
• Package Version: 2.1.1
• Operating system: Windows 10
• [x] nodejs
  • version: v20.11.1
• [ ] browser
  • name/version:
• [ ] typescript
  • version:
• Is the bug related to documentation in
  • [ ] README.md
  • [ ] source code documentation
  • [ ] SDK API docs on https://docs.microsoft.com

Describe the bug Unable to create a new Job using "client.jobs.createOrUpdate" function. The issue occurs when I add inputs of type "uri_file" or "uri_folder". With input of "literal" type it works. If I call request using REST API with same params, the job is created without any problems.

Returned error: "message": "Request is invalid and/or missing fields.", "trace": "RestError: Request is invalid and/or missing fields.\n at handleErrorResponse (...\\node_modules\\@azure\\core-client\\dist\\commonjs\\deserializationPolicy.js:146:19)\n at deserializeResponseBody (...\\node_modules\\@azure\\core-client\\dist\\commonjs\\deserializationPolicy.js:82:45)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)" }

I also try to add the parameters "mode" and "description" but it does not help.

To Reproduce

let jobParams = {
  properties: {
    codeId: code.id,
    command: "python train.py --dataset_path ${{inputs.dataset_path}}",
    computeId: compute.id,
    displayName: "Training Process",
    environmentId: "azureml://registries/azureml/environments/acft-transformers-image-gpu/versions/25",
    environmentVariables: {},
    inputs: {
      dataset_path: {
        jobInputType: "uri_file",
        uri: data.id,
      },
    },
    jobType: "Command",
    outputs: {},
  },
};
await client.jobs.createOrUpdate(
  resourceGroupName,
  workspaceName,
  jobName,
  jobParams,
);

Expected behavior The job will be created.

Screenshots

Additional context

[kazrael2119]

will check

[kazrael2119]

@zoubicek , which version did you use in rest api? this one?

[zoubicek]

@kazrael2119 yes, with the following endpoint -> https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.MachineLearningServices/workspaces/{workspaceName}/jobs/{id}?api-version=2023-10-01

[kazrael2119]

@zoubicek not sure whether your issue is because of the api version. In 2.1.1, the job api uses 2022-10-01, azure-arm-machinelearning-3.0.0.zip here is the sdk generated with 2023-10-01, could you try to use this and call again?

[github-actions[bot]]

Hi @zoubicek. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

[zoubicek]

@kazrael2119 I think I am not able to try it because in version 3.0.0 you delete the "AzureMachineLearningWorkspaces" client and without any samples or documentation I do not know how to replace it. But I can confirm that REST API works even with version 2022-10-01. At the same time, I did not notice any change between these versions in the documentation. I would rather focus on the internal validation within the package, if there is a bug in the regex in the validation of URI (Job input AssetUri) for the "uri_file" or "uri_folder" types. Unfortunately, unlike the REST API, the library is not very specific when it comes to errors and only writes basic phrases without any details. But it makes the sense to me to look for a bug in the validation of URI for these kind of inputs.

[kazrael2119]

for literal type, in sdk code, it is generated as

export interface LiteralJobInput extends JobInput {
  /** Polymorphic discriminator, which specifies the different types this object can be */
  jobInputType: "literal";
  /** [Required] Literal value for the input. */
  value: string;
}

we can see this model only extends JobInput model but for other types,

export interface UriFileJobInput extends AssetJobInput, JobInput {}
export interface AssetJobInput {
  /** Input Asset Delivery Mode. */
  mode?: InputDeliveryMode;
  /** [Required] Input Asset URI. */
  uri: string;
}

they extend two models, and in request body, it doesn't send the parameter defined in AssetJobInput , So the creation failed. Maybe it is a codegen issue.

[qiaozha]

I will take a look

[kazrael2119]

Hi @zoubicek , could you try to use this package to create a job? azure-arm-machinelearning-2.2.0.zip

we fixed this issue, and it worked on my side

[zoubicek]

Hi @kazrael2119, I confirm that it works on my side as well. Thx

[kazrael2119]

released the new package here: https://www.npmjs.com/package/@azure/arm-machinelearning/v/2.2.0