search

Azure / MachineLearningNotebooks

Python notebooks with ML and deep learning examples with Azure Machine Learning Python SDK | Microsoft
https://docs.microsoft.com/azure/machine-learning/service/
MIT License
4k stars 2.49k forks source link

Parallel run example fail when executing with deserialization error #1961

Open suparno89 opened 2 months ago

suparno89 commented 2 months ago

I am trying to run a python script on multiple data inputs parallel, for this i am using the batch processing examples provided in this repo. Specifically I am looking at this one: https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/machine-learning-pipelines/parallel-run/file-dataset-partition-per-folder.ipynb

Running the exact same script initially generates a lot of value error saying "This pipeline didn't have the RawDeserializer policy; can't deserialize" (screenshot 1) and finally raises the Image build failed error (screenshot 2). I cannot find anything at the logfile mentioned in the error code. I have tried using the same environment from my azureml vm but that also didn't help.

Can someone kindly help?

image

Screenshot 2024-04-10 at 17 58 37

suparno89 commented 2 months ago

I believe this is similar to the issue: https://github.com/Azure/azure-sdk-for-python/issues/34915

Only one curated environment works here which is: AzureML-ACPT-pytorch-1.13-py38-cuda11.7-gpu