ImportError: cannot import name 'AutoMLStep' from 'azureml.train.automl

[Alla-Abdella]

ImportError: cannot import name 'AutoMLStep' from 'azureml.train.automl

[YutongTie-MSFT]

@alla15747 Hi, thanks for reaching out to us. Could you please share your environment file so that we can know the details of this issue?

[vizhur]

@alla15747 please make sure that azureml-train-automl-runtime is installed in your environment if you using sdk>=1.0.76 or azureml-train-automl if using older version

[Alla-Abdella]

I'm running the code on the compute target and not my local machine. SDK 1.0.72 How to install packages in Azure Devops environment like azureml-train-automl?

[Alla-Abdella]

(base) C:\Users\aabdel137>pip freeze absl-py==0.8.1 adal==1.2.2 alabaster==0.7.12 anaconda-client==1.7.2 anaconda-navigator==1.9.7 anaconda-project==0.8.3 ansiwrap==0.8.4 applicationinsights==0.11.9 asn1crypto==0.24.0 astor==0.8.0 astroid==2.3.1 astropy==3.2.1 atomicwrites==1.3.0 attrs==19.3.0 azure-common==1.1.23 azure-graphrbac==0.61.1 azure-mgmt-authorization==0.60.0 azure-mgmt-containerregistry==2.8.0 azure-mgmt-keyvault==2.0.0 azure-mgmt-resource==5.1.0 azure-mgmt-storage==6.0.0 azureml-contrib-interpret==1.0.72 azureml-contrib-notebook==1.0.72 azureml-core==1.0.72 azureml-dataprep==1.1.29 azureml-dataprep-native==13.1.0 azureml-explain-model==1.0.72 azureml-interpret==1.0.72.1 azureml-pipeline==1.0.72 azureml-pipeline-core==1.0.72 azureml-pipeline-steps==1.0.72 azureml-sdk==1.0.72 azureml-telemetry==1.0.72 azureml-train==1.0.72 azureml-train-core==1.0.72 azureml-train-restclients-hyperdrive==1.0.72 azureml-widgets==1.0.72 Babel==2.7.0 backcall==0.1.0 backports.functools-lru-cache==1.5 backports.os==0.1.1 backports.shutil-get-terminal-size==1.0.0 backports.tempfile==1.0 backports.weakref==1.0.post1 beautifulsoup4==4.7.1 bitarray==0.9.3 bkcharts==0.2 bleach==3.1.0 bokeh==1.2.0 boto==2.49.0 Bottleneck==1.2.1 certifi==2019.6.16 cffi==1.12.3 chardet==3.0.4 Click==7.0 cloudpickle==1.2.1 clyent==1.2.2 colorama==0.4.1 comtypes==1.1.7 conda==4.7.10 conda-build==3.18.8 conda-package-handling==1.3.11 conda-verify==3.4.2 contextlib2==0.5.5 coverage==4.5.4 cryptography==2.7 cycler==0.10.0 Cython==0.29.12 cytoolz==0.10.0 dask==2.1.0 decorator==4.4.0 defusedxml==0.6.0 distributed==2.1.0 distro==1.4.0 docker==4.1.0 docutils==0.14 dotnetcore2==2.1.10 entrypoints==0.3 et-xmlfile==1.0.1 fastcache==1.1.0 filelock==3.0.12 flake8==3.7.9 flake8-formatter-junit-xml==0.0.6 Flask==1.1.1 fusepy==3.0.1 future==0.17.1 gast==0.3.2 gevent==1.4.0 glob2==0.7 google-pasta==0.1.7 greenlet==0.4.15 grpcio==1.24.3 h5py==2.9.0 heapdict==1.0.0 html5lib==1.0.1 idna==2.8 imageio==2.5.0 imagesize==1.1.0 importlib-metadata==0.23 interpret-community==0.1.0.3.3 interpret-core==0.1.18 ipykernel==5.1.1 ipython==7.6.1 ipython-genutils==0.2.0 ipywidgets==7.5.0 isodate==0.6.0 isort==4.3.21 itsdangerous==1.1.0 jdcal==1.4.1 jedi==0.13.3 jeepney==0.4.1 Jinja2==2.10.1 jmespath==0.9.4 joblib==0.13.2 json5==0.8.4 jsonpickle==1.2 jsonschema==3.0.1 junit-xml==1.8 jupyter==1.0.0 jupyter-client==5.3.1 jupyter-console==6.0.0 jupyter-core==4.5.0 jupyterlab==1.0.2 jupyterlab-server==1.0.0 Keras-Applications==1.0.8 Keras-Preprocessing==1.1.0 keyring==18.0.0 kiwisolver==1.1.0 kmodes==0.10.1 lazy-object-proxy==1.4.2 libarchive-c==2.8 llvmlite==0.29.0 locket==0.2.0 lxml==4.3.4 Markdown==3.1.1 MarkupSafe==1.1.1 matplotlib==3.1.0 mccabe==0.6.1 menuinst==1.4.16 mistune==0.8.4 mkl-fft==1.0.12 mkl-random==1.0.2 mkl-service==2.0.2 mock==3.0.5 more-itertools==7.2.0 mpmath==1.1.0 msgpack==0.6.1 msrest==0.6.10 msrestazure==0.6.2 multipledispatch==0.6.0 navigator-updater==0.2.1 nbconvert==5.5.0 nbformat==4.4.0 ndg-httpsclient==0.5.1 networkx==2.3 nltk==3.4.4 nose==1.3.7 notebook==6.0.0 numba==0.44.1 numexpr==2.6.9 numpy==1.16.4 numpydoc==0.9.1 oauthlib==3.1.0 olefile==0.46 openpyxl==2.6.2 packaging==19.2 pandas==0.24.2 pandocfilters==1.4.2 papermill==1.2.1 parso==0.5.0 partd==1.0.0 path.py==12.0.1 pathlib2==2.3.4 pathspec==0.6.0 patsy==0.5.1 pep8==1.7.1 pickleshare==0.7.5 Pillow==6.1.0 pkginfo==1.5.0.1 pluggy==0.13.0 ply==3.11 prometheus-client==0.7.1 prompt-toolkit==2.0.9 protobuf==3.10.0 psutil==5.6.3 py==1.8.0 py4j==0.10.7 pyasn1==0.4.7 pycodestyle==2.5.0 pycosat==0.6.3 pycparser==2.19 pycrypto==2.6.1 pycurl==7.43.0.3 pyflakes==2.1.1 Pygments==2.4.2 PyJWT==1.7.1 pylint==2.4.2 pyodbc==4.0.26 pyOpenSSL==19.0.0 pyparsing==2.4.2 pypiwin32==223 pyreadline==2.1 pyrsistent==0.14.11 PySocks==1.7.0 pyspark==2.4.4 pytest==5.2.2 pytest-arraydiff==0.3 pytest-astropy==0.5.0 pytest-cov==2.7.1 pytest-doctestplus==0.3.0 pytest-openfiles==0.3.2 pytest-remotedata==0.3.1 python-dateutil==2.8.0 python-dotenv==0.10.3 pytz==2019.1 PyWavelets==1.0.3 pywin32==223 pywinpty==0.5.5 PyYAML==5.1.1 pyzmq==18.0.0 QtAwesome==0.5.7 qtconsole==4.5.1 QtPy==1.8.0 requests==2.22.0 requests-oauthlib==1.2.0 rope==0.14.0 ruamel-yaml==0.15.46 ruamel.yaml==0.15.89 scikit-image==0.15.0 scikit-learn==0.21.2 scipy==1.2.1 seaborn==0.9.0 SecretStorage==3.1.1 Send2Trash==1.5.0 shap==0.29.3 simplegeneric==0.8.1 singledispatch==3.4.0.3 six==1.12.0 sklearn==0.0 snowballstemmer==1.9.0 sortedcollections==1.1.2 sortedcontainers==2.1.0 soupsieve==1.8 Sphinx==2.1.2 sphinxcontrib-applehelp==1.0.1 sphinxcontrib-devhelp==1.0.1 sphinxcontrib-htmlhelp==1.0.2 sphinxcontrib-jsmath==1.0.1 sphinxcontrib-qthelp==1.0.2 sphinxcontrib-serializinghtml==1.1.3 sphinxcontrib-websupport==1.1.2 spyder==3.3.6 spyder-kernels==0.5.1 SQLAlchemy==1.3.5 statsmodels==0.10.0 sympy==1.4 tables==3.5.2 tblib==1.4.0 tenacity==5.1.5 tensorboard==1.14.0 tensorflow==1.14.0 tensorflow-estimator==1.14.0 tensorflow-gpu==1.14.0 termcolor==1.1.0 terminado==0.8.2 testpath==0.4.2 textwrap3==0.9.2 tf-estimator-nightly==1.14.0.dev2019031401 toolz==0.10.0 tornado==6.0.3 tqdm==4.37.0 traitlets==4.3.2 typed-ast==1.4.0 unicodecsv==0.14.1 unittest-xml-reporting==2.5.2 urllib3==1.24.2 wcwidth==0.1.7 webencodings==0.5.1 websocket-client==0.56.0 Werkzeug==0.15.4 widgetsnbextension==3.5.0 win-inet-pton==1.1.0 win-unicode-console==0.5 wincertstore==0.2 wrapt==1.11.2 xlrd==1.2.0 XlsxWriter==1.1.8 xlwings==0.15.8 xlwt==1.3.0 zict==1.0.0 zipp==0.6.0

[vizhur]

AutoML became a part of default distribution (azureml-sdk) since 1.0.83 https://docs.microsoft.com/en-us/azure/machine-learning/azure-machine-learning-release-notes#2020-01-06

if your client, that I believe pins the version of azureml sdk packages for the remote environment is 1.0.83, you will have automl on remote.

If you want to stay with 1.0.72 you can either reference automl extras azureml-sdk[automl] or explicitly reference azureml-train-automl (prefered).

I would recommend to do both, upgrade client to the latest version and explicitly reference packages you need for your particular scenario not relying on metapackages like azureml-sdk

Our reference doc will help you to get a set of the packages needed for your scenario https://docs.microsoft.com/en-us/python/api/overview/azure/ml/?view=azure-ml-py

By design every AzureML Python SDK package will bring necessary internal dependencies (of course except some corner cases :) )

[Alla-Abdella]

Thanks Vizhur, do you mind showing an example on how to reference azureml-train-automl in Azure Devops or Portal? Thank you for the links!

[vizhur]

Not sure about your particular scenario, would you mind to share your ADO scenario so I can think of how to update it?

[Alla-Abdella]

MY scenario is implementing MLOPs example with automl step.

[vizhur]

Let me try to pull some relevant folks into the thread

[SKrupa]

For AutoML, all the remote dependencies will get taken care of and will match whatever local dependencies are installed, e.g. if you have azureml-train-automl==1.0.72 installed, that version will be installed remotely for the training job. We provide 2 clients to submitting these remote jobs currently, a thin client for submitting some types of remote jobs which is included as part of azureml-sdk, and a fuller client which enables more experiences such as Pipeline runs as part of azureml-train-automl. Since it looks like you are trying to use Pipelines, you will need to install the full azureml-train-automl client.

Furthermore, the namespace for AutoMLStep changed recently, if you are using <1.0.76 the namespace would be "from azureml.train.automl import AutoMLStep", for >=1.0.76, you'll want to use "from azureml.train.automl.runtime import AutoMLStep" instead.

[Alla-Abdella]

I'm have sdk 1.0.72 installed. And I'm using from azureml.train.automl import AutoMLStep. Is there anyway to check the sdk version on the compute target machine?

[SKrupa]

From your pip freeze, it doesn't look like you have the AutoML SDK installed. For the pipelines experience, you will need to have the SDK installed locally, not just on the target compute. Could you run "pip install azureml-train-automl"?

[YutongTie-MSFT]

@alla15747 We will now proceed to close this thread. If there are further questions regarding this matter, please respond here and @YutongTie-MSFT and we will gladly continue the discussion.

[CESARDELATORRE]

@SKrupa - Are you running your own code or a particular notebook sample from this repo?