search

Azure / MachineLearningNotebooks

Python notebooks with ML and deep learning examples with Azure Machine Learning Python SDK | Microsoft
https://docs.microsoft.com/azure/machine-learning/service/
MIT License
4.1k stars 2.52k forks source link

AutoMLRun.get_output() gives error #1015

Closed danjohnvelasco closed 4 years ago

danjohnvelasco commented 4 years ago

I'm running my notebooks on ml.azure.com. The steps from the examples notebooks runs smoothly until the automl_run.get_output() part.

It gives me this error: ImportError: cannot import name '_import_all_legacy_submodules'

I'm using: azureml-sdk v1.7.0 ML studio notebooks

Stacktrace: ImportError Traceback (most recent call last)

in ----> 1 automl_run.get_output() /anaconda/envs/azureml_py36/lib/python3.6/site-packages/azureml/train/automl/run.py in get_output(self, iteration, metric, return_onnx_model, return_split_onnx_model, **kwargs) 560 ) 561 else: --> 562 fitted_model = self._download_automl_model(curr_run) 563 564 return curr_run, fitted_model /anaconda/envs/azureml_py36/lib/python3.6/site-packages/azureml/train/automl/run.py in _download_automl_model(self, curr_run) 775 import azureml.train.automl.runtime 776 with open(self.local_model_path, "rb") as model_file: --> 777 fitted_model = pickle.load(model_file) # type: Optional[Any] 778 except ImportError as e: 779 # Check to see if importing azureml.train.automl.runtime specifically failed /anaconda/envs/azureml_py36/lib/python3.6/site-packages/azureml/automl/runtime/__init__.py in 5 import sys 6 from azureml.automl.core.shared import logging_utilities ----> 7 from ._shared_package_legacy_import import _handle_legacy_shared_package_imports 8 9 /anaconda/envs/azureml_py36/lib/python3.6/site-packages/azureml/automl/runtime/_shared_package_legacy_import.py in 5 from os.path import dirname 6 ----> 7 from azureml.automl.core._shared_package_legacy_import import _import_all_legacy_submodules 8 9 ImportError: cannot import name '_import_all_legacy_submodules'
SnehaAgrawal-MSFT commented 4 years ago

@danjohnvelasco Could you please share the document you are referring to help you better on this.

danjohnvelasco commented 4 years ago

@SnehaAgrawal-MSFT Here's the tutorial that I'm following: https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/automated-machine-learning/regression/auto-ml-regression.ipynb

EDIT: I also tried importing some python modules like pickle and joblib and it gave me that same error: ImportError: cannot import name '_import_all_legacy_submodules'

ram-msft commented 4 years ago

@danjohnvelasco Can you please follow the below to install automl.

Pip install azureml[automl]==1.7.0

This will install azureml with the automl version 1.7 and the run time.

areed145 commented 4 years ago

I'm having the same issue within Azure Machine Learning using remote compute: The experiment failed. Finalizing run... Cleaning up all outstanding Run operations, waiting 300.0 seconds 42 items cleaning up... Cleanup took 15.721911668777466 seconds Traceback (most recent call last): File "register_model.py", line 109, in <module> best_run, fitted_model = automl_run.get_output() File "/azureml-envs/azureml_3d51ee67fd7a4d64e3eddfd0baa1c682/lib/python3.6/site-packages/azureml/train/automl/run.py", line 595, in get_output fitted_model = self._download_automl_model(curr_run) File "/azureml-envs/azureml_3d51ee67fd7a4d64e3eddfd0baa1c682/lib/python3.6/site-packages/azureml/train/automl/run.py", line 824, in _download_automl_model fitted_model = pickle.load(model_file) # type: Optional[Any] File "/azureml-envs/azureml_3d51ee67fd7a4d64e3eddfd0baa1c682/lib/python3.6/site-packages/azureml/automl/runtime/featurization/__init__.py", line 8, in <module> from .data_transformer import DataTransformer, TransformerAndMapper File "/azureml-envs/azureml_3d51ee67fd7a4d64e3eddfd0baa1c682/lib/python3.6/site-packages/azureml/automl/runtime/featurization/data_transformer.py", line 54, in <module> from ..featurizer.transformer import (AutoMLTransformer, CategoricalFeaturizers, DateTimeFeaturesTransformer, File "/azureml-envs/azureml_3d51ee67fd7a4d64e3eddfd0baa1c682/lib/python3.6/site-packages/azureml/automl/runtime/featurizer/transformer/__init__.py", line 28, in <module> from .timeseries import TimeSeriesTransformer, TimeSeriesPipelineType, NumericalizeTransformer, \ File "/azureml-envs/azureml_3d51ee67fd7a4d64e3eddfd0baa1c682/lib/python3.6/site-packages/azureml/automl/runtime/featurizer/transformer/timeseries/__init__.py", line 65, in <module> from azureml.automl.core.shared.rolling_origin_validator import RollingOriginValidator ImportError: cannot import name 'RollingOriginValidator'

I've tried numerous CondaDependencies configurations and none seem to help...

cartacioS commented 4 years ago

Hi @areed145 can you please run `pip freeze' in your notebook and share the output here? This can help us better guide you.

Thanks!

lostmygithubaccount commented 4 years ago

closing given no response, feel free to reopen

riedwaanb commented 3 years ago

I have run into the the same issue. Here is the pip freeze : adal==1.2.4 affine==2.3.0 aiodns==2.0.0 aiohttp==3.7.2 altair==4.1.0 ansiwrap==0.8.4 antlr4-python3-runtime==4.7.2 applicationinsights==0.11.9 argcomplete==1.11.1 argon2-cffi==20.1.0 asn1crypto==1.3.0 astroid==2.3.3 async-generator==1.10 async-timeout==3.0.1 attrs==19.3.0 azure-ai-formrecognizer==3.0.0 azure-batch==9.0.0 azure-cli-command-modules-nspkg==2.0.3 azure-cli-core==2.7.0 azure-cli-nspkg==3.0.4 azure-cli-telemetry==1.0.4 azure-cognitiveservices-speech==1.14.0 azure-cognitiveservices-vision-face==0.4.0 azure-common==1.1.25 azure-core==1.6.0 azure-cosmos==3.1.2 azure-cosmosdb-nspkg==2.0.2 azure-cosmosdb-table==1.0.6 azure-datalake-store==0.0.48 azure-functions-devops-build==0.0.22 azure-graphrbac==0.60.0 azure-identity==1.2.0 azure-keyvault==1.1.0 azure-loganalytics==0.1.0 azure-mgmt-advisor==2.0.1 azure-mgmt-apimanagement==0.1.0 azure-mgmt-appconfiguration==0.4.0 azure-mgmt-applicationinsights==0.1.1 azure-mgmt-authorization==0.52.0 azure-mgmt-batch==7.0.0 azure-mgmt-batchai==2.0.0 azure-mgmt-billing==0.2.0 azure-mgmt-botservice==0.2.0 azure-mgmt-cdn==4.1.0rc1 azure-mgmt-cognitiveservices==5.0.0 azure-mgmt-compute==12.0.0 azure-mgmt-consumption==2.0.0 azure-mgmt-containerinstance==1.5.0 azure-mgmt-containerregistry==3.0.0rc12 azure-mgmt-containerservice==9.0.1 azure-mgmt-core==1.0.0 azure-mgmt-cosmosdb==0.14.0 azure-mgmt-datalake-analytics==0.2.1 azure-mgmt-datalake-nspkg==3.0.1 azure-mgmt-datalake-store==0.5.0 azure-mgmt-datamigration==0.1.0 azure-mgmt-deploymentmanager==0.2.0 azure-mgmt-devtestlabs==4.0.0 azure-mgmt-dns==2.1.0 azure-mgmt-eventgrid==2.2.0 azure-mgmt-eventhub==3.0.0 azure-mgmt-hdinsight==1.4.0 azure-mgmt-imagebuilder==0.2.1 azure-mgmt-iotcentral==3.0.0 azure-mgmt-iothub==0.12.0 azure-mgmt-iothubprovisioningservices==0.2.0 azure-mgmt-keyvault==2.2.0 azure-mgmt-kusto==0.3.0 azure-mgmt-loganalytics==0.6.0 azure-mgmt-managedservices==1.0.0 azure-mgmt-managementgroups==0.2.0 azure-mgmt-maps==0.1.0 azure-mgmt-marketplaceordering==0.2.1 azure-mgmt-media==2.2.0 azure-mgmt-monitor==0.9.0 azure-mgmt-msi==0.2.0 azure-mgmt-netapp==0.8.0 azure-mgmt-network==10.2.0 azure-mgmt-nspkg==3.0.2 azure-mgmt-policyinsights==0.4.0 azure-mgmt-privatedns==0.1.0 azure-mgmt-rdbms==2.2.0 azure-mgmt-recoveryservices==0.4.0 azure-mgmt-recoveryservicesbackup==0.6.0 azure-mgmt-redhatopenshift==0.1.0 azure-mgmt-redis==7.0.0rc1 azure-mgmt-relay==0.1.0 azure-mgmt-reservations==0.6.0 azure-mgmt-resource==8.0.1 azure-mgmt-search==2.1.0 azure-mgmt-security==0.1.0 azure-mgmt-servicebus==0.6.0 azure-mgmt-servicefabric==0.4.0 azure-mgmt-signalr==0.3.0 azure-mgmt-sql==0.18.0 azure-mgmt-sqlvirtualmachine==0.5.0 azure-mgmt-storage==9.0.0 azure-mgmt-trafficmanager==0.51.0 azure-mgmt-web==0.46.0 azure-multiapi-storage==0.3.3 azure-nspkg==3.0.2 azure-storage-blob==12.4.0 azure-storage-common==1.4.2 azureml-automl-core==1.21.0 azureml-automl-runtime==1.2.0 azureml-contrib-notebook==1.2.0 azureml-contrib-pipeline-steps==1.2.0 azureml-contrib-server==1.2.0 azureml-contrib-services==1.2.0 azureml-core==1.21.0.post1 azureml-dataprep==2.8.2 azureml-dataprep-native==28.0.0 azureml-dataprep-rslex==1.6.0 azureml-dataset-runtime==1.21.0 azureml-defaults==1.2.0 azureml-explain-model==1.2.0 azureml-interpret==1.2.0 azureml-model-management-sdk==1.0.1b6.post1 azureml-pipeline==1.21.0 azureml-pipeline-core==1.21.0 azureml-pipeline-steps==1.21.0 azureml-sdk==1.21.0 azureml-telemetry==1.21.0 azureml-train==1.21.0 azureml-train-automl==1.2.0 azureml-train-automl-client==1.21.0 azureml-train-automl-runtime==1.2.0 azureml-train-core==1.21.0 azureml-train-restclients-hyperdrive==1.21.0 azureml-widgets==1.21.0 backcall==0.2.0 backports.functools-lru-cache==1.6.1 backports.tempfile==1.0 backports.weakref==1.0.post1 bcrypt==3.1.7 beautifulsoup4==4.9.3 bleach==3.1.5 boto3==1.11.17 botocore==1.14.17 brotlipy==0.7.0 cachetools==4.0.0 Cartopy==0.18.0 cchardet==2.1.7 certifi==2020.4.5.2 cffi==1.14.0 chardet==3.0.4 click==7.1.2 click-plugins==1.1.1 cligj==0.5.0 cloudpickle==1.4.1 cognitive-face==1.5.0 colorama==0.4.3 configparser==3.7.4 contextily==1.0.1 contextlib2==0.6.0.post1 cryptography==2.9.2 cycler==0.10.0 Cython==0.29.16 databricks-cli==0.14.0 decorator==4.4.2 defusedxml==0.6.0 descartes==1.1.0 dill==0.3.1.1 distro==1.5.0 docker==4.2.1 docutils==0.15.2 dotnetcore2==2.1.14 entrypoints==0.3 esda==2.3.1 fabric==2.5.0 fastcache==1.1.0 Fiona==1.8.13.post1 fire==0.3.0 flake8==3.7.9 Flask==1.0.3 fusepy==3.0.1 future==0.18.2 GDAL==3.0.2 gensim==3.8.1 geographiclib==1.50 geojson==2.5.0 geopandas==0.8.1 geoplot==0.4.1 geopy==2.0.0 giddy==2.3.3 google-api-core==1.16.0 google-auth==1.12.0 google-cloud-core==1.3.0 google-cloud-storage==1.26.0 google-resumable-media==0.5.0 googleapis-common-protos==1.51.0 gunicorn==19.9.0 humanfriendly==8.2 idna==2.9 idna-ssl==1.1.0 ijson==2.6.1 importlib-metadata==1.6.1 imutils==0.5.3 interpret-community==0.7.0 interpret-core==0.1.20 invoke==1.4.1 ipykernel==5.3.0 ipython==7.15.0 ipython-genutils==0.2.0 ipywidgets==7.5.1 isodate==0.6.0 isort==4.3.21 itsdangerous==1.1.0 javaproperties==0.5.1 jedi==0.17.0 jeepney==0.4.3 Jinja2==2.11.2 jmespath==0.10.0 joblib==0.17.0 jsmin==2.2.2 json-logging-py==0.2 jsondiff==1.2.0 JsonForm==0.0.2 jsonpickle==1.4.1 jsonschema==3.2.0 JsonSir==0.0.2 jupyter==1.0.0 jupyter-client==6.1.3 jupyter-console==6.1.0 jupyter-core==4.6.3 jupyterlab-pygments==0.1.2 keras2onnx==1.6.0 kiwisolver==1.2.0 knack==0.7.1 lazy-object-proxy==1.4.3 liac-arff==2.4.0 libpysal==4.3.0 lightgbm==2.3.0 llvmlite==0.34.0 mapclassify==2.3.0 MarkupSafe==1.1.1 matplotlib==3.3.2 mccabe==0.6.1 mercantile==1.1.6 mistune==0.8.4 mkl-fft==1.2.0 mkl-random==1.1.1 mkl-service==2.3.0 mock==4.0.2 mpmath==1.1.0 msal==1.0.0 msal-extensions==0.1.3 msrest==0.6.16 msrestazure==0.6.3 multidict==5.0.0 munch==2.5.0 nbclient==0.5.1 nbconvert==5.6.1 nbformat==5.0.7 ndg-httpsclient==0.5.1 nest-asyncio==1.4.3 networkx==2.5 nimbusml==1.7.0 nose==1.3.7 notebook==6.0.3 numba==0.51.2 numexpr==2.7.1 numpy==1.16.2 oauthlib==3.1.0 olefile==0.46 onnx==1.6.0 onnxconverter-common==1.6.0 onnxmltools==1.4.1 opencv-contrib-python==4.2.0.34 oscrypto==1.2.0 OWSLib==0.20.0 packaging==20.4 pandas==1.1.3 pandocfilters==1.4.2 papermill==1.2.1 paramiko==2.7.1 parso==0.7.0 pathspec==0.8.0 patsy==0.5.1 pep8==1.7.1 pickleshare==0.7.5 Pillow==8.0.0 pipupgrade==1.7.1 pkginfo==1.5.0.1 pmdarima==1.1.1 portalocker==1.7.0 prometheus-client==0.8.0 prompt-toolkit==3.0.5 protobuf==3.11.3 psutil==5.7.0 py-cpuinfo==5.0.0 pyarrow==0.17.1 pyasn1==0.4.8 pyasn1-modules==0.2.8 pycares==3.1.1 pycodestyle==2.5.0 pycparser==2.20 pycryptodomex==3.9.7 pydub==0.24.1 pyepsg==0.4.0 pyflakes==2.1.1 Pygments==2.6.1 PyJWT==1.7.1 pykdtree==1.3.1 pylint==2.4.4 PyNaCl==1.4.0 pyOpenSSL==19.1.0 pyparsing==2.4.7 pypiwin32==223 pyproj==2.6.1.post1 pyreadline==2.1 pyrsistent==0.16.0 PySAL==1.14.4 pyshp==2.1.2 PySocks==1.7.1 python-dateutil==2.8.1 python-decouple==3.3 Python-EasyConfig==0.1.7 python-twitter==3.5 pytz==2019.1 pywin32==228 pywinpty==0.5.7 PyYAML==5.3.1 pyzmq==19.0.1 qtconsole==4.7.4 QtPy==1.9.0 quantecon==0.4.8 rasterio==1.1.0 requests==2.24.0 requests-oauthlib==1.3.0 Resource==0.2.1 rsa==4.0 Rtree==0.9.4 ruamel.yaml==0.15.89 ruamel.yaml.clib==0.2.0 s3transfer==0.3.3 scikit-learn==0.23.2 scipy==1.5.2 scp==0.13.2 seaborn==0.11.0 SecretStorage==3.1.2 Send2Trash==1.5.0 shap==0.34.0 Shapely==1.7.1 simplegeneric==0.8.1 simplification==0.5.1 six==1.15.0 skl2onnx==1.4.9 sklearn==0.0 sklearn-pandas==1.7.0 smart-open==1.10.0 snuggs==1.4.7 soupsieve==2.0.1 splot==1.1.3 spreg==1.1.2.post1 sshtunnel==0.1.5 statsmodels==0.12.0 swagger-client==1.0.0 sympy==1.6.2 tables==3.6.1 tabulate==0.8.7 tenacity==6.1.0 termcolor==1.1.0 terminado==0.8.3 testpath==0.4.4 textwrap3==0.9.2 threadpoolctl==2.1.0 toolz==0.11.1 topojson==1.0rc11 tornado==6.0.4 tqdm==4.43.0 traitlets==4.3.3 typed-ast==1.4.1 typing==3.7.4.3 typing-extensions==3.7.4.1 urllib3==1.25.9 vsts==0.1.25 vsts-cd-manager==1.0.2 wcwidth==0.2.4 webencodings==0.5.1 websocket-client==0.56.0 Werkzeug==0.16.1 widgetsnbextension==3.5.1 win-inet-pton==1.1.0 wincertstore==0.2 wrapt==1.11.2 wxPython==4.0.7.post2 xmltodict==0.12.0 yarl==1.6.2 zipp==3.1.0

jeff-shepherd commented 3 years ago

@riedwaanb This may be an issue with mixed versions of the azureml packages. I see most packages are on the latest version, 1.21.0. However some are on 1.2.0:

azureml-automl-runtime==1.2.0 azureml-contrib-notebook==1.2.0 azureml-contrib-pipeline-steps==1.2.0 azureml-contrib-server==1.2.0 azureml-contrib-services==1.2.0 azureml-defaults==1.2.0 azureml-explain-model==1.2.0 azureml-interpret==1.2.0 azureml-train-automl==1.2.0 azureml-train-automl-runtime==1.2.0

You can try upgrading these using command like: pip install --upgrade azureml-train-automl==1.21.0