search

Azure / MachineLearningNotebooks

Python notebooks with ML and deep learning examples with Azure Machine Learning Python SDK | Microsoft
https://docs.microsoft.com/azure/machine-learning/service/
MIT License
4.09k stars 2.52k forks source link

Crash on automl/_azureautomlclient.py 💣 #234

Closed kasketis closed 5 years ago

kasketis commented 5 years ago

Hi,

I have tried to run several examples over automl, but I always get the following crash (this one is from how-to-use-azureml/automated-machine-learning/classification/auto-ml-classification.ipynb)

Parent Run ID: AutoML_c67ece63-159c-49b2-8168-23479a9302c5
*******************************************************************************************
ITERATION: The iteration being evaluated.
PIPELINE: A summary description of the pipeline being evaluated.
DURATION: Time taken for the current iteration.
METRIC: The result of computing score on the fitted pipeline.
BEST: The best observed score thus far.
*******************************************************************************************

 ITERATION   PIPELINE                                       DURATION      METRIC      BEST

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-8-f1a87b55c520> in <module>
----> 1 local_run = experiment.submit(automl_config, show_output = True)

/anaconda/envs/py36/lib/python3.6/site-packages/azureml/core/experiment.py in submit(self, config, tags, **kwargs)
    139         submit_func = get_experiment_submit(config)
    140         with self._log_context("submit config {}".format(config.__class__.__name__)):
--> 141             run = submit_func(config, self.workspace, self.name, **kwargs)
    142         if tags is not None:
    143             run.set_tags(tags)

/anaconda/envs/py36/lib/python3.6/site-packages/azureml/train/automl/automlconfig.py in _automl_static_submit(automl_config_object, workspace, experiment_name, **kwargs)
     44 
     45     return automl_estimator.fit(**automl_config_object.fit_params,
---> 46                                 show_output=show_output)
     47 
     48 

/anaconda/envs/py36/lib/python3.6/site-packages/azureml/train/automl/_azureautomlclient.py in fit(self, run_configuration, compute_target, X, y, sample_weight, X_valid, y_valid, sample_weight_valid, data, label, columns, cv_splits_indices, show_output, existing_run)
    285             self._fit_local(X=X, y=y, sample_weight=sample_weight, X_valid=X_valid, y_valid=y_valid,
    286                             data=data, label=label, columns=columns, cv_splits_indices=cv_splits_indices,
--> 287                             existing_run=existing_run, sample_weight_valid=sample_weight_valid)
    288         else:
    289             self._console_logger.write(

/anaconda/envs/py36/lib/python3.6/site-packages/azureml/train/automl/_azureautomlclient.py in _fit_local(self, X, y, sample_weight, X_valid, y_valid, sample_weight_valid, data, label, columns, cv_splits_indices, existing_run)
    564 
    565                 self._fit_iteration(ci,
--> 566                                     transformed_data_ntext=transformed_data_context)
    567 
    568             self._status = constants.Status.Completed

/anaconda/envs/py36/lib/python3.6/site-packages/azureml/train/automl/_azureautomlclient.py in _fit_iteration(self, ci, transformed_data_context)
    606         pipeline_id = pipeline_dto.pipeline_id
    607         pipeline_script = pipeline_dto.pipeline_spec
--> 608         train_frac = (pipeline_dto.training_percent or 100) / 100
    609         # print(pipeline_script)
    610         self.logger.info("Received pipeline: {0}".format(pipeline_script))

TypeError: unsupported operand type(s) for /: 'str' and 'int'

I use a DS3v2 Ubuntu, my AzureML.core version is 1.0.2 and below is my package list

Package                               Version   
------------------------------------- ----------
absl-py                               0.6.1     
adal                                  1.2.0     
alabaster                             0.7.10    
alembic                               1.0.5     
anaconda-client                       1.6.14    
anaconda-project                      0.8.2     
appdirs                               1.4.3     
applicationinsights                   0.11.7    
asn1crypto                            0.24.0    
astor                                 0.7.1     
astroid                               1.6.3     
astropy                               3.0.2     
async-generator                       1.10      
attrs                                 18.1.0    
audioread                             2.1.6     
autovizwidget                         0.12.6    
azure                                 4.0.0     
azure-applicationinsights             0.1.0     
azure-batch                           4.1.3     
azure-common                          1.1.16    
azure-cosmosdb-nspkg                  2.0.2     
azure-cosmosdb-table                  1.0.5     
azure-datalake-store                  0.0.39    
azure-eventgrid                       1.2.0     
azure-graphrbac                       0.40.0    
azure-keyvault                        1.1.0     
azure-loganalytics                    0.1.0     
azure-mgmt                            4.0.0     
azure-mgmt-advisor                    1.0.1     
azure-mgmt-applicationinsights        0.1.1     
azure-mgmt-authorization              0.50.0    
azure-mgmt-batch                      5.0.1     
azure-mgmt-batchai                    2.0.0     
azure-mgmt-billing                    0.2.0     
azure-mgmt-cdn                        3.0.0     
azure-mgmt-cognitiveservices          3.0.0     
azure-mgmt-commerce                   1.0.1     
azure-mgmt-compute                    4.3.1     
azure-mgmt-consumption                2.0.0     
azure-mgmt-containerinstance          1.4.0     
azure-mgmt-containerregistry          2.4.0     
azure-mgmt-containerservice           4.2.2     
azure-mgmt-cosmosdb                   0.4.1     
azure-mgmt-datafactory                0.6.0     
azure-mgmt-datalake-analytics         0.6.0     
azure-mgmt-datalake-nspkg             3.0.1     
azure-mgmt-datalake-store             0.5.0     
azure-mgmt-datamigration              1.0.0     
azure-mgmt-devspaces                  0.1.0     
azure-mgmt-devtestlabs                2.2.0     
azure-mgmt-dns                        2.1.0     
azure-mgmt-eventgrid                  1.0.0     
azure-mgmt-eventhub                   2.2.0     
azure-mgmt-hanaonazure                0.1.1     
azure-mgmt-iotcentral                 0.1.0     
azure-mgmt-iothub                     0.5.0     
azure-mgmt-iothubprovisioningservices 0.2.0     
azure-mgmt-keyvault                   1.1.0     
azure-mgmt-loganalytics               0.2.0     
azure-mgmt-logic                      3.0.0     
azure-mgmt-machinelearningcompute     0.4.1     
azure-mgmt-managementgroups           0.1.0     
azure-mgmt-managementpartner          0.1.0     
azure-mgmt-maps                       0.1.0     
azure-mgmt-marketplaceordering        0.1.0     
azure-mgmt-media                      1.0.0     
azure-mgmt-monitor                    0.5.2     
azure-mgmt-msi                        0.2.0     
azure-mgmt-network                    2.4.0     
azure-mgmt-notificationhubs           2.0.0     
azure-mgmt-nspkg                      3.0.2     
azure-mgmt-policyinsights             0.1.0     
azure-mgmt-powerbiembedded            2.0.0     
azure-mgmt-rdbms                      1.5.0     
azure-mgmt-recoveryservices           0.3.0     
azure-mgmt-recoveryservicesbackup     0.3.0     
azure-mgmt-redis                      5.0.0     
azure-mgmt-relay                      0.1.0     
azure-mgmt-reservations               0.2.1     
azure-mgmt-resource                   2.0.0     
azure-mgmt-scheduler                  2.0.0     
azure-mgmt-search                     2.0.0     
azure-mgmt-servicebus                 0.5.3     
azure-mgmt-servicefabric              0.2.0     
azure-mgmt-signalr                    0.1.1     
azure-mgmt-sql                        0.9.1     
azure-mgmt-storage                    2.0.0     
azure-mgmt-subscription               0.2.0     
azure-mgmt-trafficmanager             0.50.0    
azure-mgmt-web                        0.35.0    
azure-nspkg                           3.0.2     
azure-servicebus                      0.21.1    
azure-servicefabric                   6.3.0.0   
azure-servicemanagement-legacy        0.20.6    
azure-storage-blob                    1.4.0     
azure-storage-common                  1.4.0     
azure-storage-file                    1.4.0     
azure-storage-queue                   1.4.0     
Babel                                 2.5.3     
backcall                              0.1.0     
backports.shutil-get-terminal-size    1.0.0     
backports.tempfile                    1.0       
backports.weakref                     1.0.post1 
bcrypt                                3.1.6     
beautifulsoup4                        4.6.0     
bitarray                              0.8.1     
bitstring                             3.1.5     
bkcharts                              0.2       
blaze                                 0.11.3    
bleach                                2.1.3     
blobxfer                              1.5.5     
bokeh                                 0.12.16   
boto                                  2.48.0    
boto3                                 1.9.59    
botocore                              1.12.59   
Bottleneck                            1.2.1     
bz2file                               0.98      
cached-property                       1.5.1     
catboost                              0.10.3    
certifi                               2018.8.24 
cffi                                  1.11.5    
chainer                               4.5.0     
chainercv                             0.11.0    
chainermn                             1.3.1     
chainerrl                             0.5.0     
chardet                               3.0.4     
Click                                 7.0       
cloudpickle                           0.5.3     
clyent                                1.2.2     
cntk-gpu                              2.5.1     
color                                 0.1       
colorama                              0.3.9     
colour                                0.1.5     
configparser                          3.5.0     
contextlib2                           0.5.5     
cryptography                          2.4.2     
cupy                                  4.5.0     
cycler                                0.10.0    
Cython                                0.28.2    
cytoolz                               0.9.0.1   
dask                                  0.17.5    
databricks-cli                        0.8.2     
datashape                             0.5.4     
decorator                             4.3.0     
distributed                           1.21.8    
dlib                                  19.15.0   
docker                                3.7.0     
docker-pycreds                        0.4.0     
docutils                              0.14      
easydict                              1.9       
entrypoints                           0.2.3     
enum34                                1.1.6     
et-xmlfile                            1.0.1     
fastcache                             1.0.2     
fasteners                             0.14.1    
fastparquet                           0.2.0     
fastrlock                             0.4       
filelock                              3.0.4     
flake8                                3.6.0     
flaky                                 3.4.0     
Flask                                 1.0.2     
Flask-Cors                            3.0.4     
future                                0.17.1    
gast                                  0.2.0     
gensim                                3.6.0     
gevent                                1.3.0     
glob2                                 0.6       
gmpy2                                 2.0.8     
graphviz                              0.8.4     
greenlet                              0.4.13    
grpcio                                1.16.1    
gym                                   0.10.9    
h2o                                   3.13.0.369
h5py                                  2.7.1     
hdijupyterutils                       0.12.6    
heapdict                              1.0.0     
horovod                               0.15.2    
html5lib                              1.0.1     
hyperopt                              0.1.1     
idna                                  2.6       
imageio                               2.3.0     
imagesize                             1.0.0     
importlib2                            3.5.0.2   
imutils                               0.5.2     
ipykernel                             4.8.2     
ipython                               6.4.0     
ipython-genutils                      0.2.0     
ipywidgets                            7.2.1     
isodate                               0.6.0     
isort                                 4.3.4     
itsdangerous                          0.24      
jdcal                                 1.4       
jedi                                  0.12.0    
jeepney                               0.4       
Jinja2                                2.10      
jmespath                              0.9.3     
joblib                                0.13.0    
jsonpickle                            1.1       
jsonschema                            2.6.0     
jupyter                               1.0.0     
jupyter-client                        5.2.3     
jupyter-console                       5.2.0     
jupyter-contrib-core                  0.3.3     
jupyter-contrib-nbextensions          0.5.1     
jupyter-core                          4.4.0     
jupyter-highlight-selected-word       0.2.0     
jupyter-latex-envs                    1.4.6     
jupyter-nbextensions-configurator     0.4.0     
jupyterhub                            0.9.4     
jupyterlab                            0.34.12   
jupyterlab-launcher                   0.13.1    
Keras                                 2.2.4     
Keras-Applications                    1.0.6     
Keras-Preprocessing                   1.0.5     
kiwisolver                            1.0.1     
Lasagne                               0.2.dev1  
lazy-object-proxy                     1.3.1     
librosa                               0.6.2     
lightgbm                              2.1.2     
llvmlite                              0.23.1    
locket                                0.2.0     
lxml                                  4.2.1     
Mako                                  1.0.7     
Markdown                              3.0.1     
MarkupSafe                            1.0       
matplotlib                            2.2.2     
mccabe                                0.6.1     
mistune                               0.8.3     
mkl-fft                               1.0.0     
mkl-random                            1.0.1     
mock                                  2.0.0     
monotonic                             1.5       
more-itertools                        4.1.0     
mpi4py                                3.0.0     
mpmath                                1.0.0     
msgpack                               0.6.0     
msgpack-python                        0.5.6     
msrest                                0.6.2     
msrestazure                           0.5.1     
multipledispatch                      0.5.0     
mxnet                                 1.3.0     
mxnet-model-server                    1.0.0     
nb-anacondacloud                      1.4.0     
nb-conda                              2.2.1     
nb-conda-kernels                      2.1.0     
nbconvert                             5.3.1     
nbformat                              4.4.0     
nbpresent                             3.0.2     
nbsphinx                              0.3.5     
ndg-httpsclient                       0.5.1     
networkx                              2.1       
nltk                                  3.3       
nose                                  1.3.7     
nose-parameterized                    0.6.0     
notebook                              5.5.0     
numba                                 0.38.0    
numexpr                               2.6.8     
numpy                                 1.14.5    
numpydoc                              0.8.0     
oauthlib                              2.1.0     
odo                                   0.5.1     
olefile                               0.45.1    
onnx                                  1.2.3     
onnx-chainer                          1.3.0a1   
onnx-mxnet                            0.4.2     
onnxmltools                           1.3.0     
onnxruntime                           0.1.4     
openpyxl                              2.5.3     
packaging                             17.1      
pamela                                0.3.0     
pandas                                0.23.0    
pandas-datareader                     0.6.0     
pandocfilters                         1.4.2     
paramiko                              2.4.2     
parso                                 0.2.0     
partd                                 0.3.8     
path.py                               11.0.1    
pathlib2                              2.3.2     
pathspec                              0.5.9     
patsy                                 0.5.0     
pbr                                   5.1.1     
pep8                                  1.7.1     
pexpect                               4.5.0     
pickleshare                           0.7.4     
Pillow                                5.1.0     
pip                                   18.1      
pkginfo                               1.4.2     
plotly                                3.4.2     
pluggy                                0.6.0     
ply                                   3.11      
progressbar33                         2.4       
prometheus-client                     0.4.2     
prompt-toolkit                        1.0.15    
protobuf                              3.6.1     
psutil                                5.4.5     
psycopg2                              2.7.5     
ptyprocess                            0.5.2     
py                                    1.5.3     
pyasn1                                0.4.5     
PyAudio                               0.2.11    
pycodestyle                           2.4.0     
pycosat                               0.6.3     
pycparser                             2.18      
pycrypto                              2.6.1     
pycuda                                2018.1.1  
pycurl                                7.43.0.1  
pydicom                               1.2.1     
pydocumentdb                          2.3.3     
pydot-ng                              2.0.0     
pyflakes                              2.0.0     
pygame                                1.9.4     
pyglet                                1.3.2     
Pygments                              2.2.0     
pygpu                                 0.7.6     
PyJWT                                 1.7.0     
pykerberos                            1.2.1     
pylint                                1.8.4     
pymongo                               3.7.2     
PyNaCl                                1.3.0     
pyodbc                                4.0.24    
pyOpenSSL                             18.0.0    
pyparsing                             2.2.0     
PySocks                               1.6.8     
pytest                                3.5.1     
pytest-arraydiff                      0.2       
pytest-astropy                        0.3.0     
pytest-doctestplus                    0.1.3     
pytest-openfiles                      0.3.0     
pytest-remotedata                     0.2.1     
pytest-runner                         4.2       
python-dateutil                       2.7.5     
python-editor                         1.0.3     
python-oauth2                         1.1.0     
pytools                               2018.5.2  
pytz                                  2018.4    
PyWavelets                            0.5.2     
PyYAML                                3.12      
pyzmq                                 17.0.0    
QtAwesome                             0.4.4     
qtconsole                             4.3.1     
QtPy                                  1.4.1     
readme-renderer                       24.0      
requests                              2.21.0    
requests-file                         1.4.3     
requests-ftp                          0.3.1     
requests-kerberos                     0.12.0    
requests-oauthlib                     1.0.0     
requests-toolbelt                     0.8.0     
resampy                               0.2.1     
retrying                              1.3.3     
rope                                  0.10.7    
ruamel-yaml                           0.15.35   
ruamel.yaml                           0.15.51   
s3transfer                            0.1.13    
scikit-image                          0.13.1    
scikit-learn                          0.20.0    
scipy                                 1.1.0     
seaborn                               0.9.0     
SecretStorage                         3.1.1     
Send2Trash                            1.5.0     
setuptools                            39.1.0    
simplegeneric                         0.8.1     
singledispatch                        3.4.0.3   
six                                   1.11.0    
smart-open                            1.7.1     
snowballstemmer                       1.2.1     
sortedcollections                     0.6.1     
sortedcontainers                      1.5.10    
SoundFile                             0.10.2    
sparkmagic                            0.12.6    
Sphinx                                1.7.4     
sphinx-rtd-theme                      0.4.2     
sphinxcontrib-websupport              1.0.1     
spyder                                3.2.8     
SQLAlchemy                            1.2.7     
statsmodels                           0.9.0     
sudospawner                           0.6.0.dev0
sympy                                 1.1.1     
tables                                3.4.3     
tabulate                              0.8.2     
tblib                                 1.3.2     
tensorboard                           1.12.0    
tensorflow-gpu                        1.12.0    
termcolor                             1.1.0     
terminado                             0.8.1     
testpath                              0.3.1     
tf2onnx                               0.4.0     
Theano                                1.0.3     
thrift                                0.11.0    
toolz                                 0.9.0     
torch                                 1.0.0     
torchvision                           0.2.1     
tornado                               5.0.2     
tqdm                                  4.28.1    
traitlets                             4.3.2     
twine                                 1.12.1    
typing                                3.6.4     
typing-extensions                     3.6.6     
unicodecsv                            0.14.1    
urllib3                               1.24.1    
wcwidth                               0.1.7     
webencodings                          0.5.1     
websocket-client                      0.55.0    
Werkzeug                              0.14.1    
wget                                  3.2       
wheel                                 0.31.1    
widgetsnbextension                    3.2.1     
wrapt                                 1.10.11   
xgboost                               0.80      
xlrd                                  1.1.0     
XlsxWriter                            1.0.4     
xlwt                                  1.3.0     
zict                                  0.1.3

Any clue?

wchill commented 5 years ago

Hi @kasketis, this should have been fixed in 1.0.8. Please try updating your SDK packages.

kasketis commented 5 years ago

SDK was the case. There were mismatched versions under the various python kernels of the DS3v2. Works as expected with 1.0.17, thanks 🍺