search

aws / sagemaker-python-sdk

A library for training and deploying machine learning models on Amazon SageMaker
https://sagemaker.readthedocs.io/
Apache License 2.0
2.1k stars 1.14k forks source link

[NON-PY-SDK] pandas version issue with MXNet 1.6 environment #1428

Closed ehsanmok closed 4 years ago

ehsanmok commented 4 years ago

Describe the bug The mxnet_p36 env seems to enforce pandas==0.24 somehow which makes using awslabs/gluonts (pip installed from master) to fail (with and without upgrading pip) because GluonTS requires pandas~=1.0

ContextualVersionConflict: (pandas 0.24.2 (/home/ec2-user/anaconda3/envs/mxnet_p36/lib/python3.6/site-packages), Requirement.parse('pandas~=1.0'), {'gluonts'})

To reproduce Run this in a notebook cell in mxnet_p36 env

%%bash
if [[ `python -m pip freeze | grep enum34` ]]; then
    python -m pip uninstall -y enum34
fi
python -m pip install --upgrade -q sagemaker
python -m pip install -q git+https://github.com/awslabs/gluon-ts@master

Also it seems pandas requirement was not specified anywhere for MXNet here.

System information A description of your system. Please provide:

laurenyu commented 4 years ago

sorry for the delayed response here - just to clarify, you're referring to the mxnet_p36 environment in a SageMaker Notebook Instance, right? (just want to make sure I route this to the correct team)

ehsanmok commented 4 years ago

Yes, that's right. I observed the package installation order causes random dependecies error. For example, on mxnet_p36 if I install python -m pip install -q altair==4.1 first then python -m pip install -q git+https://github.com/awslabs/gluon-ts@master and using gluonts results in the error above but if I reverse the order it seems fine!

I suspect there's something wrong with how pip handles stuff in notebook instances and the fact that notebook instances are still using pip v10.0 instead of the newest v20.0 may contribute to the issue as well.

laurenyu commented 4 years ago

gotcha, thanks for the clarification. I'm going to pass this along to the team that owns Notebook Instances to see if they have any insight. thanks for your patience!

neelamgehlot commented 4 years ago

Hi @ehsanmok ,

We have deployed the newer version of pandas. This change will be deployed to all regions in coming week.

In the meantime, you can update the version of pandas using conda update pandas in mxnet_p36 environment to unblock yourself.

Please re-open this issue if you still didn't get >1.0 version of pandas by end of next week.

Thanks, Neelam