search

qtangs / sagemaker-notebook-container

Docker images that replicate the Amazon SageMaker Notebook instance.
MIT License
58 stars 30 forks source link

Sagemaker module can't be imported #3

Closed mludvig closed 2 years ago

mludvig commented 4 years ago

Hi, in the latest qtangs/sagemaker-notebook:python3-tensorflow-p36 (a5c579747df1) docker image I'm unable to do import sagemaker, it fails with incompatibility between sagemaker and python-dateutil.

The installed python-dateutil is version 2.8.1 but sagemaker requires <2.8.1. I had to downgrade it in the shell with conda install -n python3 'python-dateutil<2.8.1', now import sagemaker works.

Not sure if we need sagemaker upgraded or dateutil downgraded but as it is it doesn't work.

qtangs commented 4 years ago

Thanks for reporting the bug.

The root cause is related to this line in Dockerfile: $HOME/envs/create_env_file.sh python3

This line creates the conda env file by selecting a subset of all libraries from the original SageMaker environments (so as to reduce Docker image size). The subset is defined by include_libraries.txt. Since I put only the high-level libraries (like sagemaker, pandas, scikit-learn) in there, latest versions of dependent libraries are taken during conda env create, which results in python-dateutil having version 2.8.1.

I have corrected this and now include all dependent libraries in include_libraries.txt. Now python-dateutil in python3 environment should have version 2.7.3, just like to original environment in this code: https://github.com/qtangs/sagemaker-notebook-container/blob/master/envs/env_exports/python3.yml#L233

Please try the new image (currently building) and let me know if you can run import sagemaker successfully without any changes.