search

aws / amazon-sagemaker-feedback

Amazon SageMaker Public Feedback Dashboard
Creative Commons Attribution Share Alike 4.0 International
4 stars 1 forks source link

Sagemaker Studio Data Science 3.0 image now has package conflicts #56

Closed HFulcher closed 3 months ago

HFulcher commented 4 months ago

Product Version

Issue Description

As of today our team can no longer use Pandas with the Data Science 3.0 image.

We intermittently have the following error when calling import pandas as pd

/opt/conda/lib/python3.10/site-packages/pandas/core/computation/expressions.py:21: UserWarning: Pandas requires version '2.8.4' or newer of 'numexpr' (version '2.8.3' currently installed).
  from pandas.core.computation.check import NUMEXPR_INSTALLED
/opt/conda/lib/python3.10/site-packages/pandas/core/arrays/masked.py:60: UserWarning: Pandas requires version '1.3.6' or newer of 'bottleneck' (version '1.3.5' currently installed).
  from pandas.core import (

When Pandas is working, if calling read_csv against and S3 URI we receive:

ImportError: Pandas requires version '2022.11.0' or newer of 'fsspec' (version '0.7.4' currently installed)

Calling !pip list in the notebook provides the following

Package                              Version
------------------------------------ --------------------
alabaster                            0.7.12
anaconda-client                      1.11.0
anaconda-project                     0.11.1
anyio                                3.5.0
appdirs                              1.4.4
argon2-cffi                          21.3.0
argon2-cffi-bindings                 21.2.0
arrow                                1.2.2
astroid                              3.0.2
astropy                              6.0.0
astropy-iers-data                    0.2024.1.29.0.30.37
asttokens                            2.4.1
atomicwrites                         1.4.0
attrs                                23.2.0
Authlib                              1.3.0
Automat                              20.2.0
autopep8                             1.6.0
autovizwidget                        0.20.4
awscli                               1.32.33
Babel                                2.9.1
backcall                             0.2.0
backports.functools-lru-cache        1.6.4
backports.tempfile                   1.0
backports.weakref                    1.0.post1
bcrypt                               3.2.0
beautifulsoup4                       4.11.1
binaryornot                          0.4.4
bitarray                             2.5.1
bkcharts                             0.2
black                                22.6.0
bleach                               4.1.0
blinker                              1.7.0
bokeh                                3.3.4
boltons                              23.0.0
boto3                                1.34.33
botocore                             1.34.33
Bottleneck                           1.3.5
brotlipy                             0.7.0
certifi                              2023.11.17
cffi                                 1.15.1
chardet                              4.0.0
charset-normalizer                   2.0.4
click                                8.1.7
cloudpickle                          2.2.1
clyent                               1.2.2
colorama                             0.4.4
colorcet                             3.0.0
comm                                 0.1.1
conda                                23.3.1
conda-content-trust                  0.1.3
conda-pack                           0.6.0
conda-package-handling               2.2.0
conda_package_streaming              0.9.0
conda-token                          0.4.0
constantly                           15.1.0
contextlib2                          21.6.0
contourpy                            1.2.0
cookiecutter                         2.5.0
cryptography                         42.0.2
cssselect                            1.1.0
cycler                               0.11.0
Cython                               0.29.32
cytoolz                              0.11.0
daal4py                              2021.6.0
dask                                 2022.7.0
datashader                           0.14.1
datashape                            0.5.4
debugpy                              1.5.1
decorator                            5.1.1
defusedxml                           0.7.1
diff-match-patch                     20200713
dill                                 0.3.8
distributed                          2022.7.0
docker                               6.1.3
docutils                             0.16
dparse                               0.6.4b0
entrypoints                          0.4
et-xmlfile                           1.1.0
exceptiongroup                       1.2.0
executing                            2.0.1
fastjsonschema                       2.16.2
filelock                             3.6.0
flake8                               4.0.1
Flask                                3.0.1
fonttools                            4.25.0
fqdn                                 1.5.1
fsspec                               2022.7.1
future                               0.18.3
gensim                               4.1.2
glob2                                0.7
gmpy2                                2.1.2
google-pasta                         0.2.0
greenlet                             1.1.1
gssapi                               1.8.3
h5py                                 3.7.0
hdijupyterutils                      0.20.4
HeapDict                             1.0.1
holoviews                            1.15.0
hvplot                               0.8.0
hyperlink                            21.0.0
idna                                 3.3
imagecodecs                          2021.8.26
imageio                              2.19.3
imagesize                            1.4.1
importlib-metadata                   4.11.3
incremental                          22.10.0
inflection                           0.5.1
iniconfig                            1.1.1
intake                               0.6.5
intervaltree                         3.1.0
ipykernel                            6.20.2
ipython                              8.21.0
ipython-genutils                     0.2.0
ipywidgets                           7.6.5
isoduration                          20.11.0
isort                                5.9.3
itemadapter                          0.3.0
itemloaders                          1.0.4
itsdangerous                         2.1.2
jdcal                                1.4.1
jedi                                 0.18.1
jeepney                              0.7.1
jellyfish                            0.9.0
Jinja2                               3.1.3
jinja2-time                          0.2.0
jmespath                             0.10.0
joblib                               1.3.2
json5                                0.9.6
jsonpatch                            1.32
jsonpointer                          2.1
jsonschema                           4.21.1
jsonschema-specifications            2023.12.1
jupyter                              1.0.0
jupyter_client                       8.6.0
jupyter-console                      6.4.3
jupyter_core                         5.7.1
jupyter-events                       0.9.0
jupyter_server                       2.12.5
jupyter_server_terminals             0.5.2
jupyterlab                           3.4.4
jupyterlab-pygments                  0.1.2
jupyterlab-server                    2.10.3
jupyterlab-widgets                   1.0.0
keyring                              23.4.0
kiwisolver                           1.4.2
krb5                                 0.5.1
lazy-object-proxy                    1.6.0
libarchive-c                         2.9
lief                                 0.14.0
llvmlite                             0.38.0
locket                               1.0.0
lxml                                 5.1.0
lz4                                  3.1.3
Markdown                             3.3.4
markdown-it-py                       3.0.0
MarkupSafe                           2.1.4
marshmallow                          3.20.2
matplotlib                           3.5.2
matplotlib-inline                    0.1.6
mccabe                               0.7.0
mdurl                                0.1.2
mistune                              3.0.2
mkl-fft                              1.3.1
mkl-random                           1.2.2
mkl-service                          2.4.0
mock                                 4.0.3
mpmath                               1.3.0
msgpack                              1.0.3
multipledispatch                     0.6.0
multiprocess                         0.70.16
munkres                              1.1.4
mypy-extensions                      0.4.3
nbclassic                            1.0.0
nbclient                             0.5.13
nbconvert                            7.14.2
nbformat                             5.9.2
nest-asyncio                         1.5.5
networkx                             2.8.4
nltk                                 3.7
nose                                 1.3.7
notebook                             6.5.6
notebook_shim                        0.2.3
numba                                0.55.1
numexpr                              2.8.3
numpy                                1.26.3
numpydoc                             1.4.0
olefile                              0.46
openpyxl                             3.0.10
overrides                            7.7.0
packaging                            21.3
pandas                               2.2.0
pandocfilters                        1.5.0
panel                                0.13.1
papermill                            2.5.0
param                                1.12.0
parsel                               1.6.0
parso                                0.8.3
partd                                1.2.0
pathos                               0.3.2
pathspec                             0.9.0
patsy                                0.5.2
pep8                                 1.7.1
pexpect                              4.8.0
pickleshare                          0.7.5
pillow                               10.2.0
pip                                  23.3.2
pkginfo                              1.8.2
platformdirs                         2.5.2
plotly                               5.9.0
pluggy                               1.4.0
ply                                  3.11
pox                                  0.3.4
poyo                                 0.5.0
ppft                                 1.7.6.8
prometheus-client                    0.14.1
prompt-toolkit                       3.0.43
Protego                              0.1.16
protobuf                             4.25.2
psutil                               5.9.0
ptyprocess                           0.7.0
pure-eval                            0.2.2
pure-sasl                            0.6.2
py                                   1.11.0
pyarrow                              15.0.0
pyasn1                               0.4.8
pyasn1-modules                       0.2.8
pycodestyle                          2.8.0
pycosat                              0.6.3
pycparser                            2.21
pyct                                 0.4.8
pycurl                               7.45.1
pydantic                             1.10.14
PyDispatcher                         2.0.5
pydocstyle                           6.1.1
pyerfa                               2.0.0
pyflakes                             2.4.0
pyfunctional                         1.4.3
Pygments                             2.17.2
PyHamcrest                           2.0.2
PyHive                               0.7.0
pykerberos                           1.2.4
pylint                               3.0.3
pyls-spyder                          0.4.0
pyodbc                               4.0.34
pyOpenSSL                            24.0.0
pyparsing                            3.0.9
PyQt5-sip                            12.11.0
pyrsistent                           0.18.0
PySocks                              1.7.1
pyspnego                             0.10.2
pytest                               8.0.0
python-dateutil                      2.8.2
python-json-logger                   2.0.7
python-lsp-black                     1.2.1
python-lsp-jsonrpc                   1.0.0
python-lsp-server                    1.5.0
python-slugify                       5.0.2
python-snappy                        0.6.0
pytz                                 2022.1
pyviz-comms                          2.0.2
PyWavelets                           1.3.0
pyxdg                                0.27
PyYAML                               6.0
pyzmq                                25.1.2
QDarkStyle                           3.0.2
qstylizer                            0.1.10
QtAwesome                            1.0.3
qtconsole                            5.3.2
QtPy                                 2.2.0
queuelib                             1.5.0
referencing                          0.33.0
regex                                2022.7.9
requests                             2.31.0
requests-file                        1.5.1
requests-kerberos                    0.14.0
rfc3339-validator                    0.1.4
rfc3986-validator                    0.1.1
rich                                 13.7.0
rope                                 0.22.0
rpds-py                              0.17.1
rsa                                  4.7.2
Rtree                                0.9.7
ruamel.yaml                          0.17.21
ruamel.yaml.clib                     0.2.6
ruamel-yaml-conda                    0.15.100
s3fs                                 0.4.2
s3transfer                           0.10.0
safety-schemas                       0.0.1
sagemaker                            2.206.0
sagemaker-data-insights              0.3.3
sagemaker-datawrangler               0.4.3
sagemaker-headless-execution-driver  0.0.12
sagemaker-scikit-learn-extension     2.5.0
sagemaker-studio-analytics-extension 0.0.20
sagemaker-studio-sparkmagic-lib      0.1.4
sasl                                 0.3.1
schema                               0.7.5
scikit-image                         0.19.2
scikit-learn                         1.4.0
scikit-learn-intelex                 2021.20221004.171507
scipy                                1.12.0
Scrapy                               2.6.2
seaborn                              0.11.2
SecretStorage                        3.3.1
Send2Trash                           1.8.2
service-identity                     18.1.0
setuptools                           69.0.3
sip                                  6.6.2
six                                  1.16.0
smart-open                           5.2.1
smclarify                            0.5
smdebug-rulesconfig                  1.0.1
sniffio                              1.2.0
snowballstemmer                      2.2.0
sortedcollections                    2.1.0
sortedcontainers                     2.4.0
soupsieve                            2.3.1
sparkmagic                           0.20.4
Sphinx                               5.0.2
sphinxcontrib-applehelp              1.0.2
sphinxcontrib-devhelp                1.0.2
sphinxcontrib-htmlhelp               2.0.0
sphinxcontrib-jsmath                 1.0.1
sphinxcontrib-qthelp                 1.0.3
sphinxcontrib-serializinghtml        1.1.5
spyder                               5.3.3
spyder-kernels                       2.3.3
SQLAlchemy                           1.4.39
stack-data                           0.6.3
statsmodels                          0.13.2
sympy                                1.10.1
tables                               3.6.1
tabulate                             0.8.10
TBB                                  0.2
tblib                                1.7.0
tenacity                             8.0.1
terminado                            0.13.1
testpath                             0.6.0
text-unidecode                       1.3
textdistance                         4.2.1
threadpoolctl                        2.2.0
three-merge                          0.1.1
thrift                               0.16.0
thrift-sasl                          0.4.3
tifffile                             2021.7.2
tinycss                              0.4
tinycss2                             1.2.1
tldextract                           3.2.0
toml                                 0.10.2
tomli                                2.0.1
tomlkit                              0.11.1
toolz                                0.11.2
tornado                              6.4
tqdm                                 4.64.1
traitlets                            5.14.1
Twisted                              23.10.0
typer                                0.9.0
typing_extensions                    4.9.0
tzdata                               2023.4
ujson                                5.9.0
Unidecode                            1.2.0
uri-template                         1.3.0
urllib3                              2.2.0
w3lib                                1.21.0
watchdog                             2.1.6
wcwidth                              0.2.5
webcolors                            1.13
webencodings                         0.5.1
websocket-client                     0.58.0
Werkzeug                             3.0.1
whatthepatch                         1.0.2
wheel                                0.42.0
widgetsnbextension                   3.5.2
wrapt                                1.14.1
wurlitzer                            3.0.2
xarray                               0.20.1
xlrd                                 2.0.1
XlsxWriter                           3.0.3
xyzservices                          2023.10.1
yapf                                 0.31.0
zict                                 2.1.0
zipp                                 3.8.0
zope.interface                       5.4.0
zstandard                            0.15.2

Expected Behavior

Pandas should be imported and be able to read_csv from S3 targets with no failures

Observed Behavior

See above

Product Category

JupyterLab

Feedback Category

Configuration and Setup, Error Handling and Debugging

Other Details

No response

rekha-seshadrinathan commented 4 months ago

Thank you for reporting this issue. Our team is investigating.

HFulcher commented 4 months ago

Hi @rekha-seshadrinathan, has there been any movement on this?

HFulcher commented 3 months ago

Marking this as closed as the issue seems to have resolved itself. Very disappointed in the customer service though. Why advertise this as a place to raise bugs if you aren't going to communicate? Makes me wonder why we pay a premium to use AWS where smaller companies are much more responsive.

rekha-seshadrinathan commented 3 months ago

Apologies for our delayed communication. A fix was deployed on 2/28.