search

conda-forge / sparkmagic-feedstock

A conda-smithy repository for sparkmagic.
BSD 3-Clause "New" or "Revised" License
1 stars 9 forks source link

`pip check` failure for environments with sparkmagic installed #36

Closed just4brown closed 1 year ago

just4brown commented 1 year ago

Solution to issue cannot be found in the documentation.

Issue

Hi!

When installing sparkmagic from conda-forge, the pip check command fails and reports an inconsistent environment.

Why it matters: when creating new recipes for conda-forge, its recommended to use pip check as a sanity test for the installation. Most recipes that depend on sparkmagic will likely fail the sanity test because of this inconsistency.

Repro:

conda create --name sparkmagic-test
conda activate sparkmagic-test
conda install conda-forge::sparkmagic -y
pip check
sparkmagic 0.21.0 has requirement pandas<2.0.0,>=0.17.1, but you have pandas 2.0.3.

Installed packages

# packages in environment at :
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main
_openmp_mutex             5.1                       1_gnu
aiofiles                  22.1.0          py311h06a4308_0
aiosqlite                 0.18.0          py311h06a4308_0
anyio                     3.5.0           py311h06a4308_0
argon2-cffi               21.3.0             pyhd3eb1b0_0
argon2-cffi-bindings      21.2.0          py311h5eee18b_0
asttokens                 2.0.5              pyhd3eb1b0_0
attrs                     22.1.0          py311h06a4308_0
autovizwidget             0.20.0          py311h06a4308_0
babel                     2.11.0          py311h06a4308_0
backcall                  0.2.0              pyhd3eb1b0_0
beautifulsoup4            4.12.2          py311h06a4308_0
blas                      1.0                         mkl
bleach                    4.1.0              pyhd3eb1b0_0
bottleneck                1.3.5           py311hbed6279_0
brotlipy                  0.7.0           py311h5eee18b_1002
bzip2                     1.0.8                h7b6447c_0
ca-certificates           2023.08.22           h06a4308_0
certifi                   2023.7.22       py311h06a4308_0
cffi                      1.15.1          py311h5eee18b_3
charset-normalizer        2.0.4              pyhd3eb1b0_0
comm                      0.1.2           py311h06a4308_0
cryptography              41.0.3          py311hdda0065_0
cyrus-sasl                2.1.28               h52b45da_1
dbus                      1.13.18              hb2f20db_0
debugpy                   1.6.7           py311h6a678d5_0
decorator                 5.1.1              pyhd3eb1b0_0
defusedxml                0.7.1              pyhd3eb1b0_0
entrypoints               0.4             py311h06a4308_0
executing                 0.8.3              pyhd3eb1b0_0
expat                     2.5.0                h6a678d5_0
fontconfig                2.14.1               h4c34cd2_2
freetype                  2.12.1               h4a9f257_0
giflib                    5.2.1                h5eee18b_3
glib                      2.69.1               he621ea3_2
gst-plugins-base          1.14.1               h6a678d5_1
gstreamer                 1.14.1               h5eee18b_1
hdijupyterutils           0.20.0          py311h06a4308_0
icu                       58.2                 he6710b0_3
idna                      3.4             py311h06a4308_0
intel-openmp              2023.1.0         hdb19cb5_46305
ipykernel                 6.25.0          py311h92b7b1e_0
ipython                   8.15.0          py311h06a4308_0
ipython_genutils          0.2.0              pyhd3eb1b0_1
ipywidgets                8.0.4           py311h06a4308_0
jedi                      0.18.1          py311h06a4308_1
jinja2                    3.1.2           py311h06a4308_0
jpeg                      9e                   h5eee18b_1
json5                     0.9.6              pyhd3eb1b0_0
jsonschema                4.17.3          py311h06a4308_0
jupyter                   1.0.0           py311h06a4308_8
jupyter_client            7.4.9           py311h06a4308_0
jupyter_console           6.6.3           py311h06a4308_0
jupyter_core              5.3.0           py311h06a4308_0
jupyter_events            0.6.3           py311h06a4308_0
jupyter_server            1.23.4          py311h06a4308_0
jupyter_server_fileid     0.9.0           py311h06a4308_0
jupyter_server_ydoc       0.8.0           py311h06a4308_1
jupyter_ydoc              0.2.4           py311h06a4308_0
jupyterlab                3.6.3           py311h06a4308_0
jupyterlab_pygments       0.1.2                      py_0
jupyterlab_server         2.22.0          py311h06a4308_0
jupyterlab_widgets        3.0.5           py311h06a4308_0
krb5                      1.20.1               h143b758_1
ld_impl_linux-64          2.38                 h1181459_1
lerc                      3.0                  h295c915_0
libclang                  14.0.6          default_hc6dbbc7_1
libclang13                14.0.6          default_he11475f_1
libcups                   2.4.2                h2d74bed_1
libdeflate                1.17                 h5eee18b_0
libedit                   3.1.20221030         h5eee18b_0
libevent                  2.1.12               hdbd6064_1
libffi                    3.4.4                h6a678d5_0
libgcc-ng                 11.2.0               h1234567_1
libgomp                   11.2.0               h1234567_1
libllvm14                 14.0.6               hdb19cb5_3
libpng                    1.6.39               h5eee18b_0
libpq                     12.15                hdbd6064_1
libsodium                 1.0.18               h7b6447c_0
libstdcxx-ng              11.2.0               h1234567_1
libtiff                   4.5.1                h6a678d5_0
libuuid                   1.41.5               h5eee18b_0
libwebp                   1.3.2                h11a3e52_0
libwebp-base              1.3.2                h5eee18b_0
libxcb                    1.15                 h7f8727e_0
libxkbcommon              1.0.1                h5eee18b_1
libxml2                   2.10.4               hcbfbd50_0
libxslt                   1.1.37               h2085143_0
lxml                      4.9.3           py311hdbbb534_0
lz4-c                     1.9.4                h6a678d5_0
markupsafe                2.1.1           py311h5eee18b_0
matplotlib-inline         0.1.6           py311h06a4308_0
mistune                   0.8.4           py311h5eee18b_1000
mkl                       2023.1.0         h213fc3f_46343
mkl-service               2.4.0           py311h5eee18b_1
mkl_fft                   1.3.8           py311h5eee18b_0
mkl_random                1.2.4           py311hdb19cb5_0
mock                      4.0.3              pyhd3eb1b0_0
mysql                     5.7.24               h721c034_2
nbclassic                 0.5.5           py311h06a4308_0
nbclient                  0.5.13          py311h06a4308_0
nbconvert                 6.5.4           py311h06a4308_0
nbformat                  5.9.2           py311h06a4308_0
ncurses                   6.4                  h6a678d5_0
nest-asyncio              1.5.6           py311h06a4308_0
nose                      1.3.7           pyhd3eb1b0_1008
notebook                  6.5.4           py311h06a4308_1
notebook-shim             0.2.2           py311h06a4308_0
nspr                      4.35                 h6a678d5_0
nss                       3.89.1               h6a678d5_0
numexpr                   2.8.4           py311h65dcdc2_1
numpy                     1.25.2          py311h08b1b3b_0
numpy-base                1.25.2          py311hf175353_0
openssl                   3.0.10               h7f8727e_2
packaging                 23.1            py311h06a4308_0
pandas                    2.0.3           py311ha02d727_0
pandocfilters             1.5.0              pyhd3eb1b0_0
parso                     0.8.3              pyhd3eb1b0_0
pcre                      8.45                 h295c915_0
pexpect                   4.8.0              pyhd3eb1b0_3
pickleshare               0.7.5           pyhd3eb1b0_1003
pip                       23.2.1          py311h06a4308_0
platformdirs              3.10.0          py311h06a4308_0
plotly                    5.9.0           py311h06a4308_0
ply                       3.11            py311h06a4308_0
prometheus_client         0.14.1          py311h06a4308_0
prompt-toolkit            3.0.36          py311h06a4308_0
prompt_toolkit            3.0.36               hd3eb1b0_0
psutil                    5.9.0           py311h5eee18b_0
ptyprocess                0.7.0              pyhd3eb1b0_2
pure_eval                 0.2.2              pyhd3eb1b0_0
pycparser                 2.21               pyhd3eb1b0_0
pygments                  2.15.1          py311h06a4308_1
pyopenssl                 23.2.0          py311h06a4308_0
pyqt                      5.15.7          py311h6a678d5_0
pyqt5-sip                 12.11.0         py311h6a678d5_0
pyrsistent                0.18.0          py311h5eee18b_0
pysocks                   1.7.1           py311h06a4308_0
pyspnego                  0.8.0           py311h5eee18b_2
python                    3.11.5               h955ad1f_0
python-dateutil           2.8.2              pyhd3eb1b0_0
python-fastjsonschema     2.16.2          py311h06a4308_0
python-json-logger        2.0.7           py311h06a4308_0
python-tzdata             2023.3             pyhd3eb1b0_0
pytz                      2023.3.post1    py311h06a4308_0
pyyaml                    6.0             py311h5eee18b_1
pyzmq                     23.2.0          py311h6a678d5_0
qt-main                   5.15.2               h7358343_9
qt-webengine              5.15.9               h9ab4d14_7
qtconsole                 5.4.2           py311h06a4308_0
qtpy                      2.2.0           py311h06a4308_0
qtwebkit                  5.212                h3fafdc1_5
readline                  8.2                  h5eee18b_0
requests                  2.31.0          py311h06a4308_0
requests-kerberos         0.14.0             pyhd3eb1b0_0
rfc3339-validator         0.1.4           py311h06a4308_0
rfc3986-validator         0.1.1           py311h06a4308_0
ruamel.yaml               0.17.21         py311h5eee18b_0
send2trash                1.8.0              pyhd3eb1b0_1
setuptools                68.0.0          py311h06a4308_0
sip                       6.6.2           py311h6a678d5_0
six                       1.16.0             pyhd3eb1b0_1
sniffio                   1.2.0           py311h06a4308_1
soupsieve                 2.4             py311h06a4308_0
sparkmagic                0.21.0             pyhd8ed1ab_0    conda-forge
sqlite                    3.41.2               h5eee18b_0
stack_data                0.2.0              pyhd3eb1b0_0
tbb                       2021.8.0             hdb19cb5_0
tenacity                  8.2.2           py311h06a4308_0
terminado                 0.17.1          py311h06a4308_0
tinycss2                  1.2.1           py311h06a4308_0
tk                        8.6.12               h1ccaba5_0
toml                      0.10.2             pyhd3eb1b0_0
tornado                   6.3.2           py311h5eee18b_0
traitlets                 5.7.1           py311h06a4308_0
typing-extensions         4.7.1           py311h06a4308_0
typing_extensions         4.7.1           py311h06a4308_0
tzdata                    2023c                h04d1e81_0
urllib3                   1.26.16         py311h06a4308_0
wcwidth                   0.2.5              pyhd3eb1b0_0
webencodings              0.5.1           py311h06a4308_1
websocket-client          0.58.0          py311h06a4308_4
wheel                     0.38.4          py311h06a4308_0
widgetsnbextension        4.0.5           py311h06a4308_0
xz                        5.4.2                h5eee18b_0
y-py                      0.5.9           py311h52d8a92_0
yaml                      0.2.5                h7b6447c_0
ypy-websocket             0.8.2           py311h06a4308_0
zeromq                    4.3.4                h2531618_0
zlib                      1.2.13               h5eee18b_0
zstd                      1.5.5                hc292b87_0

Environment info

active environment : sparkmagic
    active env location : /home/jubrownd/miniconda3/envs/sparkmagic
            shell level : 2
       user config file : /home/jubrownd/.condarc
 populated config files :
          conda version : 23.1.0
    conda-build version : not installed
         python version : 3.10.9.final.0
       virtual packages : __archspec=1=x86_64
                          __glibc=2.26=0
                          __linux=5.4.254=0
                          __unix=0=0
       base environment : /home/jubrownd/miniconda3  (writable)
      conda av data dir : /home/jubrownd/miniconda3/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /home/jubrownd/miniconda3/pkgs
                          /home/jubrownd/.conda/pkgs
       envs directories : /home/jubrownd/miniconda3/envs
                          /home/jubrownd/.conda/envs
               platform : linux-64
             user-agent : conda/23.1.0 requests/2.24.0 CPython/3.10.9 Linux/5.4.254-175.358.amzn2int.x86_64 amzn/2 glibc/2.26
                UID:GID : 11326493:100
             netrc file : None
           offline mode : False
devstein commented 1 year ago

Thanks for flagging this @just4brown!

Just to make sure I understand, the quick fix would be to pin this feedstock to pandas<2.0.0 like we did in the Sparkmagic repository?

just4brown commented 1 year ago

Correct, I believe the discrepancy in the feedstock's requirements constraint is allowing conda to resolve a newer version for pandas than sparkmagic can allow.

Related question (if you know the answer), is there an estimate for when the newer major version of pandas will be supported by sparkmagic?

devstein commented 1 year ago

@just4brown Ok I can update!

Related question (if you know the answer), is there an estimate for when the newer major version of pandas will be supported by sparkmagic?

No I'm stretched for time right now and have not investigated. If you can take a stab at it or even narrow down the breaking change(s), I can make sure it gets released

devstein commented 1 year ago

@just4brown Can you let me know if the issue is fixed for you?

just4brown commented 1 year ago

@devstein confirmed pip check passes now. Thanks!