CDAT / cdms

8 stars 10 forks source link

Attempt to import numpy.ma.rank failing with numpy 1.18.1 #384

Closed xylar closed 4 years ago

xylar commented 4 years ago

Describe the bug With the latest cdms2 from conda-forge, I'm getting an import error related to numpy.ma.rank, which appears not to exist.

To Reproduce Steps to reproduce the behavior:

conda create -y -n test -c conda-forge cdms2=3.1.4
conda activate test
python -c "import cdms2.tvariable"

The result is:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/xylar/miniconda3/envs/test/lib/python3.7/site-packages/cdms2/__init__.py", line 8, in <module>
    from . import dataset
  File "/home/xylar/miniconda3/envs/test/lib/python3.7/site-packages/cdms2/dataset.py", line 24, in <module>
    from .coord import FileAxis2D, DatasetAxis2D
  File "/home/xylar/miniconda3/envs/test/lib/python3.7/site-packages/cdms2/coord.py", line 17, in <module>
    from .fvariable import FileVariable
  File "/home/xylar/miniconda3/envs/test/lib/python3.7/site-packages/cdms2/fvariable.py", line 7, in <module>
    from .variable import DatasetVariable
  File "/home/xylar/miniconda3/envs/test/lib/python3.7/site-packages/cdms2/variable.py", line 17, in <module>
    from .avariable import AbstractVariable
  File "/home/xylar/miniconda3/envs/test/lib/python3.7/site-packages/cdms2/avariable.py", line 1933, in <module>
    from . import MV2 as MV  # noqa
  File "/home/xylar/miniconda3/envs/test/lib/python3.7/site-packages/cdms2/MV2.py", line 12, in <module>
    from numpy.ma import indices, innerproduct, masked, put, putmask, rank, ravel  # noqa
ImportError: cannot import name 'rank' from 'numpy.ma' (/home/xylar/miniconda3/envs/test/lib/python3.7/site-packages/numpy/ma/__init__.py)

Expected behavior Import should succeed.

Desktop (please complete the following information):

Environment Information

`conda info`

``` active environment : test active env location : /home/xylar/miniconda3/envs/test shell level : 2 user config file : /home/xylar/.condarc populated config files : /home/xylar/.condarc conda version : 4.8.2 conda-build version : 3.18.11 python version : 3.7.3.final.0 virtual packages : __glibc=2.23 base environment : /home/xylar/miniconda3 (writable) channel URLs : https://conda.anaconda.org/conda-forge/linux-64 https://conda.anaconda.org/conda-forge/noarch https://repo.anaconda.com/pkgs/main/linux-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/r/linux-64 https://repo.anaconda.com/pkgs/r/noarch package cache : /home/xylar/miniconda3/pkgs /home/xylar/.conda/pkgs envs directories : /home/xylar/miniconda3/envs /home/xylar/.conda/envs platform : linux-64 user-agent : conda/4.8.2 requests/2.22.0 CPython/3.7.3 Linux/4.4.0-171-generic ubuntu/16.04.6 glibc/2.23 UID:GID : 1001:1001 netrc file : None offline mode : False ```

`conda config --show-sources`

``` ==> /home/xylar/.condarc <== channel_priority: strict channels: - conda-forge - defaults ```

`conda list --show-channel-urls`

``` # packages in environment at /home/xylar/miniconda3/envs/test: # # Name Version Build Channel _libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 0_gnu conda-forge attrs 19.3.0 py_0 conda-forge bzip2 1.0.8 h516909a_2 conda-forge ca-certificates 2019.11.28 hecc5488_0 conda-forge cdat_info 8.2 py_7 conda-forge cdms2 3.1.4 pypi_0 pypi cdtime 3.1.3 py38h1ac8016_0 conda-forge certifi 2019.11.28 py38_0 conda-forge cffi 1.13.2 py38h8022711_0 conda-forge chardet 3.0.4 py38_1003 conda-forge cryptography 2.8 py38h72c5cf5_1 conda-forge curl 7.65.3 hf8cf82a_0 conda-forge decorator 4.4.1 py_0 conda-forge distarray 2.12.2 py_1 conda-forge esmf 8.0.0 nompi_he6d0a24_4 conda-forge esmpy 8.0.0 nompi_py38hf0e99fa_1 conda-forge future 0.18.2 py38_0 conda-forge g2clib 1.6.0 hf3f1b0b_9 conda-forge hdf4 4.2.13 hf30be14_1003 conda-forge hdf5 1.10.5 nompi_h3c11f04_1104 conda-forge idna 2.8 py38_1000 conda-forge importlib_metadata 1.5.0 py38_0 conda-forge inflect 4.0.0 py38_1 conda-forge ipython_genutils 0.2.0 py_1 conda-forge jaraco.itertools 5.0.0 py_0 conda-forge jasper 1.900.1 h07fcdf6_1006 conda-forge jpeg 9c h14c3975_1001 conda-forge jsonschema 3.2.0 py38_0 conda-forge jupyter_core 4.6.1 py38_0 conda-forge krb5 1.16.4 h2fd8d38_0 conda-forge lazy-object-proxy 1.4.3 py38h516909a_1 conda-forge ld_impl_linux-64 2.33.1 h53a641e_8 conda-forge libblas 3.8.0 11_openblas conda-forge libcblas 3.8.0 11_openblas conda-forge libcdms 3.1.2 ha54dda3_8 conda-forge libcf 1.0.3 py38hdee1361_5 conda-forge libcurl 7.65.3 hda55be3_0 conda-forge libdrs 3.1.2 h9b7ed86_9 conda-forge libdrs_f 3.1.2 h322d3c2_5 conda-forge libedit 3.1.20170329 hf8c457e_1001 conda-forge libffi 3.2.1 he1b5a44_1006 conda-forge libgcc-ng 9.2.0 h24d8f2e_2 conda-forge libgfortran-ng 7.3.0 hdf63c60_5 conda-forge libgomp 9.2.0 h24d8f2e_2 conda-forge liblapack 3.8.0 11_openblas conda-forge libnetcdf 4.7.3 nompi_h9f9fd6a_101 conda-forge libopenblas 0.3.6 h6e990d7_6 conda-forge libpng 1.6.37 hed695b0_0 conda-forge libssh2 1.8.2 h22169c7_2 conda-forge libstdcxx-ng 9.2.0 hdf63c60_2 conda-forge libtiff 4.1.0 hc3755c2_3 conda-forge libuuid 2.32.1 h14c3975_1000 conda-forge lz4-c 1.8.3 he1b5a44_1001 conda-forge more-itertools 8.2.0 py_0 conda-forge nbformat 5.0.4 py_0 conda-forge ncurses 6.1 hf484d3e_1002 conda-forge netcdf-fortran 4.5.2 nompi_h09cde99_103 conda-forge numpy 1.18.1 py38h95a1406_0 conda-forge openblas 0.3.6 h6e990d7_6 conda-forge openssl 1.1.1d h516909a_0 conda-forge pip 20.0.2 py38_1 conda-forge pycparser 2.19 py38_1 conda-forge pyopenssl 19.1.0 py38_0 conda-forge pyrsistent 0.15.7 py38h516909a_0 conda-forge pysocks 1.7.1 py38_0 conda-forge python 3.8.1 h357f687_2 conda-forge readline 8.0 hf8c457e_0 conda-forge regrid2 3.1.4 pypi_0 pypi requests 2.22.0 py38_1 conda-forge setuptools 45.1.0 py38_0 conda-forge six 1.14.0 py38_0 conda-forge sqlite 3.30.1 hcee41ef_0 conda-forge tk 8.6.10 hed695b0_0 conda-forge traitlets 4.3.3 py38_0 conda-forge urllib3 1.25.7 py38_0 conda-forge wheel 0.34.2 py38_0 conda-forge xz 5.2.4 h14c3975_1001 conda-forge zipp 2.1.0 py_0 conda-forge zlib 1.2.11 h516909a_1006 conda-forge zstd 1.4.4 h3b9ef0a_1 conda-forge ```

Additional context I think numpy.rank might work fine, and that numpy.ma.rank was eliminated in the most recent release of numpy.

xylar commented 4 years ago

I can confirm that I do not see this issue with numpy<1.18:

conda create -y -n test -c conda-forge cdms2=3.1.4 numpy=1.17
conda activate test
python -c "import cdms2"

(No output)

muryanto1 commented 4 years ago

fix was not correct -- working on the fix and adding a testcase.

xylar commented 4 years ago

@muryanto1, could you explain what was wrong with the fix?

By the way, my test environment is still picking up cdms2 build 4 (rather than build 7 that we just merged) but I think that's because of a different dependency not being up-to-date with conda-forge pinnings. So as far as I know, https://github.com/conda-forge/cdms2-feedstock/pull/47 should have fixed things from my perspective. But maybe there's something more subtle?

xylar commented 4 years ago

As we discussed, the issue is that numpy.rank isn't the right replacement, and numpy.ndims is actually what's needed.

It's odd to me that the old numpy.ma.rank is equivalent to numpy.ndims but whatever works...

muryanto1 commented 4 years ago

@xylar correct, I am replacing calls to numpy.rank() with numpy.ndims(), @jasonb5 helped me with the fix. we are testing the fix.

xylar commented 4 years ago

I believe this was addressed by #387 and can be closed.