Updated cdms2 version does not work with eofs

lee1043 commented 5 years ago

Describe the bug Updated version of cdms2 does not work with most recent eofs (version below). This issue is critical for modes of variability code, which I am working on it to merge into PMP master branch. @doutriaux1 @dnadeau4 @gleckler1 @ajdawson any idea? Thank you in advance.

# Name                    Version                   Build  Channel
cdms2                     3.1.0.2018.11.14.22.42.g7e8de8f  py36h481b005_0    cdat/label/nightly
eofs                      1.3.0                      py_1    conda-forge

To Reproduce python simple_test.py (use below two files, rename to remove .txt) simple_test.py.txt sample_input.nc.txt

Error message:

Traceback (most recent call last):
  File "simple_test.py", line 8, in <module>
    solver = Eof(d)
  File "/export_backup/lee1043/anaconda2/envs/pmp_nightly_20181128/lib/python3.6/site-packages/eofs/cdms.py", line 165, in __init__
    self._dataset_name = cdms2_name(dataset).replace(' ', '_')
  File "/export_backup/lee1043/anaconda2/envs/pmp_nightly_20181128/lib/python3.6/site-packages/eofs/tools/cdms.py", line 149, in cdms2_name
    return filter(None, names)[0]
TypeError: 'filter' object is not subscriptable

Expected behavior It should return below image without having any error. It has been okay using previous version of cdms2 as below:

# Name                    Version                   Build  Channel
cdms2                     3.0.2018.08.30.01.24.ge41c556  py27h14c3975_0    cdat/label/nightly

simple_test

Desktop (please complete the following information):

OS: Linux (on Crunchy), Mac

Environment Information

`conda info`

``` active environment : pmp_nightly_20181128 active env location : /export_backup/lee1043/anaconda2/envs/pmp_nightly_20181128 shell level : 2 user config file : /export_backup/lee1043/.condarc populated config files : /export_backup/lee1043/.condarc conda version : 4.5.11 conda-build version : not installed python version : 2.7.13.final.0 base environment : /export_backup/lee1043/anaconda2 (writable) channel URLs : https://repo.anaconda.com/pkgs/main/linux-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/free/linux-64 https://repo.anaconda.com/pkgs/free/noarch https://repo.anaconda.com/pkgs/r/linux-64 https://repo.anaconda.com/pkgs/r/noarch https://repo.anaconda.com/pkgs/pro/linux-64 https://repo.anaconda.com/pkgs/pro/noarch package cache : /export_backup/lee1043/anaconda2/pkgs /export_backup/lee1043/.conda/pkgs envs directories : /export_backup/lee1043/anaconda2/envs /export_backup/lee1043/.conda/envs platform : linux-64 user-agent : conda/4.5.11 requests/2.14.2 CPython/2.7.13 Linux/2.6.32-696.30.1.el6.x86_64 rhel/6.10 glibc/2.12 UID:GID : 55431:1026 netrc file : None offline mode : False ```

`conda list --show-channel-urls`

``` # packages in environment at /export_backup/lee1043/anaconda2/envs/pmp_nightly_20181128: # # Name Version Build Channel asn1crypto 0.24.0 py36_3 cdat-forge blas 1.0 mkl defaults bokeh 0.13.0 py36_0 cdat-forge bzip2 1.0.6 h470a237_2 cdat-forge ca-certificates 2018.8.24 ha4d7672_0 conda-forge cdat_info 8.0.2018.10.30.14.29.gc2e9630 py36_0 cdat/label/nightly cdms2 3.1.0.2018.11.14.22.42.g7e8de8f py36h481b005_0 cdat/label/nightly cdp 1.3.3.2018.11.14.15.52 py36_0 cdat/label/nightly cdtime 3.1.0.2018.11.01.17.45.gb8c370e py36h9ac9557_0 cdat/label/nightly cdutil 8.0.2018.11.12.19.20.g3bd6383 py_0 cdat/label/nightly certifi 2018.8.24 py36_1 conda-forge cffi 1.11.5 py36h5e8e0c9_1 cdat-forge chardet 3.0.4 py36_3 cdat-forge cia 0.0.6 0 cdat-forge clapack 3.2.1 h470a237_1 cdat-forge click 6.7 py_1 cdat-forge cloudpickle 0.5.5 py_0 cdat-forge cryptography 2.3.1 py36hdffb7b8_0 cdat-forge cryptography-vectors 2.3.1 py36_0 cdat-forge curl 7.61.0 h93b3f91_2 cdat-forge cycler 0.10.0 py36h93f1223_0 defaults cytoolz 0.9.0.1 py36h470a237_0 cdat-forge dask 0.19.1 py_0 cdat-forge dask-core 0.19.1 py_0 cdat-forge dbus 1.13.2 h714fa37_1 defaults decorator 4.3.0 py36_0 defaults distarray 2.12.2 py36_0 cdat-forge distributed 1.23.1 py36_0 cdat-forge dv3d 8.0.2018.10.25.14.21.g4afbd51 py_0 cdat/label/nightly eofs 1.3.0 py_1 conda-forge esmf 7.1.0r 1 cdat-forge esmpy 7.1.0r py36_1 cdat-forge expat 2.2.5 hfc679d8_2 cdat-forge ffmpeg 4.0 hc8c182b_0 cdat-forge fontconfig 2.12.6 h49f89f6_0 defaults freetype 2.8.1 hfa320df_1 cdat-forge future 0.16.0 py36_2 cdat-forge g2clib 1.6.0 3 cdat-forge genutil 8.1.2018.07.24.00.41.gc18c308.npy1.14 py36hb3ffb1f_0 cdat/label/nightly glib 2.56.1 h000015b_0 defaults gmp 6.1.2 hfc679d8_0 cdat-forge gnutls 3.5.19 h2a4e5f8_1 cdat-forge gst-plugins-base 1.14.0 hbbd80ab_1 defaults gstreamer 1.14.0 hb453b48_1 defaults hdf4 4.2.13 h951d187_2 cdat-forge hdf5 1.10.1 2 cdat-forge heapdict 1.0.0 py36_0 cdat-forge icu 58.2 h9c2bf20_1 defaults idna 2.7 py36_2 cdat-forge intel-openmp 2018.0.3 0 cdat-forge ipython_genutils 0.2.0 py36_0 defaults jasper 1.900.1 hff1ad4c_5 cdat-forge jinja2 2.10 py_1 cdat-forge jpeg 9c h470a237_1 cdat-forge jsonschema 2.6.0 py36_0 defaults jupyter_core 4.4.0 py36_0 defaults kiwisolver 1.0.1 py36hf484d3e_0 defaults krb5 1.14.6 0 cdat-forge lapack 3.6.1 1 cdat-forge libcdms 3.0.2018.10.17.21.09.g4f79235 h9ac9557_0 cdat/label/nightly libcf 3.0.2018.06.28.01.19.g994aa30 py36h14c3975_0 cdat/label/nightly libdrs 3.0.2018.10.18.21.40.g36264ac h9ac9557_0 cdat/label/nightly libdrs_f 3.0.2018.10.18.21.40.g36264ac h9ac9557_0 cdat/label/nightly libffi 3.2.1 hfc679d8_5 cdat-forge libgcc 7.2.0 h69d50b8_2 cdat-forge libgcc-ng 8.2.0 hdf63c60_1 defaults libgfortran 3.0.0 1 cdat-forge libgfortran-ng 7.2.0 hdf63c60_3 cdat-forge libiconv 1.15 h470a237_3 cdat-forge libnetcdf 4.6.1 2 cdat-forge libpng 1.6.35 ha92aebf_2 cdat-forge libssh2 1.8.0 h5b517e9_2 cdat-forge libstdcxx-ng 7.2.0 hdf63c60_3 cdat-forge libtiff 4.0.9 he6b73bb_2 cdat-forge libxcb 1.13 h1bed415_1 defaults libxml2 2.9.8 h26e45fe_1 defaults locket 0.2.0 py_2 cdat-forge markupsafe 1.0 py36h470a237_1 cdat-forge matplotlib 2.2.2 py36h0e671d2_1 defaults mkl 2018.0.3 1 cdat-forge mkl_fft 1.0.6 py36_0 cdat-forge mkl_random 1.0.1 py36_0 cdat-forge modes_of_variability-v0.1-68 g37ed74e modes_of_variability-v0.1-70 gdbc6e4b modes_of_variability-v0.1-71 ga2c8992 mpi 1.0 mpich cdat-forge mpich 3.2.1 h26a2512_4 cdat-forge msgpack-python 0.5.6 py36h2d50403_2 cdat-forge nbformat 4.4.0 py36_0 defaults ncurses 6.1 hfc679d8_1 cdat-forge netcdf-fortran 4.4.4 7 cdat-forge nettle 3.3 0 cdat-forge numpy 1.15.0 py36h1b885b7_0 cdat-forge numpy-base 1.15.0 py36h3dfced4_0 cdat-forge openssl 1.0.2p h470a237_0 conda-forge ossuuid 1.6.2 hfc679d8_0 cdat-forge output_viewer 1.2.5 py36_0 cdat-forge packaging 17.1 py_0 cdat-forge pandas 0.23.4 py36hf8a1672_0 cdat-forge partd 0.3.8 py_1 cdat-forge pcmdi_metrics 1.2.2018.11.15.19.02.g9cf5870 pyh39e3cac_0 pcmdi/label/nightly pcre 8.42 h439df22_0 defaults pip 18.0 py36_1 cdat-forge proj4 4.9.3 h470a237_8 cdat-forge psutil 5.4.7 py36h470a237_1 cdat-forge pycparser 2.18 py_1 cdat-forge pyopenssl 18.0.0 py36_0 cdat-forge pyparsing 2.2.0 py_1 cdat-forge pyqt 5.9.2 py36h751905a_0 defaults pysocks 1.6.8 py36_2 cdat-forge python 3.6.6 h5001a0f_0 cdat-forge python-dateutil 2.7.3 py_0 cdat-forge pytz 2018.5 py_0 cdat-forge pyyaml 3.13 py36h470a237_1 cdat-forge qt 5.9.5 h7e424d6_0 defaults readline 7.0 haf1bffa_1 cdat-forge requests 2.19.1 py36_1 cdat-forge setuptools 40.2.0 py36_0 cdat-forge sip 4.19.8 py36hf484d3e_0 defaults six 1.11.0 py36_1 cdat-forge sortedcontainers 2.0.4 py_1 cdat-forge sqlite 3.24.0 h2f33b56_1 cdat-forge tblib 1.3.2 py_1 cdat-forge tk 8.6.8 0 cdat-forge toolz 0.9.0 py_0 cdat-forge tornado 5.1 py36h470a237_1 cdat-forge traitlets 4.3.2 py36h674d592_0 defaults udunits2 2.2.27.6 h3a4f0e9_1 cdat-forge urllib3 1.23 py36_1 cdat-forge vcs 8.0.2018.11.12.23.15.gd5511950 py_0 cdat/label/nightly vcsaddons 8.0.2018.11.06.13.49.gca89c17 py36h14c3975_0 cdat/label/nightly vtk-cdat 8.0.1.8.0 py36_2 cdat/label/nightly wheel 0.31.1 py36_1 cdat-forge x264 20131218 0 cdat-forge xz 5.2.4 h470a237_1 cdat-forge yaml 0.1.7 h470a237_1 cdat-forge zict 0.1.3 py_0 cdat-forge zlib 1.2.11 h470a237_3 cdat-forge ```

ajdawson commented 5 years ago

I think this is a Python2 vs Python3 issue rather than latest cdms. It is a Python 3 compatibility issue in eofs, specifically the cdms interface in eofs. The cdms code in eofs has not been tested on Python 3 so there are bound to be a few problems. Please feel free to close this and open a new issue on the eofs repository.

lee1043 commented 5 years ago

@ajdawson thank you for your opinion. I will close this issue here and open there at eofs repository.

CDAT / cdms

Updated cdms2 version does not work with eofs #294

Environment Information