search

CCI-Tools / cate

ESA CCI Toolbox (Cate)
MIT License
51 stars 15 forks source link

ModuleNotFoundError: No module named 'shapefile' #397

Open forman opened 7 years ago

forman commented 7 years ago

After installation (current 0.9.0.dev7, Windows 10) and in the Cate CLI:

> cate -h
error: unexpected exception while loading Cate plugin with entry point 'cate_ds'
--------------------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\Norman\cate-0.9.0.dev7\lib\site-packages\cate\core\plugin.py", line 88, in _load_plugins
    plugin = entry_point.load()
  File "C:\Users\Norman\cate-0.9.0.dev7\lib\site-packages\pkg_resources\__init__.py", line 2340, in load
    self.require(*args, **kwargs)
  File "C:\Users\Norman\cate-0.9.0.dev7\lib\site-packages\pkg_resources\__init__.py", line 2363, in require
    items = working_set.resolve(reqs, env, installer, extras=self.extras)
  File "C:\Users\Norman\cate-0.9.0.dev7\lib\site-packages\pkg_resources\__init__.py", line 867, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'owslib' distribution was not found and is required by the application
--------------------------------------------------------------------------------
error: unexpected exception while loading Cate plugin with entry point 'cate_ops'
--------------------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\Norman\cate-0.9.0.dev7\lib\site-packages\cate\core\plugin.py", line 88, in _load_plugins
    plugin = entry_point.load()
  File "C:\Users\Norman\cate-0.9.0.dev7\lib\site-packages\pkg_resources\__init__.py", line 2340, in load
    self.require(*args, **kwargs)
  File "C:\Users\Norman\cate-0.9.0.dev7\lib\site-packages\pkg_resources\__init__.py", line 2363, in require
    items = working_set.resolve(reqs, env, installer, extras=self.extras)
  File "C:\Users\Norman\cate-0.9.0.dev7\lib\site-packages\pkg_resources\__init__.py", line 867, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'owslib' distribution was not found and is required by the application
--------------------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\Norman\cate-0.9.0.dev7\Scripts\cate-script.py", line 6, in <module>
    from cate.cli.main import main
  File "C:\Users\Norman\cate-0.9.0.dev7\lib\site-packages\cate\cli\main.py", line 116, in <module>
    from cate.ops.io import open_dataset
  File "C:\Users\Norman\cate-0.9.0.dev7\lib\site-packages\cate\ops\__init__.py", line 56, in <module>
    from .plot import plot_map, plot, plot_data_frame
  File "C:\Users\Norman\cate-0.9.0.dev7\lib\site-packages\cate\ops\plot.py", line 75, in <module>
    import cartopy.crs as ccrs
  File "C:\Users\Norman\cate-0.9.0.dev7\lib\site-packages\cartopy\__init__.py", line 111, in <module>
    import cartopy.feature
  File "C:\Users\Norman\cate-0.9.0.dev7\lib\site-packages\cartopy\feature.py", line 32, in <module>
    import cartopy.io.shapereader as shapereader
  File "C:\Users\Norman\cate-0.9.0.dev7\lib\site-packages\cartopy\io\shapereader.py", line 52, in <module>
    import shapefile
ModuleNotFoundError: No module named 'shapefile'

Coming from cartopy which requires pyshp, which should actually be available:

> conda list
# packages in environment at C:\Users\Norman\cate-0.9.0.dev7:
#
asn1crypto                0.22.0                   py36_0    conda-forge
bkcharts                  0.2                      py36_0    conda-forge
bokeh                     0.12.7                   py36_0
bottleneck                1.2.1                    py36_1    conda-forge
ca-certificates           2017.7.27.1                   0    conda-forge
cartopy                   0.15.1                   py36_4    conda-forge
cate-cli                  0.9.0.dev7               py36_0    ccitools
certifi                   2017.7.27.1              py36_0    conda-forge
cffi                      1.10.0                   py36_0    conda-forge
chardet                   3.0.4                    py36_0    conda-forge
click                     6.7                      py36_0    conda-forge
click-plugins             1.0.3                    py36_0    conda-forge
cligj                     0.4.0                    py36_0    conda-forge
cloudpickle               0.4.0                    py36_0    conda-forge
conda                     4.3.25                   py36_0
conda-env                 2.6.0                         0    conda-forge
cryptography              2.0.3                    py36_0    conda-forge
curl                      7.52.1                   vc14_0  [vc14]
cycler                    0.10.0                   py36_0    conda-forge
cython                    0.26.1                   py36_0    conda-forge
dask                      0.15.2                   py36_0    conda-forge
dask-core                 0.15.2                   py36_0    conda-forge
descartes                 1.1.0                    py36_0    conda-forge
distributed               1.18.3                   py36_0    conda-forge
expat                     2.2.1                    vc14_0  [vc14]  conda-forge
fiona                     1.7.9                    py36_1    conda-forge
freetype                  2.7                      vc14_1  [vc14]  conda-forge
freexl                    1.0.2                    vc14_2  [vc14]  conda-forge
gdal                      2.1.3               py36_vc14_5  [vc14]  conda-forge
geopandas                 0.3.0                    py36_0    conda-forge
geos                      3.5.1                    vc14_1  [vc14]  conda-forge
h5netcdf                  0.4.2                      py_0    conda-forge
h5py                      2.7.1                    py36_1    conda-forge
hdf4                      4.2.12                   vc14_0  [vc14]  conda-forge
hdf5                      1.8.18                   vc14_0  [vc14]  conda-forge
heapdict                  1.0.0                    py36_1
icu                       58.1                     vc14_1  [vc14]  conda-forge
idna                      2.6                      py36_0
jdcal                     1.3                      py36_0    conda-forge
jinja2                    2.9.6                    py36_0    conda-forge
jpeg                      9b                       vc14_0  [vc14]  conda-forge
kealib                    1.4.7                    vc14_2  [vc14]  conda-forge
krb5                      1.14.2                   vc14_0  [vc14]  conda-forge
libiconv                  1.14                     vc14_4  [vc14]  conda-forge
libnetcdf                 4.4.1.1                  vc14_5  [vc14]  conda-forge
libpng                    1.6.30                   vc14_1  [vc14]
libpq                     9.6.3                    vc14_0  [vc14]  conda-forge
libspatialindex           1.8.5                    vc14_1  [vc14]  conda-forge
libspatialite             4.3.0a                  vc14_15  [vc14]  conda-forge
libtiff                   4.0.6                    vc14_7  [vc14]  conda-forge
libxml2                   2.9.5                    vc14_0  [vc14]  conda-forge
libxslt                   1.1.29                   vc14_5  [vc14]  conda-forge
llvmlite                  0.20.0                   py36_0
locket                    0.2.0                    py36_1    conda-forge
lxml                      4.0.0                    py36_0    conda-forge
markupsafe                1.0                      py36_0    conda-forge
matplotlib                2.0.2                    py36_2    conda-forge
menuinst                  1.4.7                    py36_0
mkl                       2017.0.1                      0
msgpack-python            0.4.8                    py36_0    conda-forge
munch                     2.2.0                    py36_0    conda-forge
netcdf4                   1.2.9                    py36_1    conda-forge
numba                     0.35.0              np113py36_0
numpy                     1.13.1                   py36_0
olefile                   0.44                     py36_0    conda-forge
openjpeg                  2.1.2                    vc14_3  [vc14]  conda-forge
openssl                   1.0.2l                   vc14_0  [vc14]  conda-forge
owslib                    0.15.0                     py_0    conda-forge
pandas                    0.20.3                   py36_1    conda-forge
partd                     0.3.8                    py36_0    conda-forge
pcre                      8.39                     vc14_0  [vc14]  conda-forge
pillow                    4.2.1                    py36_0    conda-forge
proj4                     4.9.3                    vc14_4  [vc14]  conda-forge
psutil                    5.2.2                    py36_0    conda-forge
psycopg2                  2.7.3.1                  py36_0    conda-forge
pycosat                   0.6.2                    py36_0    conda-forge
pycparser                 2.18                     py36_0    conda-forge
pyepsg                    0.3.2                    py36_0    conda-forge
pyopenssl                 17.2.0                   py36_0    conda-forge
pyparsing                 2.2.0                    py36_0    conda-forge
pyproj                    1.9.5.1                  py36_0    conda-forge
pyqt                      5.6.0                    py36_4    conda-forge
pysal                     1.13.0                   py36_0    conda-forge
pyshp                     1.2.12                     py_0    conda-forge
pysocks                   1.6.7                    py36_0    conda-forge
python                    3.6.2                         0    conda-forge
python-dateutil           2.6.1                    py36_0    conda-forge
pytz                      2017.2                   py36_0    conda-forge
pywin32                   220                      py36_2
pyyaml                    3.12                     py36_1    conda-forge
qt                        5.6.2                    vc14_2  [vc14]  conda-forge
requests                  2.18.4                   py36_1    conda-forge
rtree                     0.8.3                    py36_0    conda-forge
ruamel_yaml               0.11.14                  py36_1
scipy                     0.19.1              np113py36_0
setuptools                36.4.0                   py36_1
shapely                   1.6.1                    py36_1    conda-forge
sip                       4.18                     py36_1    conda-forge
six                       1.11.0                   py36_1    conda-forge
sortedcontainers          1.5.7                    py36_0    conda-forge
sqlalchemy                1.1.13                   py36_0    conda-forge
sqlite                    3.13.0                   vc14_1  [vc14]
tblib                     1.3.2                    py36_0    conda-forge
toolz                     0.8.2                    py36_0    conda-forge
tornado                   4.5.2                    py36_0    conda-forge
urllib3                   1.22                     py36_0    conda-forge
vc                        14                            0    conda-forge
vs2015_runtime            14.0.25420                    0    conda-forge
win_inet_pton             1.0.1                    py36_1    conda-forge
wincertstore              0.2                      py36_0    conda-forge
xarray                    0.9.6                    py36_0    conda-forge
xerces-c                  3.1.4                    vc14_2  [vc14]  conda-forge
yaml                      0.1.6                    vc14_0  [vc14]  conda-forge
zict                      0.1.3                      py_0    conda-forge
zlib                      1.2.11                   vc14_0  [vc14]  conda-forge

> conda list
# packages in environment at C:\Users\Norman\cate-0.9.0.dev7:
#
asn1crypto                0.22.0                   py36_0    conda-forge
bkcharts                  0.2                      py36_0    conda-forge
bokeh                     0.12.7                   py36_0
bottleneck                1.2.1                    py36_1    conda-forge
ca-certificates           2017.7.27.1                   0    conda-forge
cartopy                   0.15.1                   py36_4    conda-forge
cate-cli                  0.9.0.dev7               py36_0    ccitools
certifi                   2017.7.27.1              py36_0    conda-forge
cffi                      1.10.0                   py36_0    conda-forge
chardet                   3.0.4                    py36_0    conda-forge
click                     6.7                      py36_0    conda-forge
click-plugins             1.0.3                    py36_0    conda-forge
cligj                     0.4.0                    py36_0    conda-forge
cloudpickle               0.4.0                    py36_0    conda-forge
conda                     4.3.25                   py36_0
conda-env                 2.6.0                         0    conda-forge
cryptography              2.0.3                    py36_0    conda-forge
curl                      7.52.1                   vc14_0  [vc14]
cycler                    0.10.0                   py36_0    conda-forge
cython                    0.26.1                   py36_0    conda-forge
dask                      0.15.2                   py36_0    conda-forge
dask-core                 0.15.2                   py36_0    conda-forge
descartes                 1.1.0                    py36_0    conda-forge
distributed               1.18.3                   py36_0    conda-forge
expat                     2.2.1                    vc14_0  [vc14]  conda-forge
fiona                     1.7.9                    py36_1    conda-forge
freetype                  2.7                      vc14_1  [vc14]  conda-forge
freexl                    1.0.2                    vc14_2  [vc14]  conda-forge
gdal                      2.1.3               py36_vc14_5  [vc14]  conda-forge
geopandas                 0.3.0                    py36_0    conda-forge
geos                      3.5.1                    vc14_1  [vc14]  conda-forge
h5netcdf                  0.4.2                      py_0    conda-forge
h5py                      2.7.1                    py36_1    conda-forge
hdf4                      4.2.12                   vc14_0  [vc14]  conda-forge
hdf5                      1.8.18                   vc14_0  [vc14]  conda-forge
heapdict                  1.0.0                    py36_1
icu                       58.1                     vc14_1  [vc14]  conda-forge
idna                      2.6                      py36_0
jdcal                     1.3                      py36_0    conda-forge
jinja2                    2.9.6                    py36_0    conda-forge
jpeg                      9b                       vc14_0  [vc14]  conda-forge
kealib                    1.4.7                    vc14_2  [vc14]  conda-forge
krb5                      1.14.2                   vc14_0  [vc14]  conda-forge
libiconv                  1.14                     vc14_4  [vc14]  conda-forge
libnetcdf                 4.4.1.1                  vc14_5  [vc14]  conda-forge
libpng                    1.6.30                   vc14_1  [vc14]
libpq                     9.6.3                    vc14_0  [vc14]  conda-forge
libspatialindex           1.8.5                    vc14_1  [vc14]  conda-forge
libspatialite             4.3.0a                  vc14_15  [vc14]  conda-forge
libtiff                   4.0.6                    vc14_7  [vc14]  conda-forge
libxml2                   2.9.5                    vc14_0  [vc14]  conda-forge
libxslt                   1.1.29                   vc14_5  [vc14]  conda-forge
llvmlite                  0.20.0                   py36_0
locket                    0.2.0                    py36_1    conda-forge
lxml                      4.0.0                    py36_0    conda-forge
markupsafe                1.0                      py36_0    conda-forge
matplotlib                2.0.2                    py36_2    conda-forge
menuinst                  1.4.7                    py36_0
mkl                       2017.0.1                      0
msgpack-python            0.4.8                    py36_0    conda-forge
munch                     2.2.0                    py36_0    conda-forge
netcdf4                   1.2.9                    py36_1    conda-forge
numba                     0.35.0              np113py36_0
numpy                     1.13.1                   py36_0
olefile                   0.44                     py36_0    conda-forge
openjpeg                  2.1.2                    vc14_3  [vc14]  conda-forge
openssl                   1.0.2l                   vc14_0  [vc14]  conda-forge
owslib                    0.15.0                     py_0    conda-forge
pandas                    0.20.3                   py36_1    conda-forge
partd                     0.3.8                    py36_0    conda-forge
pcre                      8.39                     vc14_0  [vc14]  conda-forge
pillow                    4.2.1                    py36_0    conda-forge
proj4                     4.9.3                    vc14_4  [vc14]  conda-forge
psutil                    5.2.2                    py36_0    conda-forge
psycopg2                  2.7.3.1                  py36_0    conda-forge
pycosat                   0.6.2                    py36_0    conda-forge
pycparser                 2.18                     py36_0    conda-forge
pyepsg                    0.3.2                    py36_0    conda-forge
pyopenssl                 17.2.0                   py36_0    conda-forge
pyparsing                 2.2.0                    py36_0    conda-forge
pyproj                    1.9.5.1                  py36_0    conda-forge
pyqt                      5.6.0                    py36_4    conda-forge
pysal                     1.13.0                   py36_0    conda-forge
pyshp                     1.2.12                     py_0    conda-forge
pysocks                   1.6.7                    py36_0    conda-forge
python                    3.6.2                         0    conda-forge
python-dateutil           2.6.1                    py36_0    conda-forge
pytz                      2017.2                   py36_0    conda-forge
pywin32                   220                      py36_2
pyyaml                    3.12                     py36_1    conda-forge
qt                        5.6.2                    vc14_2  [vc14]  conda-forge
requests                  2.18.4                   py36_1    conda-forge
rtree                     0.8.3                    py36_0    conda-forge
ruamel_yaml               0.11.14                  py36_1
scipy                     0.19.1              np113py36_0
setuptools                36.4.0                   py36_1
shapely                   1.6.1                    py36_1    conda-forge
sip                       4.18                     py36_1    conda-forge
six                       1.11.0                   py36_1    conda-forge
sortedcontainers          1.5.7                    py36_0    conda-forge
sqlalchemy                1.1.13                   py36_0    conda-forge
sqlite                    3.13.0                   vc14_1  [vc14]
tblib                     1.3.2                    py36_0    conda-forge
toolz                     0.8.2                    py36_0    conda-forge
tornado                   4.5.2                    py36_0    conda-forge
urllib3                   1.22                     py36_0    conda-forge
vc                        14                            0    conda-forge
vs2015_runtime            14.0.25420                    0    conda-forge
win_inet_pton             1.0.1                    py36_1    conda-forge
wincertstore              0.2                      py36_0    conda-forge
xarray                    0.9.6                    py36_0    conda-forge
xerces-c                  3.1.4                    vc14_2  [vc14]  conda-forge
yaml                      0.1.6                    vc14_0  [vc14]  conda-forge
zict                      0.1.3                      py_0    conda-forge
zlib                      1.2.11                   vc14_0  [vc14]  conda-forge

Specifications

Cate 0.9.0.dev7, Windows 10

forman commented 7 years ago

It is strange, shapefile is the main module of pyshp, see https://pypi.python.org/pypi/pyshp, but:

> python -c "import sys; print(sys.executable)"
C:\Users\Norman\cate-0.9.0.dev7\python.exe

> conda list pyshp
# packages in environment at C:\Users\Norman\cate-0.9.0.dev7:
#
pyshp                     1.2.12                     py_0    conda-forge

> python -c "import shapefile"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'shapefile'
forman commented 7 years ago

Found the reason. There is a site-packages folder in the wrong location which includes the desired module.py:

> cd C:\Users\Norman\cate-0.9.0.dev7\site-packages

> dir
...
23.09.2017  12:08    <DIR>          .
23.09.2017  12:08    <DIR>          ..
23.09.2017  12:05    <DIR>          h5netcdf
23.09.2017  12:05    <DIR>          h5netcdf-0.4.2-py3.6.egg-info
23.09.2017  12:06    <DIR>          owslib
23.09.2017  12:06    <DIR>          OWSLib-0.15.0-py3.6.egg-info
23.09.2017  12:07    <DIR>          pyshp-1.2.12-py3.6.egg-info
24.08.2017  17:57            52.285 shapefile.py
23.09.2017  12:08    <DIR>          zict
23.09.2017  12:08    <DIR>          zict-0.1.3-py3.6.egg-info
forman commented 7 years ago

Google gives:

It seems that the problem is caused by conda when installing packages that usually go via pip. But Cate used to work before.

forman commented 7 years ago

Not really fixed, but my workaround is in cate.core.__init__.py:

extra_path = os.path.join(sys.prefix, 'site-packages')
if os.path.isdir(extra_path) and extra_path not in sys.path:
    sys.path.append(extra_path)