search

matplotlib / basemap

Plot on map projections (with coastlines and political boundaries) using matplotlib
MIT License
775 stars 392 forks source link

addcyclic giving error on map + list #440

Closed xylar closed 5 years ago

xylar commented 5 years ago

The following simple test:

#!/usr/bin/env python

from mpl_toolkits.basemap import addcyclic
import numpy

lon = numpy.arange(0., 360., 0.5)
lat = numpy.arange(-90., 90.5, 0.5)

Lon, Lat = numpy.meshgrid(lon, lat)

field = numpy.sin(numpy.deg2rad(Lat))

fieldCyc, latCyc, lonCyc = addcyclic(field, Lat, Lon)

produces this error:

  File "./test_addcyclic.py", line 13, in <module>
    fieldCyc, lonCyc = addcyclic(field, Lon)
  File "/home/xylar/miniconda3/envs/mpas_analysis_py3/lib/python3.6/site-packages/mpl_toolkits/basemap/__init__.py", line 5111, in addcyclic
    return list(map(_addcyclic,arr[:-1]) + [_addcyclic_lon(arr[-1])])
TypeError: unsupported operand type(s) for +: 'map' and 'list'
Details: My conda environment is as follows: ``` # packages in environment at /home/xylar/miniconda3/envs/mpas_analysis_py3: # # Name Version Build Channel asn1crypto 0.24.0 py36_1003 conda-forge atomicwrites 1.2.1 py_0 conda-forge attrs 18.2.0 py_0 conda-forge basemap 1.2.0 py36h50ae964_0 conda-forge blas 1.1 openblas conda-forge bokeh 1.0.1 py36_1000 conda-forge bottleneck 1.2.1 py36h7eb728f_1 conda-forge bzip2 1.0.6 h470a237_2 conda-forge ca-certificates 2018.10.15 ha4d7672_0 conda-forge certifi 2018.10.15 py36_1000 conda-forge cffi 1.11.5 py36h5e8e0c9_1 conda-forge cftime 1.0.2.1 py36h7eb728f_0 conda-forge chardet 3.0.4 py36_1003 conda-forge click 7.0 py_0 conda-forge cloudpickle 0.6.1 py_0 conda-forge cmocean 1.2 py_0 conda-forge cryptography 2.3.1 py36hdffb7b8_0 conda-forge cryptography-vectors 2.3.1 py36_1000 conda-forge curl 7.62.0 h74213dd_0 conda-forge cycler 0.10.0 py_1 conda-forge cytoolz 0.9.0.1 py36h470a237_1 conda-forge dask 0.20.0 py_0 conda-forge dask-core 0.20.0 py_0 conda-forge dbus 1.13.0 h3a4f0e9_0 conda-forge distributed 1.24.0 py36_1000 conda-forge esmf 7.1.0r h35eb876_3 conda-forge expat 2.2.5 hfc679d8_2 conda-forge fontconfig 2.13.1 h65d0f4c_0 conda-forge freetype 2.9.1 h6debe1e_4 conda-forge geos 3.6.2 hfc679d8_3 conda-forge gettext 0.19.8.1 h5e8e0c9_1 conda-forge glib 2.56.2 h464dc38_1 conda-forge gsl 2.2.1 blas_openblashddceaf2_6 [blas_openblas] conda-forge gst-plugins-base 1.12.5 hde13a9d_0 conda-forge gstreamer 1.12.5 h5856ed1_0 conda-forge h5netcdf 0.6.2 py_0 conda-forge h5py 2.8.0 py36h097b052_4 conda-forge hdf4 4.2.13 h951d187_2 conda-forge hdf5 1.10.3 hc401514_2 conda-forge heapdict 1.0.0 py36_1000 conda-forge icu 58.2 hfc679d8_0 conda-forge idna 2.7 py36_1002 conda-forge jinja2 2.10 py_1 conda-forge jpeg 9c h470a237_1 conda-forge kiwisolver 1.0.1 py36h2d50403_2 conda-forge krb5 1.16.2 hbb41f41_0 conda-forge libcurl 7.62.0 hbdb9355_0 conda-forge libedit 3.1.20170329 haf1bffa_1 conda-forge libffi 3.2.1 hfc679d8_5 conda-forge libgcc-ng 7.2.0 hdf63c60_3 conda-forge libgfortran 3.0.0 1 conda-forge libiconv 1.15 h470a237_3 conda-forge libnetcdf 4.6.1 h9cd6fdc_11 conda-forge libpng 1.6.35 ha92aebf_2 conda-forge libssh2 1.8.0 h5b517e9_2 conda-forge libstdcxx-ng 7.2.0 hdf63c60_3 conda-forge libtiff 4.0.9 he6b73bb_2 conda-forge libuuid 2.32.1 h470a237_2 conda-forge libxcb 1.13 h470a237_2 conda-forge libxml2 2.9.8 h422b904_5 conda-forge libxslt 1.1.32 h88dbc4e_2 conda-forge locket 0.2.0 py_2 conda-forge lxml 4.2.5 py36hc9114bc_0 conda-forge markupsafe 1.1.0 py36h470a237_0 conda-forge matplotlib 2.2.3 py36h8e2386c_0 conda-forge more-itertools 4.3.0 py36_1000 conda-forge mpi 1.0 mpich conda-forge mpich 3.2.1 h26a2512_5 conda-forge msgpack-python 0.5.6 py36h2d50403_3 conda-forge nco 4.7.7 he007dc3_1 conda-forge ncurses 6.1 hfc679d8_1 conda-forge netcdf-fortran 4.4.4 h71ea97b_10 conda-forge netcdf4 1.4.2 py36hac939d9_0 conda-forge numpy 1.15.4 py36_blas_openblashb06ca3d_0 [blas_openblas] conda-forge olefile 0.46 py_0 conda-forge openblas 0.3.3 ha44fe06_1 conda-forge openssl 1.0.2p h470a237_1 conda-forge packaging 18.0 py_0 conda-forge pandas 0.23.4 py36hf8a1672_0 conda-forge partd 0.3.9 py_0 conda-forge pcre 8.41 hfc679d8_3 conda-forge pillow 5.3.0 py36hc736899_0 conda-forge pip 18.1 py36_1000 conda-forge pluggy 0.8.0 py_0 conda-forge progressbar2 3.38.0 py_1 conda-forge proj4 4.9.3 h470a237_8 conda-forge psutil 5.4.8 py36h470a237_0 conda-forge pthread-stubs 0.4 h470a237_1 conda-forge py 1.7.0 py_0 conda-forge pycparser 2.19 py_0 conda-forge pyopenssl 18.0.0 py36_1000 conda-forge pyparsing 2.3.0 py_0 conda-forge pyproj 1.9.5.1 py36h508ed2a_6 conda-forge pyqt 5.6.0 py36h8210e8a_7 conda-forge pyshp 2.0.0 py_0 conda-forge pysocks 1.6.8 py36_1002 conda-forge pytest 3.10.0 py36_1000 conda-forge pytest-runner 4.2 py_1 conda-forge python 3.6.6 h5001a0f_3 conda-forge python-dateutil 2.7.5 py_0 conda-forge python-utils 2.3.0 py_1 conda-forge pytz 2018.7 py_0 conda-forge pyyaml 3.13 py36h470a237_1 conda-forge qt 5.6.2 hf70d934_9 conda-forge readline 7.0 haf1bffa_1 conda-forge requests 2.20.0 py36_1000 conda-forge scipy 1.1.0 py36_blas_openblashb06ca3d_202 [blas_openblas] conda-forge setuptools 40.5.0 py36_0 conda-forge sip 4.18.1 py36hfc679d8_0 conda-forge six 1.11.0 py36_1001 conda-forge sortedcontainers 2.0.5 py_0 conda-forge sqlite 3.25.3 hb1c47c0_0 conda-forge tblib 1.3.2 py_1 conda-forge tk 8.6.8 ha92aebf_0 conda-forge toolz 0.9.0 py_1 conda-forge tornado 5.1.1 py36h470a237_0 conda-forge udunits2 2.2.27.6 h3a4f0e9_1 conda-forge urllib3 1.23 py36_1001 conda-forge wheel 0.32.2 py36_0 conda-forge xarray 0.10.9 py36_0 conda-forge xorg-libxau 1.0.8 h470a237_6 conda-forge xorg-libxdmcp 1.1.2 h470a237_7 conda-forge xz 5.2.4 h470a237_1 conda-forge yaml 0.1.7 h470a237_1 conda-forge zict 0.1.3 py_0 conda-forge zlib 1.2.11 h470a237_3 conda-forge ``` The same test works fine for this environment: ``` # packages in environment at /home/xylar/miniconda3/envs/mpas_analysis: # # Name Version Build Channel alabaster 0.7.10 py36_1 conda-forge asn1crypto 0.24.0 py36_0 conda-forge attrs 18.1.0 py_0 conda-forge babel 2.5.3 py36_0 conda-forge basemap 1.1.0 py36_4 conda-forge blas 1.0 mkl bokeh 0.12.16 py36_0 conda-forge bottleneck 1.2.1 py36_1 conda-forge ca-certificates 2018.4.16 0 conda-forge certifi 2018.4.16 py36_0 conda-forge cffi 1.11.5 py36_0 conda-forge cftime 1.0.0 py36_0 conda-forge chardet 3.0.4 py36_0 conda-forge click 6.7 py_1 conda-forge cloudpickle 0.5.3 py_0 conda-forge cmocean 1.1 py_0 conda-forge commonmark 0.5.4 py_1 conda-forge cryptography 2.2.1 py36_0 conda-forge curl 7.59.0 1 conda-forge cycler 0.10.0 py36_0 conda-forge cytoolz 0.9.0.1 py36_0 conda-forge dask 0.17.5 py_0 conda-forge dask-core 0.17.5 py_0 conda-forge dbus 1.11.0 0 conda-forge distributed 1.21.8 py36_0 conda-forge docutils 0.14 py36_0 conda-forge esmf 7.1.0r 1 conda-forge expat 2.2.5 0 conda-forge fontconfig 2.12.6 0 conda-forge freetype 2.8.1 0 conda-forge future 0.16.0 py36_0 conda-forge geos 3.6.2 1 conda-forge gettext 0.19.8.1 0 conda-forge glib 2.55.0 0 conda-forge gsl 2.2.1 h0c605f7_3 gst-plugins-base 1.8.0 0 conda-forge gstreamer 1.8.0 1 conda-forge gsw 3.2.1 py36_1 conda-forge h5netcdf 0.5.1 py_0 conda-forge h5py 2.8.0 py36h470a237_0 conda-forge hdf4 4.2.13 0 conda-forge hdf5 1.10.1 2 conda-forge heapdict 1.0.0 py36_0 conda-forge icu 58.2 0 conda-forge idna 2.6 py36_1 conda-forge imagesize 1.0.0 py36_0 conda-forge intel-openmp 2018.0.0 8 jinja2 2.10 py36_0 conda-forge jpeg 9b 2 conda-forge kiwisolver 1.0.1 py36_1 conda-forge krb5 1.14.6 0 conda-forge libffi 3.2.1 3 conda-forge libgcc 7.2.0 h69d50b8_2 libgcc-ng 7.2.0 hdf63c60_3 libgfortran 3.0.0 1 libgfortran-ng 7.2.0 hdf63c60_3 libiconv 1.15 0 conda-forge libnetcdf 4.6.1 2 conda-forge libpng 1.6.34 0 conda-forge libssh2 1.8.0 2 conda-forge libstdcxx-ng 7.2.0 hdf63c60_3 libtiff 4.0.9 0 conda-forge libxcb 1.13 0 conda-forge libxml2 2.9.8 0 conda-forge libxslt 1.1.32 0 conda-forge locket 0.2.0 py36_1 conda-forge lxml 4.2.1 py36_0 conda-forge markupsafe 1.0 py36_0 conda-forge matplotlib 2.2.2 py36_1 conda-forge mkl 2018.0.2 1 mkl_fft 1.0.2 py36_0 conda-forge mkl_random 1.0.1 py36_0 conda-forge more-itertools 4.1.0 py_0 conda-forge mpi 1.0 mpich conda-forge mpich 3.2.1 0 conda-forge msgpack-python 0.5.6 py36h2d50403_2 conda-forge nco 4.7.4 1 conda-forge ncurses 5.9 10 conda-forge netcdf-fortran 4.4.4 7 conda-forge netcdf4 1.4.0 py36_0 conda-forge numpy 1.14.2 py36hdbf6ddf_1 numpydoc 0.8.0 py36_0 conda-forge olefile 0.45.1 py36_0 conda-forge openblas 0.2.20 7 conda-forge openssl 1.0.2o 0 conda-forge packaging 17.1 py_0 conda-forge pandas 0.23.0 py36_1 conda-forge partd 0.3.8 py36_0 conda-forge pcre 8.41 1 conda-forge pillow 5.1.0 py36_0 conda-forge pip 9.0.3 py36_0 conda-forge pluggy 0.6.0 py_0 conda-forge progressbar2 3.37.0 py36_0 conda-forge psutil 5.4.5 py36_0 conda-forge py 1.5.3 py_0 conda-forge pycparser 2.18 py36_0 conda-forge pygments 2.2.0 py36_0 conda-forge pyopenssl 18.0.0 py36_0 conda-forge pyparsing 2.2.0 py36_0 conda-forge pyproj 1.9.5.1 py36_0 conda-forge pyqt 5.6.0 py36_5 conda-forge pyshp 1.2.12 py_0 conda-forge pysocks 1.6.8 py36_1 conda-forge pytest 3.5.1 py36_0 conda-forge pytest-runner 4.2 py_0 conda-forge python 3.6.5 1 conda-forge python-dateutil 2.7.3 py_0 conda-forge python-utils 2.3.0 py36_0 conda-forge pytz 2018.4 py_0 conda-forge pyyaml 3.12 py36_1 conda-forge qt 5.6.2 7 conda-forge readline 7.0 0 conda-forge recommonmark 0.4.0 py_1 conda-forge requests 2.18.4 py36_1 conda-forge scipy 1.1.0 py36hfc37229_0 setuptools 39.2.0 py36_0 conda-forge sip 4.18 py36_1 conda-forge six 1.11.0 py36_1 conda-forge snowballstemmer 1.2.1 py36_0 conda-forge sortedcontainers 1.5.10 py36_0 conda-forge sphinx 1.7.4 py36_0 conda-forge sphinx_rtd_theme 0.3.1 py36_0 conda-forge sphinxcontrib-websupport 1.0.1 py36_0 conda-forge sqlite 3.20.1 2 conda-forge tabulate 0.8.2 py_0 conda-forge tblib 1.3.2 py36_0 conda-forge tk 8.6.7 0 conda-forge toolz 0.9.0 py_0 conda-forge tornado 5.0.2 py36_0 conda-forge typing 3.6.4 py36_0 conda-forge udunits2 2.2.26 0 conda-forge urllib3 1.22 py36_0 conda-forge wheel 0.31.0 py36_0 conda-forge xarray 0.10.4 py36_0 conda-forge xorg-libxau 1.0.8 3 conda-forge xorg-libxdmcp 1.1.2 3 conda-forge xz 5.2.3 0 conda-forge yaml 0.1.7 0 conda-forge zict 0.1.3 py_0 conda-forge zlib 1.2.11 0 conda-forge ``` So far, I haven't been able to figure out which package change is responsible, but I will investigate further.
xylar commented 5 years ago

@ocefpaf, the line giving the error is one you edited 7 months ago. I don't think that edit is responsible for the problem but maybe you would have an idea of what's causing this error.

xylar commented 5 years ago

Hmm, I tested with simple conda environments where I installed only python=3.6, numpy (latest) and either basemap 1.2.0 and 1.1.0. Things seem to have broken in 1.2.0, so it does seem like the change from 7 months ago may actually be responsible.

xylar commented 5 years ago

I think the fix is that https://github.com/matplotlib/basemap/blob/master/lib/mpl_toolkits/basemap/__init__.py#L5102 should be:

return list(map(_addcyclic,arr[:-1])) + [_addcyclic_lon(arr[-1])]

That is, the map should be converted to a list and then added to the other list, rather than the map being added to a list and then converted to a list.

I'll make a PR for this as soon as I'm able.

ocefpaf commented 5 years ago

That is, the map should be converted to a list and then added to the other list, rather than the map being added to a list and then converted to a list.

Yep. I probably botched that in the 2to3 removal. Do you want to send a PR fixing it?

xylar commented 5 years ago

Yes, I'll do that soon. Basemap takes awhile to clone...

davidmnielsen commented 5 years ago

This fix leads me to another error:

TypeError: invalid indexer array, does not have integer dtype: array([slice(None, None, None), slice(0, 1, None)], dtype=object)

Could you please give me a hand on this?

davidmnielsen commented 5 years ago

The error is reproducible by following the steps posted in here:

https://github.com/davidmnielsen/polarplots.git

xylar commented 5 years ago

@davidmnielsen, I think you should probably open a separate issue for your error, since this one has been closed. I don't have the expertise to help you with your error myself, I'm afraid.

severin1992 commented 5 years ago

Any news on that? I encounter the same issue:

<ipython-input-13-e5896fefedc7> in <module>
     29 
     30 # Make the plot continuous
---> 31 temp_cyclic, lons_cyclic = addcyclic(tTemp, lons)
     32 # Shift the grid so lons go from -180 to 180 instead of 0 to 360.
     33 #air_cyclic, lons_cyclic = shiftgrid(180., air_cyclic, lons_cyclic, start=False)

~/anaconda3/envs/AirPred/lib/python3.6/site-packages/mpl_toolkits/basemap/__init__.py in addcyclic(*arr, **kwargs)
   5109         return _addcyclic_lon(arr[-1])
   5110     else:
-> 5111         return list(map(_addcyclic,arr[:-1]) + [_addcyclic_lon(arr[-1])])
   5112 
   5113 def _choosecorners(width,height,**kwargs):

TypeError: unsupported operand type(s) for +: 'map' and 'list'

After trying: 

my_example_nc_file = '/Users/Severin/Desktop/ecmwf_era5_17010100.nc'
#fh becomes the file handle of the open netCDF file
fh = Dataset(my_example_nc_file, mode='r') 

import matplotlib.pyplot as plt
from mpl_toolkits.basemap import Basemap, addcyclic, shiftgrid
import matplotlib.cm as cm
import numpy as np

lons = fh.variables['lon'][:]
lats = fh.variables['lat'][:]
tTemp = fh.variables['T2'][0,:,:]

fh.close()

fig=plt.figure(figsize=(12, 8) )

m=Basemap(projection='mill', lat_ts=10, llcrnrlon=lons.min(), \
  urcrnrlon=lons.max(),llcrnrlat=lats.min(),urcrnrlat=lats.max(), \
  resolution='c')

m.drawcoastlines()
m.drawmapboundary()

# Make the plot continuous
temp_cyclic, lons_cyclic = addcyclic(tTemp, lons)

FYI: basemap 1.2.0 py36h0acbc05_0 matplotlib 3.0.2 py36h54f8f79_0

conda update basemap doesn't ensure that the according line is changed to 

return list(map(_addcyclic,arr[:-1])) + [_addcyclic_lon(arr[-1])]

I don't know whether that's a smart thing to do, but I just changed the line by myself. Now I got this message: 


/Users/Severin/anaconda3/envs/AirPred/lib/python3.6/site-packages/numpy/ma/core.py:3174: 
FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; 
use `arr[tuple(seq)]` instead of `arr[seq]`. 
In the future this will be interpreted as an array index, `arr[np.array(seq)]`, 
which will result either in an error or a different result.
  dout = self.data[indx] ```
xylar commented 5 years ago

This issue has been fixed but there hasn't been a new release of basemap since then. @WeatherGod, is there any plan for a release anytime soon?

WeatherGod commented 5 years ago

I am sorry, I hadn't realized that this critical fix was unreleased. I'll tag a new release now.

samayamadhavan commented 4 years ago

Hi, can you tell me what the fix to this issue is?

matheusdpv commented 3 years ago

Any news about how to fix it?

xylar commented 3 years ago

I believe this was fixed in the last release (0.18.0). We have been using https://scitools.org.uk/cartopy/docs/latest/cartopy/util/util.html#cartopy.util.add_cyclic_point successfully since that release. For example: https://github.com/MPAS-Dev/MPAS-Analysis/blob/develop/mpas_analysis/shared/plot/climatology_map.py#L144