search

SciTools / cartopy

Cartopy - a cartographic python library with matplotlib support
https://scitools.org.uk/cartopy/docs/latest
BSD 3-Clause "New" or "Revised" License
1.43k stars 365 forks source link

set_clim() doesn't update color limits for cells crossing the antimeridian #1654

Closed LiamBindle closed 4 years ago

LiamBindle commented 4 years ago

Description

@sdeastham and I have been trying SciTools/cartopy#1622 which fixes streaking of data that crosses the antimeridian. Thanks so much for this, it works great and solves a longstanding problem of ours!

We noticed that set_clim() doesn't appear to updated color limits for cells that cross the antimeridian. Do you know why this might be? Below are some snippets to reproduce.

Code to reproduce

Data files: data.zip

You can reproduce the issue like so

import cartopy
import cartopy.crs as ccrs
import matplotlib.pyplot as plt
import pyproj
import xarray as xr
import numpy as np

print(f"caropty version: {cartopy._version.version}")

data = np.load('data/data.npy')
grid = np.load('data/grid.npz')

vmin=np.quantile(data, 0.01)
vmax=np.quantile(data, 0.99)

plt.figure()
ax = plt.axes(projection=ccrs.PlateCarree())
ax.coastlines()
ax.set_global()

for nf in [0, 1, 2, 3, 4, 5]:
    x = grid['x'][nf,...] % 360
    y = grid['y'][nf,...]
    d = data[nf, ...]
    im = plt.pcolormesh(x, y, d, transform=ccrs.PlateCarree())
    im.set_clim(vmin, vmax)

plt.show()

image Notice the dark cells (affecting cells crossing the AM) in the figure above.

Instead, if we use the norm kwarg to pcolormesh() it behaves as expected.

import cartopy
import cartopy.crs as ccrs
import matplotlib.pyplot as plt
import pyproj
import xarray as xr
import numpy as np

print(f"caropty version: {cartopy._version.version}")

data = np.load('data/data.npy')
grid = np.load('data/grid.npz')

vmin=np.quantile(data, 0.01)
vmax=np.quantile(data, 0.99)
norm = plt.Normalize(vmin, vmax)

plt.figure()
ax = plt.axes(projection=ccrs.PlateCarree())
ax.coastlines()
ax.set_global()

for nf in [0, 1, 2, 3, 4, 5]:
    x = grid['x'][nf,...] % 360
    y = grid['y'][nf,...]
    d = data[nf, ...]
    im = plt.pcolormesh(x, y, d, transform=ccrs.PlateCarree(), norm=norm)

plt.show()

image

The reason we'd prefer to use set_clim() is because we have quite a bit of code that uses that method rather than norm. Do you have any thoughts?

Thanks in advance! Let me know if I can provide any more information.

Operating system

Both of use are using Ubuntu 18.

Cartopy version

0.18.1.dev113+g3565f78

conda list

My `conda list` output ``` # packages in environment at /home/liam/miniconda3/envs/py37: # # Name Version Build Channel _libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 1_llvm conda-forge affine 2.3.0 py_0 conda-forge attrs 19.3.0 py_0 conda-forge backcall 0.1.0 py_0 conda-forge beautifulsoup4 4.9.1 pypi_0 pypi bokeh 2.0.1 py37hc8dfbb8_0 conda-forge boost-cpp 1.72.0 h8e57a91_0 conda-forge brotlipy 0.7.0 py37h8f50634_1000 conda-forge bs4 0.0.1 pypi_0 pypi bzip2 1.0.8 h516909a_2 conda-forge ca-certificates 2020.6.20 hecda079_0 conda-forge cairo 1.16.0 hcf35c78_1003 conda-forge cartopy 0.18.1.dev113+g3565f78 pypi_0 pypi certifi 2020.6.20 py37hc8dfbb8_0 conda-forge cffi 1.14.0 py37hd463f26_0 conda-forge cfitsio 3.470 h3eac812_5 conda-forge cftime 1.1.3 py37h03ebfcd_0 conda-forge chardet 3.0.4 py37hc8dfbb8_1006 conda-forge click 7.1.1 pypi_0 pypi click-plugins 1.1.1 py_0 conda-forge cligj 0.5.0 pypi_0 pypi cloudpickle 1.4.1 py_0 conda-forge cryptography 2.9.2 py37hb09aad4_0 conda-forge curl 7.71.1 he644dc0_3 conda-forge cycler 0.10.0 py_2 conda-forge cython 0.29.21 pypi_0 pypi cytoolz 0.10.1 py37h516909a_0 conda-forge dask 2.15.0 py_0 conda-forge dask-core 2.15.0 py_0 conda-forge decorator 4.4.2 py_0 conda-forge distributed 2.17.0 py37hc8dfbb8_0 conda-forge entrypoints 0.3 py37hc8dfbb8_1001 conda-forge expat 2.2.9 he1b5a44_2 conda-forge fiona 1.8.13.post1 pypi_0 pypi fontconfig 2.13.1 h86ecdb6_1001 conda-forge freetype 2.10.2 he06d7ca_0 conda-forge freexl 1.0.5 h14c3975_1002 conda-forge fsspec 0.7.4 py_0 conda-forge gdal 3.0.4 py37h4b180d9_10 conda-forge geopandas 0.7.0 py_1 conda-forge geos 3.8.1 he1b5a44_0 conda-forge geotiff 1.6.0 h05acad5_0 conda-forge gettext 0.19.8.1 hc5be6a0_1002 conda-forge giflib 5.2.1 h516909a_2 conda-forge glib 2.64.3 h6f030ca_0 conda-forge hdf4 4.2.13 hf30be14_1003 conda-forge hdf5 1.10.6 nompi_h3c11f04_100 conda-forge heapdict 1.0.1 py_0 conda-forge icu 64.2 he1b5a44_1 conda-forge idna 2.9 py_1 conda-forge importlib-metadata 1.6.0 pypi_0 pypi ipykernel 5.2.1 py37h43977f1_0 conda-forge ipython 7.15.0 py37hc8dfbb8_0 conda-forge ipython_genutils 0.2.0 py_1 conda-forge jedi 0.17.0 py37hc8dfbb8_0 conda-forge jinja2 2.11.2 pyh9f0ad1d_0 conda-forge joblib 0.15.1 py_0 conda-forge jpeg 9c h14c3975_1001 conda-forge json-c 0.13.1 hbfbb72e_1002 conda-forge jsonschema 3.2.0 pypi_0 pypi jupyter_client 6.1.3 py_0 conda-forge jupyter_core 4.6.3 py37hc8dfbb8_1 conda-forge kealib 1.4.13 h33137a7_1 conda-forge kiwisolver 1.2.0 py37h99015e2_0 conda-forge krb5 1.17.1 h2fd8d38_0 conda-forge ld_impl_linux-64 2.34 h53a641e_4 conda-forge libblas 3.8.0 16_openblas conda-forge libcblas 3.8.0 16_openblas conda-forge libcurl 7.71.1 hcdd3856_3 conda-forge libdap4 3.20.6 h1d1bd15_0 conda-forge libedit 3.1.20170329 hf8c457e_1001 conda-forge libffi 3.2.1 he1b5a44_1007 conda-forge libgcc-ng 9.2.0 h24d8f2e_2 conda-forge libgdal 3.0.4 he6a97d6_10 conda-forge libgfortran-ng 7.5.0 hdf63c60_6 conda-forge libiconv 1.15 h516909a_1006 conda-forge libkml 1.3.0 hb574062_1011 conda-forge liblapack 3.8.0 16_openblas conda-forge libnetcdf 4.7.4 nompi_h84807e1_104 conda-forge libopenblas 0.3.9 h5ec1e0e_0 conda-forge libpng 1.6.37 hed695b0_1 conda-forge libpq 12.2 h5513abc_1 conda-forge libpysal 4.2.2 pypi_0 pypi libsodium 1.0.17 h516909a_0 conda-forge libspatialindex 1.9.3 he1b5a44_3 conda-forge libspatialite 4.3.0a h2482549_1038 conda-forge libssh2 1.9.0 hab1572f_2 conda-forge libstdcxx-ng 9.2.0 hdf63c60_2 conda-forge libtiff 4.1.0 hc7e4089_6 conda-forge libuuid 2.32.1 h14c3975_1000 conda-forge libwebp-base 1.1.0 h516909a_3 conda-forge libxcb 1.13 h14c3975_1002 conda-forge libxml2 2.9.10 hee79883_0 conda-forge llvm-openmp 10.0.0 hc9558a2_0 conda-forge locket 0.2.0 py_2 conda-forge lz4-c 1.9.2 he1b5a44_1 conda-forge markupsafe 1.1.1 py37h8f50634_1 conda-forge matplotlib-base 3.2.1 py37h30547a4_0 conda-forge mgwr 2.1.1 pypi_0 pypi msgpack-python 1.0.0 py37h99015e2_1 conda-forge munch 2.5.0 py_0 conda-forge nbformat 5.0.6 pypi_0 pypi ncurses 6.1 hf484d3e_1002 conda-forge netcdf4 1.5.3 nompi_py37hdc49583_105 conda-forge numpy 1.18.3 pypi_0 pypi olefile 0.46 py_0 conda-forge openjpeg 2.3.1 h981e76c_3 conda-forge openssl 1.1.1g h516909a_0 conda-forge owslib 0.19.2 py_1 conda-forge packaging 20.4 pyh9f0ad1d_0 conda-forge pandas 1.0.4 py37h0da4684_0 conda-forge parso 0.7.0 pyh9f0ad1d_0 conda-forge partd 1.1.0 py_0 conda-forge patsy 0.5.1 py_0 conda-forge pcre 8.44 he1b5a44_0 conda-forge pexpect 4.8.0 py37hc8dfbb8_1 conda-forge pickleshare 0.7.5 py37hc8dfbb8_1001 conda-forge pillow 7.1.2 py37h718be6c_0 conda-forge pip 20.1.1 py_1 conda-forge pixman 0.38.0 h516909a_1003 conda-forge plotly 4.6.0 pypi_0 pypi plotly-geo 1.0.0 pypi_0 pypi poppler 0.87.0 h4190859_1 conda-forge poppler-data 0.4.9 1 conda-forge postgresql 12.2 h8573dbc_1 conda-forge proj 7.0.0 h966b41f_4 conda-forge prompt-toolkit 3.0.5 py_0 conda-forge psutil 5.7.0 py37h8f50634_1 conda-forge pthread-stubs 0.4 h14c3975_1001 conda-forge ptyprocess 0.6.0 py_1001 conda-forge pycparser 2.20 py_0 conda-forge pyepsg 0.4.0 py_0 conda-forge pygments 2.6.1 py_0 conda-forge pyopenssl 19.1.0 py_1 conda-forge pyparsing 2.4.7 pyh9f0ad1d_0 conda-forge pyproj 2.6.1.post1 py37h34dd122_0 conda-forge pyrsistent 0.16.0 pypi_0 pypi pysal 1.14.4.post1 py37_1 pyshp 2.1.0 py_0 conda-forge pysocks 1.7.1 py37hc8dfbb8_1 conda-forge python 3.7.6 h8356626_5_cpython conda-forge python-dateutil 2.8.1 py_0 conda-forge python_abi 3.7 1_cp37m conda-forge pytz 2019.3 pypi_0 pypi pyyaml 5.3.1 py37h8f50634_0 conda-forge pyzmq 19.0.1 py37hac76be4_0 conda-forge rasterio 1.1.3 py37h900e953_0 conda-forge readline 8.0 hf8c457e_0 conda-forge requests 2.23.0 pyh8c360ce_2 conda-forge retrying 1.3.3 pypi_0 pypi rtree 0.9.4 py37h8526d28_1 conda-forge scikit-learn 0.23.0 py37h8a51577_0 conda-forge scipy 1.4.1 py37ha3d9a3c_3 conda-forge seaborn 0.10.1 1 conda-forge seaborn-base 0.10.1 py_1 conda-forge setuptools 47.1.1 py37hc8dfbb8_0 conda-forge shapely 1.7.0 py37hc88ce51_3 conda-forge six 1.15.0 pyh9f0ad1d_0 conda-forge snuggs 1.4.7 py_0 conda-forge sortedcontainers 2.1.0 py_0 conda-forge soupsieve 2.0.1 pypi_0 pypi spglm 1.0.7 pypi_0 pypi spreg 1.1.1 pypi_0 pypi sqlite 3.30.1 hcee41ef_0 conda-forge statsmodels 0.11.1 py37h8f50634_2 conda-forge tbb 2020.1 hc9558a2_0 conda-forge tblib 1.6.0 py_0 conda-forge threadpoolctl 2.1.0 pyh5ca1d4c_0 conda-forge tiledb 1.7.7 h8efa9f0_3 conda-forge tk 8.6.10 hed695b0_0 conda-forge toolz 0.10.0 py_0 conda-forge tornado 6.0.4 py37h8f50634_1 conda-forge tqdm 4.45.0 pypi_0 pypi traitlets 4.3.3 py37hc8dfbb8_1 conda-forge typing_extensions 3.7.4.2 py_0 conda-forge tzcode 2020a h516909a_0 conda-forge urllib3 1.25.9 py_0 conda-forge wcwidth 0.1.9 pyh9f0ad1d_0 conda-forge wheel 0.34.2 py_1 conda-forge xarray 0.15.1 py_0 conda-forge xerces-c 3.2.2 h8412b87_1004 conda-forge xorg-kbproto 1.0.7 h14c3975_1002 conda-forge xorg-libice 1.0.10 h516909a_0 conda-forge xorg-libsm 1.2.3 h84519dc_1000 conda-forge xorg-libx11 1.6.9 h516909a_0 conda-forge xorg-libxau 1.0.9 h14c3975_0 conda-forge xorg-libxdmcp 1.1.3 h516909a_0 conda-forge xorg-libxext 1.3.4 h516909a_0 conda-forge xorg-libxrender 0.9.10 h516909a_1002 conda-forge xorg-renderproto 0.11.1 h14c3975_1002 conda-forge xorg-xextproto 7.3.0 h14c3975_1002 conda-forge xorg-xproto 7.0.31 h14c3975_1007 conda-forge xz 5.2.5 h516909a_0 conda-forge yaml 0.2.4 h516909a_0 conda-forge zeromq 4.3.2 he1b5a44_2 conda-forge zict 2.0.0 py_0 conda-forge zipp 3.1.0 pypi_0 pypi zlib 1.2.11 h516909a_1006 conda-forge zstd 1.4.4 h6597ccf_3 conda-forge ```

pip list

My `pip list` output ``` Package Version ------------------ ---------------------- affine 2.3.0 attrs 19.3.0 backcall 0.1.0 beautifulsoup4 4.9.1 bokeh 2.0.1 brotlipy 0.7.0 bs4 0.0.1 Cartopy 0.18.1.dev113+g3565f78 certifi 2020.6.20 cffi 1.14.0 cftime 1.1.3 chardet 3.0.4 click 7.1.2 click-plugins 1.1.1 cligj 0.5.0 cloudpickle 1.4.1 cryptography 2.9.2 cycler 0.10.0 Cython 0.29.21 cytoolz 0.10.1 dask 2.15.0 decorator 4.4.2 distributed 2.17.0 entrypoints 0.3 Fiona 1.8.13.post1 fsspec 0.7.4 GDAL 3.0.4 geopandas 0.7.0 HeapDict 1.0.1 idna 2.9 importlib-metadata 1.6.0 ipykernel 5.2.1 ipython 7.15.0 ipython-genutils 0.2.0 jedi 0.17.0 Jinja2 2.11.2 joblib 0.15.1 jsonschema 3.2.0 jupyter-client 6.1.3 jupyter-core 4.6.3 kiwisolver 1.2.0 libpysal 4.2.2 locket 0.2.0 MarkupSafe 1.1.1 matplotlib 3.2.1 mgwr 2.1.1 msgpack 1.0.0 munch 2.5.0 nbformat 5.0.6 netCDF4 1.5.3 numpy 1.18.4 olefile 0.46 OWSLib 0.19.2 packaging 20.4 pandas 1.0.4 parso 0.7.0 partd 1.1.0 patsy 0.5.1 pexpect 4.8.0 pickleshare 0.7.5 Pillow 7.1.2 pip 20.1.1 plotly 4.6.0 plotly-geo 1.0.0 prompt-toolkit 3.0.5 psutil 5.7.0 ptyprocess 0.6.0 pycparser 2.20 pyepsg 0.4.0 Pygments 2.6.1 pyOpenSSL 19.1.0 pyparsing 2.4.7 pyproj 2.6.1.post1 pyrsistent 0.16.0 PySAL 1.14.4.post1 pyshp 2.1.0 PySocks 1.7.1 python-dateutil 2.8.1 pytz 2020.1 PyYAML 5.3.1 pyzmq 19.0.1 rasterio 1.1.3 requests 2.23.0 retrying 1.3.3 Rtree 0.9.4 scikit-learn 0.23.0 scipy 1.4.1 seaborn 0.10.1 setuptools 47.1.1.post20200529 Shapely 1.7.0 six 1.15.0 snuggs 1.4.7 sortedcontainers 2.1.0 soupsieve 2.0.1 spglm 1.0.7 spreg 1.1.1 statsmodels 0.11.1 tblib 1.6.0 threadpoolctl 2.1.0 toolz 0.10.0 tornado 6.0.4 tqdm 4.45.0 traitlets 4.3.3 typing-extensions 3.7.4.2 urllib3 1.25.9 wcwidth 0.1.9 wheel 0.34.2 xarray 0.15.1 zict 2.0.0 zipp 3.1.0 ```
greglucas commented 4 years ago

Those cells are a part of a separate collection. If you call im._wrapped_collection_fix.set_clim() I think that will propagate it for you.

Of course, it would be good to propagate those calls automatically for you. I recently did that for set_array() to provide animations, but didn't extend it for anything else https://github.com/SciTools/cartopy/blob/8aca30fce1e9536aa002c8495995bccf375415a3/lib/cartopy/mpl/geocollection.py

Would you be interested in opening up a PR for extending some of those other calls? Linking in this issue: https://github.com/SciTools/cartopy/issues/233

sdeastham commented 4 years ago

@greglucas that's great to hear! Regarding the update to set_array, that's also a relief - I lean heavily on that capability for video creation. However, I'm using cartopy version 0.18.1.dev113+g3565f78, and if I modify Liam's code to something like im = ax.pcolormesh(...); im.set_array(im.get_array()*0.5)), the cells crossing the antimeridian seem to disappear. Am I missing something?

Also, for what it's worth, I'd love to at least see set_clim extended that way.

greglucas commented 4 years ago

Hm... That may be due to shape mismatches? What are the shapes you're inputting and getting out of im.get_array()? My first guess is one may be flattened and messing up the set_array call.

sdeastham commented 4 years ago

So it turns out the "issue" here is not with set_array, but rather with get_array. If I use my original data directly with set_array it works fine. The "problem" is that get_array returns a masked array - so calling im.set_array(im.get_array()*0.5) results in no data being provided for the special polygons. Just calling im.set_array(data[nf,...].ravel()*0.5) gives the correct behavior. I understand though that this may be a feature rather than a bug!

LiamBindle commented 4 years ago

Thanks @greglucas, I submitted a PR (#1655) with the update based on your handling of set_array().

greglucas commented 4 years ago

@sdeastham, the get_array() is only returning the Quadmesh cells, and yes those have masked elements where the 'wrap' occurred. That was definitely an oversight on my part only doing the set_ and not the get_ :) Do you want to take a stab at a PR implementing the corresponding get_ function by returning an array that is filled with the _wrapped_collection array values?

sdeastham commented 4 years ago

@greglucas I'm a little out of my depth on that front I'm afraid, but hopefully this fix does the job (https://github.com/SciTools/cartopy/pull/1656)! It seems to work when used in a simple test, anyway.