The area of np.nan is filled with the same color as the surrounding area

[wm-ytakano]

Description

When drawing a diagram using contourf together with cartopy, the area of np.nan is filled with the same color as the surrounding area under certain conditions.

Code to reproduce

I have a 2D grid point value and I'm trying to plot it.

lon = np.load("lon.npy") # lon axis
lat = np.load("lat.npy") # lat axis
gpi = np.load("gpi.npy") # 2D array

To make this problem easier to understand, I will artificially fill two square areas with np.nan for this data.

i = np.argmin(np.abs(lon - 140))
j = np.argmin(np.abs(lat - 30))
gpi[j:j+30, i:i+30] = np.nan

i = np.argmin(np.abs(lon - 150))
j = np.argmin(np.abs(lat - 40))
gpi[j:j+30, i:i+30] = np.nan

Plotting this data using pcolormesh together with cartopy gives an image where the area filled with np.nan is correctly painted white.

ax.pcolormesh(
    lon, lat, gpi, 
    norm=norm,
    transform=ccrs.PlateCarree())

In contrast, if I plot this data using contourf together with cartopy, the area filled with np.nan that does not cross the contour line (e.g. left-bottom box and Taiwan) is painted in the same color as the surrounding area.

ax.contourf(
    lon, lat, gpi, 
    levels=levels,
    extend="max",
    transform=ccrs.PlateCarree())

Plotting this data using contourf without cartopy, the area filled with np.nan is correctly painted white.

plt.contourf(
    lon, lat, gpi, 
    levels=levels,
    extend="both")

This bug does not occur all the time, but seems to happen under certain conditions. However, I do not know under what conditions it happens.

Sample data and the complete code are available in the following repository: https://github.com/wm-ytakano/contourf_nan_filled

Full environment definition

### Operating system macOS Big Sur version 11.2.3 ### Cartopy version 0.18.0 ### conda list ``` # packages in environment at /Users/ytakano/miniconda3/envs/cartopy-env: # # Name Version Build Channel appnope 0.1.2 py39hecd8cb5_1001 argon2-cffi 20.1.0 py39h9ed2024_1 async_generator 1.10 pyhd3eb1b0_0 attrs 20.3.0 pyhd3eb1b0_0 autopep8 1.5.6 pyhd3eb1b0_0 backcall 0.2.0 pyhd3eb1b0_0 blas 1.0 openblas bleach 3.3.0 pyhd3eb1b0_0 ca-certificates 2021.1.19 hecd8cb5_1 cartopy 0.18.0 py39he6a1819_5 conda-forge certifi 2020.12.5 py39hecd8cb5_0 cffi 1.14.5 py39h2125817_0 cftime 1.4.1 py39he3068b8_0 curl 7.71.1 hb0a8c7a_1 cycler 0.10.0 py39hecd8cb5_0 decorator 5.0.3 pyhd3eb1b0_0 defusedxml 0.7.1 pyhd3eb1b0_0 entrypoints 0.3 py39hecd8cb5_0 flake8 3.9.0 pyhd3eb1b0_0 freetype 2.10.4 ha233b18_0 geos 3.8.1 hb1e8313_0 hdf4 4.2.13 h39711bb_2 hdf5 1.10.6 hdbbcd12_0 icu 58.2 h0a44026_3 importlib-metadata 3.7.3 py39hecd8cb5_1 importlib_metadata 3.7.3 hd3eb1b0_1 ipykernel 5.3.4 py39h01d92e1_0 ipython 7.22.0 py39h71a6800_0 conda-forge ipython_genutils 0.2.0 pyhd3eb1b0_1 ipywidgets 7.6.3 pyhd3eb1b0_1 jedi 0.18.0 py39hecd8cb5_1 jinja2 2.11.3 pyhd3eb1b0_0 jpeg 9b he5867d9_2 jsonschema 3.2.0 py_2 jupyter 1.0.0 py39hecd8cb5_7 jupyter_client 6.1.12 pyhd3eb1b0_0 jupyter_console 6.4.0 pyhd3eb1b0_0 jupyter_core 4.7.1 py39hecd8cb5_0 jupyterlab_pygments 0.1.2 py_0 jupyterlab_widgets 1.0.0 pyhd3eb1b0_1 kiwisolver 1.3.1 py39h23ab428_0 krb5 1.18.2 h75d18d8_0 lcms2 2.11 h92f6f08_0 libcurl 7.71.1 h8a08a2b_1 libcxx 11.0.0 h439d374_0 conda-forge libedit 3.1.20210216 h9ed2024_1 libffi 3.3 hb1e8313_2 libgfortran 3.0.1 h93005f0_2 libnetcdf 4.6.1 hfd9a460_3 libopenblas 0.3.13 h7ddc91c_0 libpng 1.6.37 ha441bb4_0 libsodium 1.0.18 h1de35cc_0 libssh2 1.9.0 ha12b0ac_1 libtiff 4.1.0 hcb84e12_1 lz4-c 1.9.3 h23ab428_0 markupsafe 1.1.1 py39h9ed2024_0 matplotlib 3.3.4 py39hecd8cb5_0 matplotlib-base 3.3.4 py39h8b3ea08_0 mccabe 0.6.1 py39hecd8cb5_1 mistune 0.8.4 py39h9ed2024_1000 nbclient 0.5.3 pyhd3eb1b0_0 nbconvert 6.0.7 py39hecd8cb5_0 nbformat 5.1.3 pyhd3eb1b0_0 ncurses 6.2 h0a44026_1 nest-asyncio 1.5.1 pyhd3eb1b0_0 netcdf4 1.5.6 py39h1695cb1_0 notebook 6.3.0 py39hecd8cb5_0 numpy 1.19.2 py39h0fa1045_0 numpy-base 1.19.2 py39h3a452eb_0 olefile 0.46 py_0 openssl 1.1.1k h9ed2024_0 packaging 20.9 pyhd3eb1b0_0 pandas 1.2.3 py39hb2f4e1b_0 pandoc 2.12 hecd8cb5_0 pandocfilters 1.4.3 py39hecd8cb5_1 parso 0.8.2 pyhd3eb1b0_0 pexpect 4.8.0 pyhd3eb1b0_3 pickleshare 0.7.5 pyhd3eb1b0_1003 pillow 8.2.0 py39h5270095_0 pip 21.0.1 py39hecd8cb5_0 proj 7.1.1 h45baca5_3 conda-forge prometheus_client 0.10.0 pyhd3eb1b0_0 prompt-toolkit 3.0.17 pyh06a4308_0 prompt_toolkit 3.0.17 hd3eb1b0_0 ptyprocess 0.7.0 pyhd3eb1b0_2 pycodestyle 2.6.0 pyhd3eb1b0_0 pycparser 2.20 py_2 pyflakes 2.2.0 pyhd3eb1b0_0 pygments 2.8.1 pyhd3eb1b0_0 pyparsing 2.4.7 pyhd3eb1b0_0 pyqt 5.9.2 py39h23ab428_6 pyrsistent 0.17.3 py39h9ed2024_0 pyshp 2.1.3 pyhd3eb1b0_0 python 3.9.2 h88f2d9e_0 python-dateutil 2.8.1 pyhd3eb1b0_0 python_abi 3.9 1_cp39 conda-forge pytz 2021.1 pyhd3eb1b0_0 pyzmq 20.0.0 py39h23ab428_1 qt 5.9.7 h468cd18_1 qtconsole 5.0.3 pyhd3eb1b0_0 qtpy 1.9.0 py_0 readline 8.1 h9ed2024_0 scipy 1.6.2 py39h4420a3a_0 seawater 3.3.4 py_1 conda-forge send2trash 1.5.0 pyhd3eb1b0_1 setuptools 52.0.0 py39hecd8cb5_0 shapely 1.7.1 py39h7edaf14_1 conda-forge sip 4.19.13 py39h23ab428_0 six 1.15.0 py39hecd8cb5_0 sqlite 3.35.3 hce871da_0 terminado 0.9.4 py39hecd8cb5_0 testpath 0.4.4 pyhd3eb1b0_0 tk 8.6.10 hb0a8c7a_0 toml 0.10.2 pyhd3eb1b0_0 tornado 6.1 py39h9ed2024_0 traitlets 5.0.5 pyhd3eb1b0_0 tzdata 2020f h52ac0ba_0 wcwidth 0.2.5 py_0 webencodings 0.5.1 py39hecd8cb5_1 wheel 0.36.2 pyhd3eb1b0_0 widgetsnbextension 3.5.1 py39hecd8cb5_0 xarray 0.17.0 pyhd3eb1b0_0 xz 5.2.5 h1de35cc_0 zeromq 4.3.4 h23ab428_0 zipp 3.4.1 pyhd3eb1b0_0 zlib 1.2.11 h1de35cc_3 zstd 1.4.9 h322a384_0 ``` ### pip list ``` Package Version ------------------- ------------------- appnope 0.1.2 argon2-cffi 20.1.0 async-generator 1.10 attrs 20.3.0 autopep8 1.5.6 backcall 0.2.0 bleach 3.3.0 Cartopy 0.18.0 certifi 2020.12.5 cffi 1.14.5 cftime 1.4.1 cycler 0.10.0 decorator 5.0.3 defusedxml 0.7.1 entrypoints 0.3 flake8 3.9.0 importlib-metadata 3.7.3 ipykernel 5.3.4 ipython 7.22.0 ipython-genutils 0.2.0 ipywidgets 7.6.3 jedi 0.18.0 Jinja2 2.11.3 jsonschema 3.2.0 jupyter 1.0.0 jupyter-client 6.1.12 jupyter-console 6.4.0 jupyter-core 4.7.1 jupyterlab-pygments 0.1.2 jupyterlab-widgets 1.0.0 kiwisolver 1.3.1 MarkupSafe 1.1.1 matplotlib 3.3.4 mccabe 0.6.1 mistune 0.8.4 nbclient 0.5.3 nbconvert 6.0.7 nbformat 5.1.3 nest-asyncio 1.5.1 netCDF4 1.5.6 notebook 6.3.0 numpy 1.19.2 olefile 0.46 packaging 20.9 pandas 1.2.3 pandocfilters 1.4.3 parso 0.8.2 pexpect 4.8.0 pickleshare 0.7.5 Pillow 8.2.0 pip 21.0.1 prometheus-client 0.10.0 prompt-toolkit 3.0.17 ptyprocess 0.7.0 pycodestyle 2.6.0 pycparser 2.20 pyflakes 2.2.0 Pygments 2.8.1 pyparsing 2.4.7 pyrsistent 0.17.3 pyshp 2.1.3 python-dateutil 2.8.1 pytz 2021.1 pyzmq 20.0.0 qtconsole 5.0.3 QtPy 1.9.0 scipy 1.6.2 seawater 3.3.4 Send2Trash 1.5.0 setuptools 52.0.0.post20210125 Shapely 1.7.1 sip 4.19.13 six 1.15.0 terminado 0.9.4 testpath 0.4.4 toml 0.10.2 tornado 6.1 traitlets 5.0.5 wcwidth 0.2.5 webencodings 0.5.1 wheel 0.36.2 widgetsnbextension 3.5.1 xarray 0.17.0 zipp 3.4.1 ```

[kdpenner]

Affects my work too, except with masked arrays. MWE:

import matplotlib.pyplot as plt
import numpy as np
import cartopy.crs as ccrs

lons = [-180, -179]
lats = [-90, -89]

lonsm, latsm = np.meshgrid(lons, lats)
lonsm = np.ma.masked_array(lonsm, mask=[[False, False], [False, True]])
latsm = np.ma.masked_array(latsm, mask=[[False, False], [False, True]])

fig = plt.figure()
ax1 = fig.add_subplot(3, 1, 1)
ax1.scatter(lonsm, latsm)

ax2 = fig.add_subplot(3, 1, 2, projection=ccrs.PlateCarree())
ax2.scatter(lonsm, latsm)

ax3 = fig.add_subplot(3, 1, 3, projection=ccrs.PlateCarree())
ax3.scatter(lonsm, latsm, transform=ccrs.PlateCarree())

Perhaps related to transform_points not respecting a mask:

>>> ccrs.PlateCarree().transform_points(ccrs.PlateCarree(), lonsm, latsm)
array([[[-180.,  -90.,    0.],
        [-179.,  -90.,    0.]],

       [[-180.,  -89.,    0.],
        [-179.,  -89.,    0.]]])

[smartlixx]

@kdpenner your problem is not related with the original problem. Masked arrays in numpy are very fragile and only the mask-aware operations can reserve masks. So mask information can be lost very easily. The best way is to use NaN.

import matplotlib.pyplot as plt
import numpy as np
import cartopy.crs as ccrs

lons = [-180., -179.]
lats = [-90., -89.]

lonsm, latsm = np.meshgrid(lons, lats)
lonsm[1,1] = np.nan  # = np.ma.masked_array(lonsm, mask=[[False, False], [False, True]])
latsm[1,1] = np.nan   # = np.ma.masked_array(latsm, mask=[[False, False], [False, True]])

fig = plt.figure()
ax1 = fig.add_subplot(3, 1, 1)
ax1.scatter(lonsm, latsm)

ax2 = fig.add_subplot(3, 1, 2, projection=ccrs.PlateCarree())
ax2.scatter(lonsm, latsm)

ax3 = fig.add_subplot(3, 1, 3, projection=ccrs.PlateCarree())
ax3.scatter(lonsm, latsm, transform=ccrs.PlateCarree())

plt.show()

The output is

[kdpenner]

I see. My issue is probably a duplicate of #1002.