Grey boxes when clipping data with kdeplot

[MichelleJansen1]

Hi!

I am struggling a bit with the KDEplot and clipping the data on my geometry. When I perform the code on the whole of Germany it works fine, and I get this map:

While when I try to perform it on the states of Germany, it gives weird results, which are different for each state. See here a few examples:

This is the code:

def sentimentKDE (data, state, title):
    proj = gcrs.Mercator()

    fig = plt.figure(figsize=(15, 10))
    ax1 = plt.subplot(121, projection=proj)
    ax2 = plt.subplot(122, projection=proj) 

    gplt.kdeplot(data[data['label']=='Positive'], shade=True, cmap='Reds',clip=geom_germany[geom_germany['GEN'] == state].geometry,ax=ax1,)
    gplt.polyplot(geom_germany[geom_germany['GEN'] == state].geometry,ax=ax1,  facecolor='white')
    ax1.set_title("Positive tweets", fontsize=20)

    gplt.kdeplot(data[data['label']=='Negative'], shade=True, cmap='Reds',clip=geom_germany[geom_germany['GEN'] == state].geometry,ax=ax2)
    gplt.polyplot(geom_germany[geom_germany['GEN'] == state].geometry,ax=ax2, facecolor='white',)
    ax2.set_title("Negative tweets", fontsize=20)

    plt.savefig(title)

Do you have any idea what is going wrong? Would love the help.

Thank you in advance!

[ResidentMario]

Can you provide (1) the output of pip list and (2) a copy of the dataset, so I can test this locally?

[MichelleJansen1]

hi yes, this is the output of pip list:

Package             Version      Location
------------------- ------------ -----------------------------------------------------------------
affine              2.3.0
attrs               21.2.0
backcall            0.2.0
beautifulsoup4      4.10.0
bertopic            0.9.3
Bottleneck          1.3.2
brotlipy            0.7.0
Cartopy             0.18.0
certifi             2021.10.8
cffi                1.15.0
charset-normalizer  2.0.4
click               7.1.2
click-plugins       1.1.1
cligj               0.7.2
colorama            0.4.4
contextily          1.2.0
cryptography        36.0.0
cycler              0.11.0
debugpy             1.5.1
decorator           5.1.0
Deprecated          1.2.12
descartes           1.1.0
entrypoints         0.3
esda                2.4.1
Fiona               1.8.13.post1
fonttools           4.25.0
GDAL                3.0.2
geographiclib       1.52
geopandas           0.9.0
geoplot             0.4.4
geopy               2.2.0
giddy               2.3.3
idna                3.3
importlib-resources 5.4.0
ipykernel           6.4.1
ipython             7.29.0
ipython-genutils    0.2.0
jedi                0.18.0
Jinja2              3.0.2
joblib              1.1.0
jsonschema          4.4.0
jupyter-client      7.1.0
jupyter-core        4.9.1
kiwisolver          1.3.1
libpysal            4.5.1        
llvmlite            0.37.0
mapclassify         2.4.3
MarkupSafe          2.0.1
matplotlib          3.5.0
matplotlib-inline   0.1.2
mercantile          1.2.1
mkl-fft             1.3.1
mkl-random          1.2.2
mkl-service         2.4.0
mplleaflet          0.0.5
mpmath              1.2.1
munch               2.5.0
munkres             1.1.4
nbformat            5.1.3
nest-asyncio        1.5.1
networkx            2.6.3
numba               0.54.1
numexpr             2.8.1
numpy               1.21.2
oauthlib            3.1.1
olefile             0.46
packaging           21.0
palettable          3.3.0
pandas              1.3.4
parso               0.8.3
patsy               0.5.2
pickleshare         0.7.5
Pillow              8.4.0
pip                 21.2.2
plotly              4.14.2
plotly-express      0.4.1
prompt-toolkit      3.0.20
pycparser           2.21
Pygments            2.10.0
pynndescent         0.5.5
pyOpenSSL           21.0.0
pyparsing           3.0.4
pyproj              2.6.1.post1
pyrsistent          0.18.1
pysal               2.1.0
pyshp               2.1.3
PySocks             1.7.1
python-dateutil     2.8.2
pytz                2021.3
pywin32             302
PyYAML              5.4.1
pyzmq               22.3.0
quantecon           0.5.2
rasterio            1.1.1
requests            2.27.1
requests-oauthlib   1.3.0
retrying            1.3.3
Rtree               0.9.7
scikit-learn        1.0.2
scipy               1.7.3
seaborn             0.11.2
setuptools          58.0.4
Shapely             1.7.1
sip                 4.19.13
six                 1.16.0
sklearn             0.0
snuggs              1.4.7
soupsieve           2.3.1
splot               1.1.4
spreg               1.2.4
statsmodels         0.13.1
sympy               1.9
threadpoolctl       2.2.0
tornado             6.1
tqdm                4.62.3
traitlets           5.1.1
tweepy              4.4.0
umap-learn          0.5.1
urllib3             1.26.7
wcwidth             0.2.5
wheel               0.37.1
win-inet-pton       1.1.0
wincertstore        0.2
wrapt               1.13.3
xyzservices         2021.11.0
zipp                3.7.0
Note: you may need to restart the kernel to use updated packages.

here you have a simplified dataset: sample.csv

[MichelleJansen1]

Oh wait I forgot the bundeslander dataset

this is the code

import geopandas as gpd
bundeslander = gpd.read_file(r'Data\bundeslander.shp')

plt.rcParams['figure.figsize'] = [16, 11]

#set crs to same projection
bundeslander = bundeslander.to_crs(epsg=4326)

geom_germany = bundeslander
geom_germany['geometry'] = geom_germany.buffer(0)

this is the shp bundeslander.zip

[MichelleJansen1]

Do you by any chance had the time to try it? @ResidentMario

If not, don't worry but then I know I need to continue looking for a solution before my thesis deadline :)

[ResidentMario]

Hi @MichelleJansen1, sorry that this is taking a while. Basically what has happened is that I saw your issue, thought about it, couldn't come up with a logical explanation for why you are seeing the behavior you are experiencing right away, and asked for a minimally demonstrative repro. Now that you've provided a repro, this is in my queue of things to look at the next time I have some time to do some maintenance work on my OS projects. To minimize context switches I like to stack up a few different things and spend a day on it, hopefully I'll get to it this weekend. :)

[MichelleJansen1]

Hi @ResidentMario , as I said don't worry about it!!! I am already very grateful that you are willing to try. I will also continue looking for it so I will let you know if I figure anything out :)

[ResidentMario]

I can't reproduce this issue locally, when I plot these states using the provided code they all work as expected. Using the following code:

import pandas as pd
import matplotlib.pyplot as plt
import geoplot as gplt
import geoplot.crs as gcrs
import geopandas as gpd
from shapely import wkt

geom_germany = gpd.read_file("/Users/alekseybilogur/Downloads/bundeslander/bundeslander.shp")
plt.rcParams['figure.figsize'] = [16, 11]
geom_germany = geom_germany.to_crs(epsg=4326)
geom_germany['geometry'] = geom_germany.buffer(0)

data = pd.read_csv("/Users/alekseybilogur/Downloads/sample.csv", index_col=0)
data = gpd.GeoDataFrame(data)
data.geometry = data.geometry.apply(wkt.loads)
data.crs = 'epsg:4326'

def sentimentKDE (data, state, title):
    proj = gcrs.Mercator()

    fig = plt.figure(figsize=(15, 10))
    ax1 = plt.subplot(121, projection=proj)
    ax2 = plt.subplot(122, projection=proj) 

    gplt.kdeplot(data[data['label']=='Positive'], shade=True, cmap='Reds',clip=geom_germany[geom_germany['GEN'] == state].geometry,ax=ax1)
    gplt.polyplot(geom_germany[geom_germany['GEN'] == state].geometry,ax=ax1,  facecolor='white')
    ax1.set_title("Positive tweets", fontsize=20)

    gplt.kdeplot(data[data['label']=='Negative'], shade=True, cmap='Reds',clip=geom_germany[geom_germany['GEN'] == state].geometry,ax=ax2)
    gplt.polyplot(geom_germany[geom_germany['GEN'] == state].geometry,ax=ax2, facecolor='white')
    ax2.set_title("Negative tweets", fontsize=20)

sentimentKDE(data, "Bayern", "FOO")
# ...

The only thing I can suggest is resetting the matplotlib rcparams to their default values. Beyond that I'm afraid I have no other advice to offer.

[MichelleJansen1]

@ResidentMario It did the trick!!! Thank you so much