search

has2k1 / plotnine

A Grammar of Graphics for Python
https://plotnine.org
MIT License
4.03k stars 217 forks source link

ValueError with geom_density_2d() #561

Closed TobiRoby closed 2 years ago

TobiRoby commented 2 years ago

Hey,

I tested a dummy example for the geom_density_2d() function and for me it is not working. Here it is working for some others.

from plotnine import aes, geom_density_2d, ggplot 
from plotnine.data import faithful

(
    ggplot(data=faithful)
    + geom_density_2d(aes(x="eruptions", y="waiting", color="..level.."))
)

The error I receive:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
*/lib/python3.7/site-packages/IPython/core/formatters.py in __call__(self, obj)
    700                 type_pprinters=self.type_printers,
    701                 deferred_pprinters=self.deferred_printers)
--> 702             printer.pretty(obj)
    703             printer.flush()
    704             return stream.getvalue()

*/lib/python3.7/site-packages/IPython/lib/pretty.py in pretty(self, obj)
    392                         if cls is not object \
    393                                 and callable(cls.__dict__.get('__repr__')):
--> 394                             return _repr_pprint(obj, self, cycle)
    395 
    396             return _default_pprint(obj, self, cycle)

*/lib/python3.7/site-packages/IPython/lib/pretty.py in _repr_pprint(obj, p, cycle)
    698     """A pprint that just redirects to the normal repr function."""
    699     # Find newlines and replace them with p.break_()
--> 700     output = repr(obj)
    701     lines = output.splitlines()
    702     with p.group():

*/lib/python3.7/site-packages/plotnine/ggplot.py in __repr__(self)
     95         Print/show the plot
     96         """
---> 97         self.__str__()
     98         return '<ggplot: (%d)>' % self.__hash__()
     99 

*/lib/python3.7/site-packages/plotnine/ggplot.py in __str__(self)
     86         Print/show the plot
     87         """
---> 88         self.draw(show=True)
     89 
     90         # Return and empty string so that print(p) is "pretty"

*/lib/python3.7/site-packages/plotnine/ggplot.py in draw(self, return_ggplot, show)
    203         self = deepcopy(self)
    204         with plot_context(self, show=show):
--> 205             self._build()
    206 
    207             # setup

*/lib/python3.7/site-packages/plotnine/ggplot.py in _build(self)
    298 
    299         # Apply and map statistics
--> 300         layers.compute_statistic(layout)
    301         layers.map_statistic(self)
    302 

*/lib/python3.7/site-packages/plotnine/layer.py in compute_statistic(self, layout)
     71     def compute_statistic(self, layout):
     72         for l in self:
---> 73             l.compute_statistic(layout)
     74 
     75     def map_statistic(self, plot):

*/lib/python3.7/site-packages/plotnine/layer.py in compute_statistic(self, layout)
    322         data = self.stat.use_defaults(data)
    323         data = self.stat.setup_data(data)
--> 324         data = self.stat.compute_layer(data, params, layout)
    325         self.data = data
    326 

*/lib/python3.7/site-packages/plotnine/stats/stat.py in compute_layer(cls, data, params, layout)
    274             return cls.compute_panel(pdata, pscales, **params)
    275 
--> 276         return groupby_apply(data, 'PANEL', fn)
    277 
    278     @classmethod

*/lib/python3.7/site-packages/plotnine/utils.py in groupby_apply(df, cols, func, *args, **kwargs)
    632         # function fn should be free to modify dataframe d, therefore
    633         # do not mark d as a slice of df i.e no SettingWithCopyWarning
--> 634         lst.append(func(d, *args, **kwargs))
    635     return pd.concat(lst, axis=axis, ignore_index=True)
    636 

*/lib/python3.7/site-packages/plotnine/stats/stat.py in fn(pdata)
    272                 return pdata
    273             pscales = layout.get_scales(pdata['PANEL'].iat[0])
--> 274             return cls.compute_panel(pdata, pscales, **params)
    275 
    276         return groupby_apply(data, 'PANEL', fn)

*/lib/python3.7/site-packages/plotnine/stats/stat.py in compute_panel(cls, data, scales, **params)
    305         stats = []
    306         for _, old in data.groupby('group'):
--> 307             new = cls.compute_group(old, scales, **params)
    308             unique = uniquecols(old)
    309             missing = unique.columns.difference(new.columns)

*/lib/python3.7/site-packages/plotnine/stats/stat_density_2d.py in compute_group(cls, data, scales, **params)
    100         if params['contour']:
    101             Z = density.reshape(len(x), len(y))
--> 102             data = contour_lines(X, Y, Z, params['levels'])
    103             # Each piece should have a distinct group
    104             groups = str(group) + '-00' + data['piece'].astype(str)

*/lib/python3.7/site-packages/plotnine/stats/stat_density_2d.py in contour_lines(X, Y, Z, levels)
    156     # Collapse the info and make it fit for dataframe columns
    157     if segments:
--> 158         x, y = np.vstack(segments).T
    159         piece = np.hstack(piece_ids)
    160         level = np.hstack(level_values)

<__array_function__ internals> in vstack(*args, **kwargs)

*/lib/python3.7/site-packages/numpy/core/shape_base.py in vstack(tup)
    280     if not isinstance(arrs, list):
    281         arrs = [arrs]
--> 282     return _nx.concatenate(arrs, 0)
    283 
    284 

<__array_function__ internals> in concatenate(*args, **kwargs)

ValueError: all the input array dimensions for the concatenation axis must match exactly, but along dimension 1, the array at index 0 has size 0 and the array at index 2 has size 3

going along with an interesting numpy warning:

*/lib/python3.7/site-packages/numpy/core/shape_base.py:121: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.

Used python packages:

affine                 2.3.0       Matrices describing affine transformation of the plane.
argon2-cffi            21.3.0      The secure Argon2 password hashing algorithm.
argon2-cffi-bindings   21.2.0      Low-level CFFI bindings for Argon2
attrs                  21.4.0      Classes Without Boilerplate
azure-common           1.1.27      Microsoft Azure Client Library for Python (Common)
azure-core             1.21.1      Microsoft Azure Core Library for Python
azure-identity         1.7.1       Microsoft Azure Identity Library for Python
azure-keyvault-secrets 4.3.0       Microsoft Azure Key Vault Secrets Client Library for Python
azure-servicebus       7.5.0       Microsoft Azure Service Bus Client Library for Python
azure-storage-blob     12.9.0      Microsoft Azure Blob Storage Client Library for Python
backcall               0.2.0       Specifications for callback functions passed in to an API
black                  21.12b0     The uncompromising code formatter.
bleach                 4.1.0       An easy safelist-based HTML-sanitizing tool.
certifi                2021.10.8   Python package for providing Mozilla's CA Bundle.
cffi                   1.15.0      Foreign Function Interface for Python calling C code.
cfgv                   3.3.1       Validate configuration and produce human readable error messages.
charset-normalizer     2.0.10      The Real First Universal Charset Detector. Open, modern and actively maintained alternative to Chardet.
click                  7.1.2       Composable command line interface toolkit
click-plugins          1.1.1       An extension module for click to enable registering CLI commands via setuptools entry-points.
cligj                  0.7.2       Click params for commmand line interfaces to GeoJSON
contextily             1.2.0       Context geo-tiles in Python
coverage               5.5         Code coverage measurement for Python
cryptography           36.0.1      cryptography is a package which provides cryptographic recipes and primitives to Python developers.
cycler                 0.11.0      Composable style cycles
debugpy                1.5.1       An implementation of the Debug Adapter Protocol for Python
decorator              5.1.1       Decorators for Humans
defusedxml             0.7.1       XML bomb protection for Python stdlib modules
descartes              1.1.0       Use geometric objects as matplotlib paths and patches
distlib                0.3.4       Distribution utilities
entrypoints            0.3         Discover and load entry points from installed packages.
fastapi                0.61.2      FastAPI framework, high performance, easy to learn, fast to code, ready for production
filelock               3.4.2       A platform independent file lock.
fiona                  1.8.20      Fiona reads and writes spatial data files
flake8                 4.0.1       the modular source code checker: pep8 pyflakes and co
fonttools              4.28.5      Tools to manipulate font files
geographiclib          1.52        The geodesic routines from GeographicLib
geopandas              0.9.0       Geographic pandas extensions
geopy                  2.2.0       Python Geocoding Toolbox
h11                    0.12.0      A pure-Python, bring-your-own-I/O implementation of HTTP/1.1
identify               2.4.4       File identification library for Python
idna                   3.3         Internationalized Domain Names in Applications (IDNA)
importlib-metadata     4.2.0       Read metadata from Python packages
importlib-resources    5.4.0       Read resources from Python packages
iniconfig              1.1.1       iniconfig: brain-dead simple config-ini parsing
ipykernel              6.7.0       IPython Kernel for Jupyter
ipython                7.31.0      IPython: Productive Interactive Computing
ipython-genutils       0.2.0       Vestigial utilities from IPython
isodate                0.6.1       An ISO 8601 date/time/duration parser and formatter
isort                  5.10.1      A Python utility / library to sort Python imports.
jedi                   0.18.1      An autocompletion tool for Python that can be used for text editors.
jinja2                 3.0.3       A very fast and expressive template engine.
joblib                 1.1.0       Lightweight pipelining with Python functions
jsonschema             4.4.0       An implementation of JSON Schema validation for Python
jupyter-client         7.1.1       Jupyter protocol implementation and client libraries
jupyter-core           4.9.1       Jupyter core package. A base package on which Jupyter projects rely.
jupyterlab-pygments    0.1.2       Pygments theme using JupyterLab CSS variables
kiwisolver             1.3.2       A fast implementation of the Cassowary constraint solver
markupsafe             2.0.1       Safely add untrusted strings to HTML/XML markup.
matplotlib             3.5.1       Python plotting package
matplotlib-inline      0.1.3       Inline Matplotlib backend for Jupyter
mccabe                 0.6.1       McCabe checker, plugin for flake8
mercantile             1.2.1       Web mercator XYZ tile utilities
mistune                0.8.4       The fastest markdown parser in pure Python
mizani                 0.7.3       Scales for Python
msal                   1.16.0      The Microsoft Authentication Library (MSAL) for Python library enables your app to access the Microsoft Cloud by supporting authentication of users with Microsoft Azure Active Directory accounts (AAD) and Microsoft Accounts ...
msal-extensions        0.3.1       
msrest                 0.6.21      AutoRest swagger generator Python client runtime.
munch                  2.5.0       A dot-accessible dictionary (a la JavaScript objects)
mypy                   0.812       Optional static typing for Python
mypy-extensions        0.4.3       Experimental type system extensions for programs checked with the mypy typechecker.
nbclient               0.5.10      A client library for executing notebooks. Formerly nbconvert's ExecutePreprocessor.
nbconvert              6.4.0       Converting Jupyter Notebooks
nbformat               5.1.3       The Jupyter Notebook format
nest-asyncio           1.5.4       Patch asyncio to allow nested event loops
nodeenv                1.6.0       Node.js virtual environment builder
notebook               6.4.7       A web-based notebook environment for interactive computing
numpy                  1.21.5      NumPy is the fundamental package for array computing with Python.
oauthlib               3.1.1       A generic, spec-compliant, thorough implementation of the OAuth request-signing logic
packaging              21.3        Core utilities for Python packages
palettable             3.3.0       Color palettes for Python
pandas                 1.3.5       Powerful data structures for data analysis, time series, and statistics
pandera                0.7.2       A light-weight and flexible validation package for pandas data structures.
pandocfilters          1.5.0       Utilities for writing pandoc filters in python
parso                  0.8.3       A Python Parser
pathspec               0.9.0       Utility library for gitignore style pattern matching of file paths.
patsy                  0.5.2       A Python package for describing statistical models and for building design matrices.
pexpect                4.8.0       Pexpect allows easy control of interactive console applications.
pickleshare            0.7.5       Tiny 'shelve'-like database with concurrency support
pillow                 9.0.0       Python Imaging Library (Fork)
platformdirs           2.4.1       A small Python module for determining appropriate platform-specific dirs, e.g. a "user data dir".
plotnine               0.8.0       A grammar of graphics for python
pluggy                 1.0.0       plugin and hook calling mechanisms for python
portalocker            2.3.2       Wraps the portalocker recipe for easy usage
pre-commit             2.16.0      A framework for managing and maintaining multi-language pre-commit hooks.
prometheus-client      0.12.0      Python client for the Prometheus monitoring system.
prompt-toolkit         3.0.24      Library for building powerful interactive command lines in Python
ptyprocess             0.7.0       Run a subprocess in a pseudo terminal
py                     1.11.0      library with cross-python path, ini-parsing, io, code, log facilities
pyarrow                3.0.0       Python library for Apache Arrow
pycodestyle            2.8.0       Python style guide checker
pycparser              2.21        C parser in Python
pydantic               1.9.0       Data validation and settings management using python 3.6 type hinting
pydocstyle             6.1.1       Python docstring style checker
pyflakes               2.4.0       passive checker of Python programs
pygments               2.11.2      Pygments is a syntax highlighting package written in Python.
pyjwt                  2.3.0       JSON Web Token implementation in Python
pyodbc                 4.0.32      DB API Module for ODBC
pyparsing              3.0.6       Python parsing module
pyproj                 3.2.1       Python interface to PROJ (cartographic projections and coordinate transformations library)
pyrsistent             0.18.1      Persistent/Functional/Immutable data structures
pytest                 6.2.5       pytest: simple powerful testing with Python
pytest-dotenv          0.5.2       A py.test plugin that parses environment files before running tests
python-dateutil        2.8.2       Extensions to the standard Python datetime module
python-dotenv          0.15.0      Add .env support to your django/flask apps in development and deployments
pytz                   2021.3      World timezone definitions, modern and historical
pyyaml                 6.0         YAML parser and emitter for Python
pyzmq                  22.3.0      Python bindings for 0MQ
rasterio               1.2.10      Fast and direct raster I/O for use with Numpy and SciPy
requests               2.27.1      Python HTTP for Humans.
requests-oauthlib      1.3.0       OAuthlib authentication support for Requests.
rope                   0.22.0      a python refactoring library...
rtree                  0.9.7       R-Tree spatial index for Python GIS
scikit-learn           0.24.2      A set of python modules for machine learning and data mining
scipy                  1.7.3       SciPy: Scientific Library for Python
send2trash             1.8.0       Send file to trash natively under Mac OS X, Windows and Linux.
setuptools-scm         6.3.2       the blessed package to manage your versions by scm tags
shapely                1.8.0       Geometric objects, predicates, and operations
six                    1.16.0      Python 2 and 3 compatibility utilities
snowballstemmer        2.2.0       This package provides 29 stemmers for 28 languages generated from Snowball algorithms.
snuggs                 1.4.7       Snuggs are s-expressions for Numpy
starlette              0.13.6      The little ASGI library that shines.
statsmodels            0.13.1      Statistical computations and models for Python
terminado              0.12.1      Tornado websocket backend for the Xterm.js Javascript terminal emulator library.
testpath               0.5.0       Test utilities for code working with files and commands
threadpoolctl          3.0.0       threadpoolctl
toml                   0.10.2      Python Library for Tom's Obvious, Minimal Language
tomli                  1.2.3       A lil' TOML parser
tornado                6.1         Tornado is a Python web framework and asynchronous networking library, originally developed at FriendFeed.
traitlets              5.1.1       Traitlets Python configuration system
typed-ast              1.4.3       a fork of Python 2 and 3 ast modules with type comment support
typing-extensions      4.0.1       Backported and Experimental Type Hints for Python 3.6+
typing-inspect         0.7.1       Runtime inspection utilities for typing module.
uamqp                  1.5.1       AMQP 1.0 Client Library for Python
urllib3                1.26.8      HTTP library with thread-safe connection pooling, file post, and more.
utm                    0.7.0       Bidirectional UTM-WGS84 converter for python
uvicorn                0.13.4      The lightning-fast ASGI server.
virtualenv             20.13.0     Virtual Python Environment builder
vulture                2.3         Find dead code
wcwidth                0.2.5       Measures the displayed width of unicode strings in a terminal
webencodings           0.5.1       Character encoding aliases for legacy web content
wrapt                  1.13.3      Module for decorators, wrappers and monkey patching.
xgboost                1.4.2       XGBoost Python Package
xyzservices            2022.1.0    Source of XYZ tiles providers
zipp                   3.7.0       Backport of pathlib-compatible object wrapper for zip files
has2k1 commented 2 years ago

I could not replicate this failure!

TobiRoby commented 2 years ago

Thx for the feedback.

Feels like a wierd error to me. I have the pyproject.toml reproducing the error 100% of the time on my system/dockerfile. But it looks like it is also depending on my environment especially the dependency manager (poetry).

[tool.poetry]
name = "plotnine-failure"
version = "0.1.0"
description = ""
authors = ["xxx"]

[tool.poetry.dependencies]
python = "~3.9.10"
pytest = "6.2.5"
pandas = "1.4.0"
plotnine = "0.8.0"
numpy = "1.22.2"

[tool.poetry.dev-dependencies]
flake8 = "4.0.1"
mypy = "0.812"
pre-commit = "2.16.0"
pydocstyle = "6.1.1"
black = "21.12b0"
isort = "5.10.1"
notebook = "^6.4.6"
rope = "^0.22.0"

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

The strangest part: I somehow got it working. I downgraded plotnine, pandas & numpy until the point of that numpy binaries were compiled on install. (<1.18.0 if I remember correctly)

Feel free to try and reproduce the error with poetry on your system. Otherwise, resolve this issue, since I am not sure that it relates to this project at all :)

has2k1 commented 2 years ago

This bug is a result of a change in API of matplotlib >= 3.5.0.

Same issue as: #567

TobiRoby commented 2 years ago

thanks for the follow up. will the fix be part of an upcoming release?

has2k1 commented 2 years ago

thanks for the follow up. will the fix be part of an upcoming release?

Yes.