import pint_pandas fails with pint 0.21: "module 'pint.compat' has no attribute 'upcast_types'"

[apjanke]

Doing an import pint_pandas is erroring out for me today when done in a Python environment with the new pint 0.21 installed. It throws an "AttributeError: module 'pint.compat' has no attribute 'upcast_types'".

[conda-work] $ conda list pint | grep -i pint
pint                      0.21               pyhd8ed1ab_0    conda-forge
pint-pandas               0.3                pyhd8ed1ab_0    conda-forge
[conda-work] $ pip list | grep -i pint
Pint                          0.21
Pint-Pandas                   0.3
[conda-work] $ ipython
Python 3.11.3 | packaged by conda-forge | (main, Apr  6 2023, 09:05:00) [Clang 14.0.6 ]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.13.2 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import pint_pandas
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[1], line 1
----> 1 import pint_pandas

File /opt/conda-envs/jank/lib/python3.11/site-packages/pint_pandas/__init__.py:3
      1 import pprint
----> 3 from .pint_array import PintArray, PintType
      5 try:
      6     from importlib.metadata import version

File /opt/conda-envs/jank/lib/python3.11/site-packages/pint_pandas/pint_array.py:995
    991     except Exception:
    992         return False
--> 995 compat.upcast_types.append(PintArray)

AttributeError: module 'pint.compat' has no attribute 'upcast_types'

In [2]:

Tested this on Windows and macOS 10.14. I can also reproduce it in a fresh Python 3.10 venv with just pip install pint pint-pandas ipython installed.

I looked in the source code for pint 0.21, and indeed it seems there's no upcast_types there. Downgrading to pint 0.20.1 fixed it for me, and import pint_pandas works there. Maybe a compatibility problem with the new pint 0.21? pint 0.21 came out a couple weeks ago I think, and pint-pandas 0.3 is from about six months ago, when pint 0.20 was still the latest. Guessing it's this Draft new upcast_types behavior with xarray and pandas thing?

The setup.cfg as of pint-pandas 0.3 has this, I think:

install_requires =
    pint>=0.20.1

Maybe that should be capped at 0.20.x, so PyPI and Anaconda can recognize the incompatibility? When I do an install of both with conda or pip, or do conda update --all in an env that has pint and pint-pandas, I end up with the incompatible version mix and a broken pint-pandas. (I dunno if there's a different thing that needs to be set up for Anaconda's dependency recognition for the conda-forge package.)

[MichaelTiemannOSC]

For now, if you use Pint-Pandas 0.3, you must pin Pint to 0.20.1. I'm sure that when Pint-Pandas 0.4 is released it will support both the released 0.21 and the soon-to-be-released 0.22.

[apjanke]

Right. But pinning is kind of inconvenient: it requires the user to be aware of this issue, figure it out/choose the right thing to google (this here bug report doesn't seem to be in the google results yet), know what "pinning" is and how to use it in their Python env manager tooling, and configure all their Python envs and virtualenvs to pin pint - if they have admin access to them, which they may not for Anaconda Enterprise or other centrally deployed environments, or envs that other users are creating for running shared code in - which probably involves manually setting up a pinned file in Anaconda, since the conda command doesn't seem to have a command to support pinning definitions, and then if they want to use the newer Pint eventually, monitor the news on pint-pandas releases so they know when it's safe to unpin the Pint version, and then go reconfigure alll their Python environments again.

Like, until the work on the new Pint-Pandas 0.4 minor release is done, maybe there could be an interim 0.3.1 patch release that just changes like that install_requres = pint>=0.20.1 to, say, install_requires = pint>=0.20.1, <0.21.0 (and maybe the Anaconda pint-pandas-feedstock if that needs a separate dependency definition), so this could be taken care of automatically for all pint-pandas users, including those who are getting an indirect dependency on pint-pandas because some other module they depend on depends on pint-pandas? That seems like it would be pretty easy and safe, and then there's no pressure on getting a Pint-Pandas 0.4 release out before it's fully baked.

[andrewgsavage]

There aren't any PRs being worked on atm, so we could just make a 0.4 release which would also fix this issue.

@hgrecco , what are your thoughts, can we get a new release? 0.4 or 0.3.1 would both be preferable to the current situation.

[hgrecco]

Pint 0.21.1rc1 and 0.22rc2 are both passing all tests and the downstream dashboard is green https://github.com/hgrecco/pint/blob/master/downstream_status.md

So I can release 0.21.1 and 0.22 tomorrow if we agree on that.

[hgrecco]

and maybe yank 0.21

[apjanke]

Woot!

[hgrecco]

@andrewgsavage ?

[andrewgsavage]

yes, we will also need a pint-pandas release

[hgrecco]

Just released Pint 0.21.1 and 0.22. And will release Pint-Pandas tomorrow if no showstoppers appear.

[apjanke]

Thanks for the quick attention here!

Seems to fix the issue for me when I installed Pint 0.22 in a venv and pulled in the current pint-pandas repo on top of it:

[test-pint-pandas] $ which python
/Users/janke/tmp/test-pint-pandas/.venv/bin/python
[test-pint-pandas] $ pip list | grep -i pint
Pint              0.22
Pint-Pandas       0.3
[test-pint-pandas] $ export PYTHONPATH=~/repos/pint-pandas
[test-pint-pandas] $ ipython
Python 3.10.11 (main, Apr 20 2023, 13:59:00) [Clang 14.0.6 ]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.13.2 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import pint_pandas

In [2]:

[hgrecco]

Pint-Pandas 0.4 is out

[apjanke]

Lookin' good on an all-PyPI environment for me. Thanks!

pint-pandas-2] $ pip list | grep -i pint
Pint                          0.22
Pint-Pandas                   0.4
[pint-pandas-2] $ ipython
Python 3.10.11 (main, Apr 20 2023, 13:59:00) [Clang 14.0.6 ]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.12.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import pint_pandas

In [2]: exit