search

googleapis / python-db-dtypes-pandas

Extension dtypes for pandas corresponding to GoogleSQL data types such as DATE, TIME, and JSON.
https://googleapis.dev/python/db-dtypes/latest/
Apache License 2.0
26 stars 6 forks source link

bug: refine constraint to avoid numpy pandas incompatability #277

Closed chalmerlowe closed 3 months ago

chalmerlowe commented 3 months ago

Pins the version of numpy used with python 3.9 to be less than 2.0.0.

There is a potentially unexpected interaction between pandas and numpy here. Depending on the version we use for each, we get different results in terms of pass/fail for the unit tests associated with python 3.9

numpy version pandas version unit-3.9 pass/fail
2.0.0 1.3.0 fail
2.0.0 2.2.2 pass
2.0.0 1.5.3 fail
1.26.4 1.5.3 pass
1.26.4 1.3.0 pass

This appears to be a known incompatibility between how pip resolves the dependency versions for numpy and pandas and the only current fix is to pin versions to something that "works" to avoid allowing pip to resolve into a failing state. (NOTE the linked issue references various versions of numpy and pandas, depending on the combination, inlcuding 3.9 throughout the thread, even though the title references python 3.12)

Fixes #275 🦕

conventional-commit-lint-gcf[bot] commented 3 months ago

🤖 I detect that the PR title and the commit message differ and there's only one commit. To use the PR title for the commit history, you can use Github's automerge feature with squashing, or use automerge label. Good luck human!

-- conventional-commit-lint bot https://conventionalcommits.org/

tswast commented 2 months ago

Why don't we pin pandas to be 2.2.2 instead?

Ideally, we'd spread out the versions of pandas we test. For pinning one of the newer versions of pandas, that should be on one of the later Python versions.

Note that currently, we test on:

Ideally, we would test on multiple 2.x versions but continue to have the latest supported Python test with latest supported pandas.

Also, we might want to consider bumping our minimum pandas to 1.5.x sometime. That's the latest 1.x release and has the most flexibility of 1.x versions with regards to extension dtypes and pyarrow support.