andrewgsavage
commented
3 months ago can you pytest skip the failing tests and they can be worked on in another PR?
Closed mutricyl closed 3 months ago
andrewgsavage
commented
3 months ago can you pytest skip the failing tests and they can be worked on in another PR?
andrewgsavage
commented
3 months ago and add a note in changes to say eval now works
mutricyl
commented
3 months ago can you pytest skip the failing tests and they can be worked on in another PR?
I agree with you in principle and I will do that. However looking a little bit into details pytest -rf states 357 failed tests. 316 of them are : TypeError: pd.api.extensions.take requires a numpy.ndarray, ExtensionArray, Index, or Series, got NumpyExtensionArray. It may be worth investigating this particular issue instead of waisting to much time in adding @pytest.mark.skip.
mutricyl
commented
3 months ago skipped tests:
NumpyExtensionArray issue in pandas take function https://github.com/pandas-dev/pandas/issues/59177all temporary skip are marked by GH #239
andrewgsavage
commented
3 months ago Looks like there's a unique function issue similar to the take issue
pint_pandas/testsuite/test_pandas_extensiontests.py::TestPintArray::test_value_counts_with_normalize[float] - TypeError: unique requires a Series, Index, ExtensionArray, or np.ndarray, got NumpyExtensionArray. FAILED
andrewgsavage
commented
3 months ago FAILED pint_pandas/testsuite/test_issues.py::TestIssue80::test_div - AssertionError: assert <Quantity(88.294507, 'millisecond')> <= (5 * <Quantity(0.689048, 'millisecond')>)
That test can be skipped, it's been causing issues for a long time
mutricyl
commented
3 months ago Argh, shame on me, I concentrated on fixing pandas 3 but did not test my changes with pandas 2.2.2 ....
mutricyl
commented
3 months ago Looks like there's a unique function issue similar to the take issue
pint_pandas/testsuite/test_pandas_extensiontests.py::TestPintArray::test_value_counts_with_normalize[float] - TypeError: unique requires a Series, Index, ExtensionArray, or np.ndarray, got NumpyExtensionArray. FAILED
I have not seen this test failling. @andrewgsavage do you remember which environment induced this issue ?
andrewgsavage
commented
3 months ago Looks like there's a unique function issue similar to the take issue pint_pandas/testsuite/test_pandas_extensiontests.py::TestPintArray::test_value_counts_with_normalize[float] - TypeError: unique requires a Series, Index, ExtensionArray, or np.ndarray, got NumpyExtensionArray. FAILED
I have not seen this test failling. @andrewgsavage do you remember which environment induced this issue ?
It's in this post https://github.com/hgrecco/pint-pandas/pull/239#issuecomment-2208326717
andrewgsavage
commented
3 months ago nice one
MichaelTiemannOSC
commented
2 months ago I found a problem. If the dataframe consists of mixed PintType elements that are not compatible, the code will now fail. Here's a diff that adds a test case to 137 and fixed the problem in PintArray:
diff --git a/pint_pandas/pint_array.py b/pint_pandas/pint_array.py
index 9c5efab..86560f0 100644
--- a/pint_pandas/pint_array.py
+++ b/pint_pandas/pint_array.py
@@ -203,6 +203,7 @@ class PintType(ExtensionDtype):
In order to be able to be able to perform operation on ``PintType``
with scalars, mix of ``PintType`` and numeric values are allowed.
+ But all ``PintType`` elements must be compatible.
Parameters
@@ -216,7 +217,13 @@ class PintType(ExtensionDtype):
if all(
isinstance(x, PintType) or pd.api.types.is_numeric_dtype(x) for x in dtypes
):
- return self
+ PintType_list = [x for x in dtypes if isinstance(x, PintType)]
+ if len(PintType_list) < 2:
+ return self
+ if all (PintType_list[0].units.is_compatible_with(x.units) for x in PintType_list[1:]):
+ return self
+ else:
+ return None
else:
return None
diff --git a/pint_pandas/testsuite/test_issues.py b/pint_pandas/testsuite/test_issues.py
index 25709c5..c8c8a31 100644
--- a/pint_pandas/testsuite/test_issues.py
+++ b/pint_pandas/testsuite/test_issues.py
@@ -277,3 +277,14 @@ class TestIssue137(BaseExtensionTests):
)
tm.assert_series_equal(df.eval("a / b"), df["a"] / df["b"])
tm.assert_series_equal(df.eval("a / c"), df["a"] / df["c"])
+
+ def test_mixed_df(self):
+ df = pd.DataFrame(
+ {
+ "a": pd.Series([1.0, 2.0, 3.0], dtype="pint[meter]"),
+ "b": pd.Series([4.0, 5.0, 6.0], dtype="pint[second]"),
+ "c": [1.0, 2.0, 3.0],
+ }
+ )
+
+ assert df["a"][0] == df.iloc[0][0]could you submit that as a PR?
MichaelTiemannOSC
commented
2 months ago Done: https://github.com/hgrecco/pint-pandas/pull/245. It exposes some more technical challenges for this PR. To really fix properly, I suspect we need to distinguish from operations that require compatibility (such as additive operations, clipping, quantiles, sorting) and those that don't (which include indexing and multiplicative operations).
pre-commit run --all-fileswith no errors