search

unionai-oss / pandera

A light-weight, flexible, and expressive statistical data testing library
https://www.union.ai/pandera
MIT License
3.05k stars 281 forks source link

no exception raise after pipe on 3.11.9 #1559

Closed lukaskiss222 closed 1 month ago

lukaskiss222 commented 1 month ago

Describe the bug Under python 3.11.9, using pipe function on Dataframe does not rise Exception

Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.

Code Sample, a copy-pastable example

import pandera as pa
from pandera.typing import Series, DataFrame
import pandas as pd

class A(pa.DataFrameModel):
    x: Series[int] = pa.Field(
        check_name=True,
    )
    @pa.check("x")
    def check_x(cls, x: Series[int]) -> Series[bool]:
        print("running")
        raise Exception("error")

df = pd.DataFrame({"x": [1, 2, 3]})
# Does not raise exception
df.pipe(DataFrame[A])

Expected behavior

under python 3.11.8 it raises exception

running
Traceback (most recent call last):
  File "/home/lukaskiss/siren/AWS/market-utils/test.py", line 16, in <module>
    df.pipe(DataFrame[A])
  File "/home/lukaskiss/.cache/pypoetry/virtualenvs/market-utils-mQwKEbgz-py3.11/lib64/python3.11/site-packages/pandas/core/generic.py", line 6142, in pipe
    return common.pipe(self, func, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lukaskiss/.cache/pypoetry/virtualenvs/market-utils-mQwKEbgz-py3.11/lib64/python3.11/site-packages/pandas/core/common.py", line 497, in pipe
    return func(obj, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/typing.py", line 1277, in __call__
    result.__orig_class__ = self
    ^^^^^^^^^^^^^^^^^^^^^
  File "/home/lukaskiss/.cache/pypoetry/virtualenvs/market-utils-mQwKEbgz-py3.11/lib/python3.11/site-packages/pandera/typing/common.py", line 179, in __setattr__
    self.__dict__ = schema_model.validate(self).__dict__
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lukaskiss/.cache/pypoetry/virtualenvs/market-utils-mQwKEbgz-py3.11/lib/python3.11/site-packages/pandera/api/pandas/model.py", line 306, in validate
    cls.to_schema().validate(
  File "/home/lukaskiss/.cache/pypoetry/virtualenvs/market-utils-mQwKEbgz-py3.11/lib/python3.11/site-packages/pandera/api/pandas/container.py", line 375, in validate
    return self._validate(
           ^^^^^^^^^^^^^^^
  File "/home/lukaskiss/.cache/pypoetry/virtualenvs/market-utils-mQwKEbgz-py3.11/lib/python3.11/site-packages/pandera/api/pandas/container.py", line 404, in _validate
    return self.get_backend(check_obj).validate(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lukaskiss/.cache/pypoetry/virtualenvs/market-utils-mQwKEbgz-py3.11/lib/python3.11/site-packages/pandera/backends/pandas/container.py", line 97, in validate
    error_handler = self.run_checks_and_handle_errors(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lukaskiss/.cache/pypoetry/virtualenvs/market-utils-mQwKEbgz-py3.11/lib/python3.11/site-packages/pandera/backends/pandas/container.py", line 172, in run_checks_and_handle_errors
    error_handler.collect_error(
  File "/home/lukaskiss/.cache/pypoetry/virtualenvs/market-utils-mQwKEbgz-py3.11/lib/python3.11/site-packages/pandera/error_handlers.py", line 38, in collect_error
    raise schema_error from original_exc
  File "/home/lukaskiss/.cache/pypoetry/virtualenvs/market-utils-mQwKEbgz-py3.11/lib/python3.11/site-packages/pandera/backends/pandas/container.py", line 192, in run_schema_component_checks
    result = schema_component.validate(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lukaskiss/.cache/pypoetry/virtualenvs/market-utils-mQwKEbgz-py3.11/lib/python3.11/site-packages/pandera/api/pandas/components.py", line 169, in validate
    return self.get_backend(check_obj).validate(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lukaskiss/.cache/pypoetry/virtualenvs/market-utils-mQwKEbgz-py3.11/lib/python3.11/site-packages/pandera/backends/pandas/components.py", line 119, in validate
    validate_column(check_obj, column_name)
  File "/home/lukaskiss/.cache/pypoetry/virtualenvs/market-utils-mQwKEbgz-py3.11/lib/python3.11/site-packages/pandera/backends/pandas/components.py", line 89, in validate_column
    error_handler.collect_error(err.reason_code, err)
  File "/home/lukaskiss/.cache/pypoetry/virtualenvs/market-utils-mQwKEbgz-py3.11/lib/python3.11/site-packages/pandera/error_handlers.py", line 38, in collect_error
    raise schema_error from original_exc
  File "/home/lukaskiss/.cache/pypoetry/virtualenvs/market-utils-mQwKEbgz-py3.11/lib/python3.11/site-packages/pandera/backends/pandas/components.py", line 68, in validate_column
    validated_check_obj = super(ColumnBackend, self).validate(
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lukaskiss/.cache/pypoetry/virtualenvs/market-utils-mQwKEbgz-py3.11/lib/python3.11/site-packages/pandera/backends/pandas/array.py", line 69, in validate
    error_handler = self.run_checks_and_handle_errors(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lukaskiss/.cache/pypoetry/virtualenvs/market-utils-mQwKEbgz-py3.11/lib/python3.11/site-packages/pandera/backends/pandas/array.py", line 150, in run_checks_and_handle_errors
    error_handler.collect_error(
  File "/home/lukaskiss/.cache/pypoetry/virtualenvs/market-utils-mQwKEbgz-py3.11/lib/python3.11/site-packages/pandera/error_handlers.py", line 38, in collect_error
    raise schema_error from original_exc
pandera.errors.SchemaError: Error while executing check function: Exception("error")
Traceback (most recent call last):
  File "/home/lukaskiss/.cache/pypoetry/virtualenvs/market-utils-mQwKEbgz-py3.11/lib/python3.11/site-packages/pandera/backends/pandas/components.py", line 212, in run_checks
    self.run_check(
  File "/home/lukaskiss/.cache/pypoetry/virtualenvs/market-utils-mQwKEbgz-py3.11/lib/python3.11/site-packages/pandera/backends/pandas/base.py", line 102, in run_check
    check_result: CheckResult = check(check_obj, *args)
                                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lukaskiss/.cache/pypoetry/virtualenvs/market-utils-mQwKEbgz-py3.11/lib/python3.11/site-packages/pandera/api/checks.py", line 227, in __call__
    return backend(check_obj, column)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lukaskiss/.cache/pypoetry/virtualenvs/market-utils-mQwKEbgz-py3.11/lib/python3.11/site-packages/pandera/backends/pandas/checks.py", line 295, in __call__
    check_output = self.apply(check_obj)
                   ^^^^^^^^^^^^^^^^^^^^^
  File "/home/lukaskiss/.cache/pypoetry/virtualenvs/market-utils-mQwKEbgz-py3.11/lib/python3.11/site-packages/multimethod/__init__.py", line 431, in __call__
    return self[sig](*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lukaskiss/.cache/pypoetry/virtualenvs/market-utils-mQwKEbgz-py3.11/lib/python3.11/site-packages/pandera/backends/pandas/checks.py", line 143, in apply
    return self.check_fn(check_obj)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lukaskiss/.cache/pypoetry/virtualenvs/market-utils-mQwKEbgz-py3.11/lib/python3.11/site-packages/pandera/api/base/model_components.py", line 130, in _adapter
    return self.check_fn(model_cls, arg)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lukaskiss/siren/AWS/market-utils/test.py", line 13, in check_x
    raise Exception("error")
Exception: error

Desktop (please complete the following information):

cosmicBboy commented 1 month ago

Yeah, I've noticed 3.11.9 has a bunch of bugs... looking into this, best guess without knowing more is it may have something to do with typing module changes https://www.python.org/downloads/release/python-3119/