ipwnponies / pytest-antilru

Bust functools.lru_cache when running pytest to avoid test pollution
MIT License
20 stars 3 forks source link

Support Python 3.8+ #21

Closed wwuck closed 2 years ago

wwuck commented 2 years ago

Is there any plan to support Python 3.8 and above? Would any of the additions to lru_cache in 3.8 and 3.9 affect this plugin?

ipwnponies commented 2 years ago

Sure, I can look into adding 3.8-3.10 support. I'm not aware of any changes in 3.7+ that fundamentally change the interface or implementation. Let me know otherwise.

wwuck commented 2 years ago

I've found one bug when running pytest on pandas code with pytest-antilru.

Example code: https://github.com/wwuck/pandas-pytest-antilru

If I run the test code without pytest-antilru, then pytest runs fine and hits the assert.

If I install pytest-antilru alongside pandas and pytest then the assertion is never hit.

$ pytest
====================================================== test session starts ======================================================
platform linux -- Python 3.9.10, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /home/tc/work/public/antilru-pandas-test
plugins: antilru-1.0.5
collected 1 item                                                                                                                

test_program.py F                                                                                                         [100%]

=========================================================== FAILURES ============================================================
________________________________________________________ test_basic_csv _________________________________________________________

    def test_basic_csv():
>       result = pd.read_csv('test.csv')

test_program.py:5: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/home/user/.pyenv/versions/3.9.10/envs/venv/lib/python3.9/site-packages/pandas/util/_decorators.py:311: in wrapper
    return func(*args, **kwargs)
/home/user/.pyenv/versions/3.9.10/envs/venv/lib/python3.9/site-packages/pandas/io/parsers/readers.py:680: in read_csv
    return _read(filepath_or_buffer, kwds)
/home/user/.pyenv/versions/3.9.10/envs/venv/lib/python3.9/site-packages/pandas/io/parsers/readers.py:575: in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
/home/user/.pyenv/versions/3.9.10/envs/venv/lib/python3.9/site-packages/pandas/io/parsers/readers.py:933: in __init__
    self._engine = self._make_engine(f, self.engine)
/home/user/.pyenv/versions/3.9.10/envs/venv/lib/python3.9/site-packages/pandas/io/parsers/readers.py:1217: in _make_engine
    self.handles = get_handle(  # type: ignore[call-overload]
/home/user/.pyenv/versions/3.9.10/envs/venv/lib/python3.9/site-packages/pandas/io/common.py:661: in get_handle
    if _is_binary_mode(path_or_buf, mode) and "b" not in mode:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

handle = 'test.csv', mode = 'r'

    def _is_binary_mode(handle: FilePath | BaseBuffer, mode: str) -> bool:
        """Whether the handle is opened in binary mode"""
        # specified by user
        if "t" in mode or "b" in mode:
            return "b" in mode

        # exceptions
        text_classes = (
            # classes that expect string but have 'b' in mode
            codecs.StreamWriter,
            codecs.StreamReader,
            codecs.StreamReaderWriter,
        )
        if issubclass(type(handle), text_classes):
            return False

>       return isinstance(handle, _get_binary_io_classes()) or "b" in getattr(
            handle, "mode", mode
        )
E       TypeError: decorating_function() missing 1 required positional argument: 'user_function'

/home/user/.pyenv/versions/3.9.10/envs/venv/lib/python3.9/site-packages/pandas/io/common.py:1128: TypeError
==================================================== short test summary info ====================================================
FAILED test_program.py::test_basic_csv - TypeError: decorating_function() missing 1 required positional argument: 'user_function'
======================================================= 1 failed in 0.33s =======================================================

If this is not a bug in pytest-antilru then I can report it to the pandas issue tracker.

ipwnponies commented 2 years ago

My conjecture is the 3.8 change to lru_cache decorator interface:

Changed in version 3.8: Added the user_function option.

In layman's terms:

@lru_cache() # Use default params
def py37_func():
   ...

@lru_cache # Look ma, a new way to decorate and use default params
def py38_func():
   ...

Let me repo to confirm, my tests are green so I'm missing test coverage on this feature.

ipwnponies commented 2 years ago

Released v1.1.0 and is available in pypi.

This adds full support for 3.8 and 3.9. Packages using the newer lru_cache interface should be working again.