kedro-org / kedro

Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.
https://kedro.org
Apache License 2.0
9.91k stars 900 forks source link

Broken tests because of pluggy patching #2979

Closed astrojuanlu closed 1 year ago

astrojuanlu commented 1 year ago

I spotted in https://github.com/kedro-org/kedro/pull/2966 that the tests were broken with this error:

mocker = <pytest_mock.plugin.MockFixture object at 0x7fd4baf121d0>

    @pytest.fixture
    def fake_plugin_distribution(mocker):
        fake_entrypoint = FakeEntryPoint()
        fake_distribution = FakeDistribution(
            entry_points=(fake_entrypoint,),
            metadata={"name": fake_entrypoint.name},
            version="0.1",
        )
>       mocker.patch(
            "pluggy._manager.importlib_metadata.distributions",
            return_value=[fake_distribution],
        )

tests/framework/cli/test_cli_hooks.py:76: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/site-packages/pytest_mock/plugin.py:193: in __call__
    return self._start_patch(self.mock_module.patch, *args, **kwargs)
/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/site-packages/pytest_mock/plugin.py:157: in _start_patch
    mocked = p.start()
/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/unittest/mock.py:1595: in start
    result = self.__enter__()
/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/unittest/mock.py:1431: in __enter__
    self.target = self.getter()
/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/unittest/mock.py:1618: in <lambda>
    getter = lambda: _importer(target)
/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/unittest/mock.py:1261: in _importer
    thing = _dot_lookup(thing, comp, import_path)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

thing = <module 'pluggy._manager' from '/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/site-packages/pluggy/_manager.py'>
comp = 'importlib_metadata', import_path = 'pluggy._manager.importlib_metadata'

    def _dot_lookup(thing, comp, import_path):
        try:
            return getattr(thing, comp)
        except AttributeError:
>           __import__(import_path)
E           ModuleNotFoundError: No module named 'pluggy._manager.importlib_metadata'; 'pluggy._manager' is not a package

(https://github.com/kedro-org/kedro/actions/runs/5992021777/job/16251000686?pr=2966)

This is the offending code:

https://github.com/kedro-org/kedro/blob/3f87895ed0e54d55be4994c462fb2030c3e13493/tests/framework/cli/test_cli_hooks.py#L68-L80

And probably has to do with pluggy 1.3.0 released two days ago https://pypi.org/project/pluggy/1.3.0/

astrojuanlu commented 1 year ago

Pluggy 1.3.0 dropped support for Python 3.7, so maybe the most sensible course of action for now is to cap the release.

noklam commented 1 year ago

I did some digging, 1.3.0 drop Python 3.7 support and import importlib.metadata instead of importlib_metadata thus breaking test.

https://github.com/pytest-dev/pluggy/compare/1.2.0...1.3.0