search

mesonbuild / meson-python

Meson PEP 517 Python build backend
https://mesonbuild.com/meson-python/
MIT License
118 stars 59 forks source link

Editable installs improvements #569

Closed dnicolodi closed 4 months ago

lesteve commented 4 months ago

Thanks for this PR!

I gave in a go to see whether that would fix the issue we were seeing in scikit-learn where some of our tests are using pkgutil.walk_packages as I mentioned in https://github.com/mesonbuild/meson-python/issues/557#issuecomment-1891684893.

Right now if I try to run the problematic test with pytest, I get an error AttributeError: 'MesonpyPathFinder' object has no attribute 'find_spec', see below for full stack-trace.

Let me know if I can provide additional information!

Full stack-trace ``` ❯ pytest sklearn/tests/test_common.py -k are_importable + /home/lesteve/micromamba/envs/del/bin/ninja ninja: no work to do. Traceback (most recent call last): File "", line 198, in _run_module_as_main File "", line 88, in _run_code File "/home/lesteve/micromamba/envs/del/lib/python3.12/site-packages/pytest/__main__.py", line 5, in raise SystemExit(pytest.console_main()) ^^^^^^^^^^^^^^^^^^^^^ File "/home/lesteve/micromamba/envs/del/lib/python3.12/site-packages/_pytest/config/__init__.py", line 197, in console_main code = main() ^^^^^^ File "/home/lesteve/micromamba/envs/del/lib/python3.12/site-packages/_pytest/config/__init__.py", line 155, in main config = _prepareconfig(args, plugins) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/lesteve/micromamba/envs/del/lib/python3.12/site-packages/_pytest/config/__init__.py", line 337, in _prepareconfig config = pluginmanager.hook.pytest_cmdline_parse( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/lesteve/micromamba/envs/del/lib/python3.12/site-packages/pluggy/_hooks.py", line 501, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/lesteve/micromamba/envs/del/lib/python3.12/site-packages/pluggy/_manager.py", line 119, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/lesteve/micromamba/envs/del/lib/python3.12/site-packages/pluggy/_callers.py", line 138, in _multicall raise exception.with_traceback(exception.__traceback__) File "/home/lesteve/micromamba/envs/del/lib/python3.12/site-packages/pluggy/_callers.py", line 121, in _multicall teardown.throw(exception) # type: ignore[union-attr] ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/lesteve/micromamba/envs/del/lib/python3.12/site-packages/_pytest/helpconfig.py", line 104, in pytest_cmdline_parse config = yield ^^^^^ File "/home/lesteve/micromamba/envs/del/lib/python3.12/site-packages/pluggy/_callers.py", line 102, in _multicall res = hook_impl.function(*args) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/lesteve/micromamba/envs/del/lib/python3.12/site-packages/_pytest/config/__init__.py", line 1094, in pytest_cmdline_parse self.parse(args) File "/home/lesteve/micromamba/envs/del/lib/python3.12/site-packages/_pytest/config/__init__.py", line 1446, in parse self._preparse(args, addopts=addopts) File "/home/lesteve/micromamba/envs/del/lib/python3.12/site-packages/_pytest/config/__init__.py", line 1320, in _preparse self.pluginmanager.consider_preparse(args, exclude_only=False) File "/home/lesteve/micromamba/envs/del/lib/python3.12/site-packages/_pytest/config/__init__.py", line 727, in consider_preparse self.consider_pluginarg(parg) File "/home/lesteve/micromamba/envs/del/lib/python3.12/site-packages/_pytest/config/__init__.py", line 753, in consider_pluginarg self.import_plugin(arg, consider_entry_points=True) File "/home/lesteve/micromamba/envs/del/lib/python3.12/site-packages/_pytest/config/__init__.py", line 801, in import_plugin __import__(importspec) File "", line 1360, in _find_and_load File "", line 1322, in _find_and_load_unlocked File "", line 1262, in _find_spec File "/home/lesteve/micromamba/envs/del/lib/python3.12/site-packages/_pytest/assertion/rewrite.py", line 104, in find_spec spec = self._find_spec(name, path) # type: ignore ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 1524, in find_spec File "", line 1498, in _get_spec AttributeError: 'MesonpyPathFinder' object has no attribute 'find_spec' ```
dnicolodi commented 4 months ago

Thanks for testing. I cut some corners in the implementation. It seems that pytest uses one of the interfaces I haven't implemented. I'll add it when I find a moment.

lesteve commented 4 months ago

Great, let me know if there is something I can help with!

dnicolodi commented 4 months ago

@lesteve Can you try again, please?

lesteve commented 4 months ago

Great! This seems to fix both https://github.com/mesonbuild/meson-python/issues/557#issuecomment-1891684893 and https://github.com/mesonbuild/meson-python/issues/568.

lesteve commented 4 months ago

I just saw that this was added on the 0.16.0 milestone, which is great news, let me know if there is anything I can do to help move it forward!

rgommers commented 4 months ago

Yes, I hope we can get this in and released soon. I need to finish testing and reviewing; holidays interfered a bit. I'll try to finalize my review in the next couple of days.