search

mesonbuild / meson-python

Meson PEP 517 Python build backend
https://mesonbuild.com/meson-python/
MIT License
125 stars 65 forks source link

Pain Points with git worktree #549

Closed jbrockmendel closed 9 months ago

jbrockmendel commented 9 months ago

A few months ago pandas moved to using meson for builds and I have found it does not play nicely with my git worktree-heavy workflow.

In particular, if I have multiple branches checked out in multiple directories, and a terminal window for each of them, then trying to build (with python3 -m pip install -ve . --no-build-isolation) causes one or more to fail at build-time. Running tests in (pytest pandas/tests) in more than one fails at test-collection time. Using a REPL with different branches in different windows is also rough.

I'm kind of hoping this is a user problem where I'm doing something dumb that can easily be changed. bc IIUC changing meson's behavior here would be really tough. Please advise.

rgommers commented 9 months ago

Thanks for the report @jbrockmendel. It looks like the issue here is that you are using editable installs in multiple separate repos, but in the same environment. Is that correct? If so, I would not expect that to work indeed - but I wouldn't expect it to work with any build system. If not, then I'm puzzled.

rgommers commented 9 months ago

I'll also note that for numpy we are using spin as the developer UX. What that does is install the package inside the repo (under build-install/) rather than into site-packages, and hence there would be no issue with having multiple installs in parallel while reusing a single environment.

jbrockmendel commented 9 months ago

but in the same environment. Is that correct? If so, I would not expect that to work indeed - but I wouldn't expect it to work with any build system

I think that's accurate. IIUC meson is symlinking [mumble] into site-packages, which didn't use to happen with python setup.py build_ext --inplace

dnicolodi commented 9 months ago

IIUC meson is symlinking [mumble] into site-packages,

meson-python editable installs do not symlink anything into site-packages. meson-python editable installs produce wheels that contain a custom Python module loader that loads code from the project build and source directory. Only one editable wheel for a given package can be installed in an a Python environment. Therefore you can have only one editable build linked into a Python environment.

The solution is to use a different venv for each version of the code you want to test.

which didn't use to happen with python setup.py build_ext --inplace

In-place builds and editable installs are not the same thing. meson-python does not support in-place build. Modern setuptools does not support them anymore either.