search

conda-incubator / conda-pypi

Better PyPI interoperability for the conda ecosystem
https://conda-incubator.github.io/conda-pypi/
MIT License
12 stars 1 forks source link

Add support for editable pip installs #36

Closed jaimergp closed 2 months ago

jaimergp commented 2 months ago

Closes #35

dholth commented 2 months ago

https://github.com/conda/conda-build/pull/5380 is a very rough outline of how I would do it using pypa/build. Especially it gives us the opportunity to install the build system (flit_core, hatch, ...) with our preferred installer instead of necessarily using pip.

jaimergp commented 2 months ago

Isn't pip installing the build system on its own? We have the concept here of "backend", which you can use with grayskull and pip. But yea it's not super perfect. I'm not sure how much I'd like to reimplement from scratch instead of delegating to pip (or uv in the future).

jaimergp commented 2 months ago

Ah, I see what you mean, installing build-system.requires with conda in... a temporary environment? 🤔

dholth commented 2 months ago

In the draft PR I've linked perhaps you can begin to imagine how we could avoid almost all pip invocation, especially if we include our own wheel installer (or https://github.com/zuzukin/whl2conda to convert to .conda and then install) pypa/build makes the wheel by delegating to each package's build system, setuptools, flit, etc. Our preferred installer takes care of the environment, dependencies. Functions from packaging handle the full PEP 508 Python dependency specifications https://packaging.pypa.io/en/stable/markers.html

In "no build isolation" mode, which is what the draft PR halfheartedly implements, you would install the build system "flit_core" or whatever into the current or target conda environment. If it was more like conda-build a temporary conda environment would be used. The build system could also be installed into a temporary directory that is added to PYTHONPATH, or it is allowed that the build system can come from the tree as in when flit builds flit.

Now make it recursive! Imagine an almost seamless experience.

jaimergp commented 2 months ago

Ok, I see, I like that! Added https://github.com/conda-incubator/conda-pypi/issues/37 to track these suggestions. Let's consider this PR a barebones prototype for now.

jezdez commented 2 months ago

@jaimergp @dholth Hold on, why would we encourage creating wheel files instead of conda files? There isn't a conda specific platform tag, so I have a hard time seeing a way to differentiate the files from regular Python wheel builds.

dholth commented 2 months ago

I'm talking about the ephemeral "installs a .pth to effect an editable install" wheels from https://peps.python.org/pep-0660/ . We would install and then call into the build system, it spits out that wheel.

We would probably pip install the ephemeral wheel, but it might be interesting to turn it into a conda package and install that way.