Open jaimergp opened 2 months ago
The PyPA build/installer integration seems backwards to me, as this package is about enabling conda users to successfully use Python packages from PyPI in their conda environments, not build new Python wheels that would be better served by conda packages in the first place.
Of course, it'd be splendid if the .conda
file format would simply be a hypothetical .whl
format plus metadata extensions. But then we're kind of back at square one with conda-press (talk).
This is only for the cases where we didn't find the corresponding conda artifact and the user insists on serving something from PyPI-compatible sources.
If there's only a sdist (or a sdist-compatible repository / directory), pip will build a wheel anyway. What happens behind that subprocess is underspecified and pip will create the build environment on its own with no conda involvement.
The intent here would be to have some control over that build environment; this is particularly important for those packages that require compilation so we provide the compilers and whatnot in an ABI compatible way.
Although at that point we could also just say "grayskull this and run conda-build on top", and that's also a valid option (albeit with its overheads). It would be one more option, I guess?
I don't see this as an immediate priority but I wanted to capture Daniel's thoughts before getting them lost in the sea of merged PRs and closed issues.
Other installers are also trying to perfect the integration of conda and pypi package, I haven't researched those yet. I'm imagining something like
$ conda pypi-build-install-develop --editable <path/to/somepackage> or <[pep 508](https://peps.python.org/pep-0508/) requirement>
<installs enscons or flit or whatever from conda packages, gets metadata>
[y/N] Accept solution?
channels:
- defaults
dependencies:
- aiofiles
- aiohttp
- click
- pip:
- json_log_formatter
- ...
<option to adjust where each package comes from with a configuration file>
Y
<it is installed>
As Jaime says, we gain control over installing the pyproject build-requires from a conda source and we also gain control over the the environment used to build the pypi package.
Possibly an automatic build of a whole tree of packages in the same way that pip wheel
populates a directory with all the wheels needed to install something, after which you can install offline from that directory of wheels. I'm less clear on how this would be designed compared to the simpler "conda develop" replacement.
There are libraries we can use for better integration instead of relying so much on
pip
subprocesses. This could be part of a refactor where we isolate the components of the package a bit better.Right now we have, roughly:
pip
's UX with a couple extra flagsgrayskull
,pip
;uv
is planned toopip
. This could be reimplemented using PyPA'sbuild
andinstaller
soconda
can provide thebuild-system.requirements
(which means we can offer compilers). This would also allow us to inject the right kind of metadata in the*.dist-info
directories, in the right way.cc @dholth
xref: