scikit-build / scikit-build-core

A next generation Python CMake adaptor and Python API for plugins
https://scikit-build-core.readthedocs.io
Apache License 2.0
247 stars 52 forks source link

`AssertionError:` "Editable installs cannot rebuild an absolute wheel.install-dir. Use an override to change if needed." #909

Open Jacobfaib opened 2 months ago

Jacobfaib commented 2 months ago

Given

[tool.scikit-build]
wheel.install-dir = "/data"

It is not possible to do editable installs:

$ pip install -e .

because scikit_build.build.wheel.py::_make_editable() throws.

Commenting out this check did not seem to break anything immediately with the wheel, so perhaps this check can be moved to runtime install of install-time.

henryiii commented 2 months ago

You can't install editably to /data. See https://peps.python.org/pep-0660/#limitations

With regard to the wheel .data directory, this PEP focuses on making the purelib and platlib categories (installed into site-packages) “editable”. It does not make special provision for the other categories such as headers, data and scripts. Package authors are encouraged to use console_scripts, make their scripts tiny wrappers around library functionality, or manage these from the source checkout during development.

henryiii commented 2 months ago

Though, it's possible that it could still work as long as you don't want the editable parts to be in /data, we should to verify that. The check might be too restrictive.

Jacobfaib commented 2 months ago

Though, it's possible that it could still work as long as you don't want the editable parts to be in /data, we should to verify that.

Yes, that is exactly what my build is doing. Some pieces are being installed into /data (and I'm OK that those aren't "editable" in the python sense), but other parts are being installed normally into site-packages. And those work with editable installs.

Hence why commenting out the assert did not brick the build.

Jacobfaib commented 2 months ago

So is it possible for scikit-build-core to defer this check? It looks like the generated _<package>_editable.py::install() has a rebuild parameter that is passed to ScikitBuildRedirectingFinder(). This in turn is then used in ScikitBuildRedirectingFinder.find_spec():

# ...
if self.rebuild_flag: # A.K.A. "rebuild"
    self.rebuild()

Perhaps the "is install-dir absolute" check can be put inside self.rebuild, that way it only fires if the feature is actually requested (and exercised).

LecrisUT commented 2 months ago

Another approach that we should eventually support is to not run cmake --install at all and redirect the editable paths to the build directory. Would that also be a workable workflow for you? You would need to do some manipulation to make the build directory be workable, without the wheel.install-dir = "/data" assumption.

Jacobfaib commented 2 months ago

cmake --install at all and redirect the editable paths to the build directory.

That would be even better. I am stuffing a bunch of non-python stuff (big fat shared libs) into the wheel. Copying these into the wheel takes a long time, so if scikit-build-core could skip doing that for editable builds and just leave them in the build directory that would dramatically speed up our workflow