Issues with editable install

I'm using scikit-build-core version 0.9.4 from conda-forge

(python 3.10 and 3.11, same symptoms)

(this is all in conda environments, which hopefully shouldn't matter)

1st -- it all seems to work fine with:

pip install ./

But when I use:

pip install -e ./

I get very strange importing issues.

I have a large package with a deeply nested structure, including a bunch of Cython

The cython stuff seems to working fine, so that's good :-)

Issue 1:

I have a sub-package (it's a few layers down):

outputtters
    __init__.py
    outputter.py
    erma_data_package.py
    erma_data_package
        data_file1
        data_file2
        ....

The erma_data_package dir has some data files need by the module -- no python files in there, and no init.py

In erma_data_package.py, there is:

from .outputter import Something

This all works fine with a regular install, and worked fine with setuptools develop mode.

However, when I use editable install with scikit build, I get:

  File "/Users/chris.barker/Hazmat/GitLab/pygnome/py_gnome/gnome/outputters/erma_data_package.py", line 29, in <module>
    from .outputter import Outputter, BaseOutputterSchema
ModuleNotFoundError: No module named 'gnome.outputters.erma_data_package.outputter'

huh? why is it looking for outputter in erma_data_package ?

that same import line works fine for a number of other modules in that subpackage.

If we change the import to be absolute, instead of relative, it does work.

So -- all I can think is that the dir with the same name as the package is confusing it in editable mode -- even though there is no init.py in there.

Renaming that dir does solve the problem.

I suspect the issue is with the _*_editable.py file -- I haven't looked closely, but it seems to be dong a bunch of logic to find dirs to include -- it may assume any dir inside a package is a package dir.

Is there any documentation of how that's supposed to work?

-CHB

Does editable mode do anything else to the importing other than adding stuff to sys.path ???

Does editable mode do anything else to the importing other than adding stuff to sys.path ???

There are two editable modes. The inplace mode is simpler and does pretty much just add stuff to sys.path, and probably would work (with the standard "inplace" downsides - it's about how setuptools worked). You can set it with editable.mode = "inplace". See https://scikit-build-core.readthedocs.io/en/latest/configuration.html#editable-installs.

The default editable mode is nicer, but a lot more complicated. It's multiplexing the built directory into the local directory, and one place it fails is with data folders (IIRC it might depend on Python version). There are several open issues, and it's something we need to work on. It's tricky, largely due to the fact that it's very Python version dependent - for example, 3.12 is the first version of Python to allow a folder to be returned for importlib.resources.files (along with a back port in some version of importlib_resources).

We also need to add editable install support to the hatchling extension.

scikit-build / scikit-build-core

Issues with editable install #757

Issue 1: