Closed Thf772 closed 9 months ago
I have not had to work with namespace packages before. Reading through PEP420 I find:
... The new namespace package:
Has a __path__ attribute set to an iterable of the path strings that were found and recorded during the scan.
Does not have a __file__ attribute.
I read the second point as saying that <ns_mod>.__file__
should not exist (so AttributeError
). From #29 I take it that for the version(s) of python you use, instead __file__
exists, but is None
. Am I misreading, or is the PEP incorrect?
I have a minimal working example here: https://gitlab.com/Thf772/setuptools-dso-multi-namespace-test
Could you translate this example into a test case? Like project2
under example/
. With this, a GHA run should answer my question about portability/history.
$ git grep project2
.github/workflows/build.yml: cd project2
.github/workflows/build.yml: # install project2.
.github/workflows/build.yml: cd project2
.github/workflows/build.yml: (cd example/project2/src && python -m use_dsodemo.cli)
testme.sh:(cd project2/src && python -m use_dsodemo.cli 2>/dev/null) && die "error: worktree must be clean"
testme.sh:cd project2
testme.sh:(cd project2/src && PYTHONPATH=$X python -m use_dsodemo.cli)
testme.sh:cd project2
testme.sh:(cd project2/src && python -m use_dsodemo.cli 2>/dev/null) && die "error: worktree must be clean"
testme.sh:cd project2
From https://github.com/mdavidsaver/setuptools_dso/pull/29 I take it that for the version(s) of python you use, instead file exists, but is None. Am I misreading, or is the PEP incorrect?
I'm using Python 3.11, I don't know if that makes any difference. I don't think you've misread the PEP, it sounds like it should trigger an AttributeError
but maybe it stays there as None
for some API compatibility reasons, or some CPython-specific shenanigans...
I'll try setting up a test case for namespace packages.
Alright, I've added tests for both editable and non-editable installs of namespace packages. These tests pass on my computer with Python 3.11 and Windows 10. Let me know if there's anything you think I should change or improve.
@mdavidsaver any news on the PR workflow? This issue is still blocking for my project.
Thanks for the run, I think the tests failed because of a stupid mistake. I added Cython to the Deps step, but only to the download command and not the install command. So when trying to install my test package that has a Cython file, it looked for the generated C file which wasn't present. It should be fixed now.
Now apparently it's old version of Python not compatible with pyproject builds... I'll have to rewrite the setup files.
I have released 2.10a1, please let me know if this is sufficient.
Thanks for the help, I'm testing the new version right now on my project. I'll let you know how it works.
Alright, everything works for me now! Thanks again.
Also, I'm just noting this here for now, but for newer Python versions, it might be interesting to refactor the code to use setuptool's SubCommand
class, which is the recommended way to add new commands supporting editable installs. I doubt old Setuptools and Python versions support it though, so it depends on whether supporting them is important for you.
I have a minimal working example here: https://gitlab.com/Thf772/setuptools-dso-multi-namespace-test
When a namespace package is installed in editable mode, trying to install an Extension depending on a DSO located in the same namespace package will fail, whether that install is editable or not, on both Linux and Windows.
Steps to reproduce:
nsp/core/__init__.py
nsp.__init__.py
filensp/dso/thedso.c
nsp/dso/theext.pyx
nsp/dso/__init__.py
thedso.c
, and an extension that depends on this DSO usingtheext.pyx
My MWE above uses different names.
Expected result
The second install succeeds
Actual result
An error occurs while building the second project:
What I think happens
Looking at the source, this exception is thrown while Setuptools-DSO is building the possible locations for an existing DSO. It looks for the init file of the root package (
nsp
in the reproduction instructions,testnsp
in my MWE), but since this is a namespace package, this file does not exist. So while the import works (noImportError
so theexcept
clause is not triggered),__file__
containsNone
and causes aTypeError
when fed toos.path.dirname
.To solve it, I think it should be possible to set up a
while
(or even better,for
) loop to look first atparts[0:1]
, see if it's a real package (__file__ is not None
), and if it's not, try again withparts[0:2]
and so on. Since I'm in this exact situation in a project of mine, I might make a PR for this, if no one does it before.