Releasing sphinx-contrib packages using implicit namespace packages

[tjni]

Is your feature request related to a problem? Please describe.

I am working on updating sphinx and sphinxcontrib packages in nixpkgs, which has its own quirks in its method for Python packaging. I don't fully understand why, but mixing packages that use setuptool's legacy namespace packages with those that use implicit namespace packages (e.g. sphinxcontrib-jquery, sphinxcontrib-applehelp) cause the latter to be considered missing when importing them.

I cannot reproduce this in a virtualenv, so it could be Nix specific, but it's also strange enough that I fear sinking tons of time into it if there's already a plan to migrate and release all the sphinxcontrib packages using either setuptools's or flit's support for implicit namespace packages. I am also willing to help with testing, PRs, or in any other way.

[AA-Turner]

Are you able to reproduce this outwith Nix?

[tjni]

I have not been able to reproduce this yet outside of Nix, so please treat this as a feature request instead of a bug report.

[tjni]

I do have a lead on what's happening and, to make a long story short, it seems to be due to behavior from a *-nspkg.pth file that setuptools generates in its legacy namespace packages path. It doesn't quite work when different packages, some which contain these files and some which do not, are installed in disjoint directories (which nix does, unlike a virtualenv). If this is a bug, it is one in setuptools. If it is hard to fix, we will probably work around it in nixpkgs.

With that said, I still am interested in releases of the sphinxcontrib modules that migrate away from this legacy setuptools code path that could let us clean up any potential workarounds, and willing to help in any way.

[tjni]

In case it's interesting to you, I described this in more detail in https://github.com/pypa/setuptools/discussions/3991.

[tjni]

I have an approach to fix this in nixpkgs by post-processing some of these packages to delete those nspkg.pth files. I would like to gradually rely less and less on that over time. Would you be amenable if I worked on migrating to implicit namespaces?

[AA-Turner]

Would you be amenable if I worked on migrating to implicit namespaces?

Yes, feel free to tag me.

A

[AA-Turner]

I have just made a new release of each of the first party sphinxcontrib packages, though the PyPI uploads are currently prevented by https://github.com/pypa/trove-classifiers/pull/150.

A

[tjni]

Thank you so much!

[AA-Turner]

Now released are:

• https://pypi.org/project/sphinxcontrib-applehelp/1.0.5/
• https://pypi.org/project/sphinxcontrib-devhelp/1.0.3/
• https://pypi.org/project/sphinxcontrib-htmlhelp/2.0.2/
• https://pypi.org/project/sphinxcontrib-qthelp/1.0.4/
• https://pypi.org/project/sphinxcontrib-serializinghtml/1.1.6/
• https://pypi.org/project/sphinxcontrib-websupport/1.2.5/

All with PEP 420 packaging. All other sphinxcontrib packages are third party (though again feel free to tag me on issues for review).

A

[tjni]

Thank you very much again for all your help. It's really tedious work, and I appreciate it.