Closed leycec closed 2 years ago
Reasonable. Now I posted #9826 to reduce the dependency to the distutils module. I hope it helps you.
After merging it, we still have 4 references in our code. We need to investigate their successor.
$ grep -r distutils sphinx tests setup.py | grep import
sphinx/setup_command.py:from distutils.cmd import Command
sphinx/setup_command.py:from distutils.errors import DistutilsExecError
tests/roots/test-setup/setup.py:from distutils.core import setup
setup.py:from distutils import log
ありがとうございます。Thanks so much, Takeshi! Your volunteer time is scarce, precious, and very much appreciated here.
I fully agree that and install- and test-time imports of distutils
are much less mission-critical than runtime imports of distutils
. Thankfully, you've already resolved all of the latter – which is as unexpected as it is impressive. You're the heroic Godzilla of the Sphinx world. Please take this warm hug for your continual good deeds. :hugs:
Now I merged #9826 to 4.x branch. So it will be released as 4.3.0 soon (maybe in this week). But I'll keep this open until removing other usages.
Note: I posted #10042 to reduce the usage of docutils. After merging it, we'll still have two implementations.
distutils.cmd.Command
. It will be marked as deprecated since v5.0 and removed at v7.0 (refs: #10040). So I'll keep it as is until removal.extract_messages
and compile_messages
commands from babel to create our package. We have to wait for the successors of them.FWIW, you can replace the import distutils
with import setuptools
and things should work the exact same; both in your setup.py
as well as sphinx.setuptools
.
The sneaky thing is the pip
already does this for you, by importing setuptools before running the setup.py file; which means you shouldn't see any change in behaviours by replacing distutils with setuptools.
Now we only have two importings:
$ grep -r distutils sphinx tests setup.py | grep import
sphinx/setup_command.py:from distutils.cmd import Command
sphinx/setup_command.py:from distutils.errors import DistutilsExecError
Is it possible to replace them by import setuptools
? I tried to replace them by from setuptools._distutils... import ...
, but it failed. Either way, we need to support old packages also. So it's not in a hurry replacing, IMO.
from setuptools import Command
And
from setuptools.errors import ExecError
It seems setuptools.errors
has been available since v60.2.0. To replace it, we need to update our dependencies. It's unacceptable.
https://github.com/pypa/setuptools/commit/974dbb03769a500c5077d37291817f30359a0d7b#diff-d46ab9d69835e789642a521f470657990ff684ea71c8df818328c54bf19f3cd5
Note: Ubuntu-18.04 ships setuptools-39.0.1. https://packages.ubuntu.com/ja/bionic/python-setuptools
v60.2.0.
59.0.0, but your point still stands.
To replace it, we need to update our dependencies.
Could you clarify this further? What would you need here, other than a setuptools > 59.0.0
constraint? Or, is that the constraint that you consider unacceptable?
I'm not sure how you model the relationship with redistributors like Ubuntu, but they're not going to be adding the latest Sphinx release without adding a newer version of setuptools. Users should generally use virtual environments and be isolated from their redistributor-provided setuptools anyway. If they can't do that, then they'd be restricted to whatever their redistributor (i.e. Ubuntu maintainers) provides; so it's not like they should be using a newer version of Sphinx anyway.
I'll also note that you can still do something like: https://github.com/scikit-image/scikit-image/pull/6044/files#diff-60f61ab7a8d1910d86d9fda2261620314edcae5894d5aaa236b821c7256badd7R14
try:
from setuptools.errors import ... as ...
except ImportError:
from distutils.errors import ... as ...
That would fix this issue and still be equally backwards compatible.
At the current release schedule, Sphinx 7 will be released in 2024, and Python 3.12 in October 2023, so we either need to close this as the setuptools integration is deprecated and we won't support it, or replace the imports with imports from setuptools
.
A
Clicked the wrong button.
A
Btw, if you distribute pyproject.toml, you can requests pip to install into isolated sandbox a version of setuptools that you need as far as I understand. That should help with end-users installing this project assuming they have new enough pip to understand PEP517. As for distros like Ubuntu 18.04 LTS, I doubt they will ever update Sphinx. They will for remainder of their existence have Sphinx 1.6.7.
Sphinx doesn't use pyproject.toml
(I have a proposal for that at #10356). We only support the latest released Sphinx, so 1.6.7 is not relevant here.
The real issue is do we care about supporting the intersection of "Python >= 3.12" and "using deprecated setup.py
commands". If we do, the solution is pretty simple, as @pradyunsg pointed out (s/distutils/setuptools/
).
A
I'll repeat something I've said before in various places: just because a redistributor ships an old version of a software does not mean that the upstream developers/maintainers are responsible for extending support for that version -- the redistributor is on the hook for that.
@pradyunsg kind of. But with PEP517 you can have best out of both worlds as said so you can define which version of setuptools you need and pip will automatically obtain it. My comment was as @AA-Turner said in the end offtopic as there is a second issue for that.
Unfortunately, our software is not perfect. It would contain a lot of bugs. Actually, we've fixed these bugs on each release. I think it's a good way to support old platforms and libraries to help real users.
Additionally, our software is not only for python developers. They might not know about python at all. So it's difficult to force to use virtualenv to install or upgrade Sphinx.
I think try-expect fallback is a good approach.
Note that pip will transparently create build virtualenvs with select build deps installed in its current versions. You actually have to have an expert running through things to avoid this.
Describe the bug
Python 3.10 officially deprecated the standard
distutils
package after standardization of PEP 632 -- Deprecate distutils module. Python 3.12 is slated to entirely remove that package – and justifiably so. It's an ongoing dumpster fire that can't be put out soon enough. :fire:Sadly, Sphinx still widely imports from
distutils
. GitHub shows at least twelve pending references. As a very temporary circumvention, globally replacingdistutils
withsetuptools._distutils
should suffice to resolve this. Of course, that's also guaranteed to blow up. Like CPython,setuptools
intends to disembowel and eventually removesetuptools._distutils
. Ain't nobody got volunteer time to maintain cruft in perpetuity.Does This Really Matter Now?
Yup. I author @beartype, which doesn't necessarily play well with Sphinx's
autodoc
extension (as detailed here). I've kludged around that incompatibility on my end with ad-hoc and inexplicably dangerous heuristics that conditionally disable @beartype when we vaguely thinkautodoc
might be active. Nobody actually knows whenautodoc
is active, of course. There's currently no public Sphinx API to externally detect that. Cue #9805.</ahem>
Because that's dangerous, we exercise that with a non-trivial functional test programmatically running
sphinx-build
from within the activepytest
process by calling thesphinx.cmd.build.main()
entry point. Works great – under Python < 3.10, that is. Under Python ≥ 3.10, Sphinx emits deprecation warnings all over the place like a flailing water buffalo besieged by leeches. For safety, our test suite treats any warning as a test failure. Chaos ensues with extremepytest
tracebacks resembling:To preserve our sanity, we're currently just ignoring that test under Python ≥ 3.10. We're not proud – but sometimes you just gotta get to sleep.
Thanks for all the Structured Docos
As always, thanks a well-documented ton for all the amazing volunteer work everyone does here. Sphinx rocks! May this issue find the final reST it so deserves. :wink:
How to Reproduce
Just look at the Sphinx codebase, maybe? Not sure what to say here. Badness is bad. The bitrot has gotta be removed – ideally sooner than later.
Expected behavior
...that Sphinx not import from
distutils
. Work with me here, people.Your project
The problem is in Sphinx itself. Ain't no small project gonna help.
Screenshots
Let's be honest: nobody wants another screenshot of my puce monochrome CLI ViM setup. Nobody.
OS
Gentoo Linux, of course! O_o
Python version
3.10
Sphinx version
4.2.0
Sphinx extensions
Irrelevant.
Extra tools
Irrelevant.
Additional context
Irrelevant.