Pip deprecation warning on editable install

[dwt]

PROBLEM REPORT

What I did:

# create a venv somehow
$ pip install --editable 'git+https://github.com/zopefoundation/Zope.git@master#egg=Zope'
[…]
  DEPRECATION: Legacy editable install of Zope from git+https://github.com/zopefoundation/Zope.git@master#egg=Zope (setup.py develop) is deprecated. pip 25.0 will enforce this behaviour change. A possible replacement is to add a pyproject.toml or enable --use-pep517, and use setuptools >= 64. If the resulting installation is not behaving as expected, try using --config-settings editable_mode=compat. Please consult the setuptools documentation for more information. Discussion can be found at https://github.com/pypa/pip/issues/11457
[…]
$ python --version
Python 3.12.5
$ pip list | grep -E 'pip|wheel|setuptools'
pip                     24.2
setuptools              73.0.0
wheel                   0.44.0
$ sw_vers
ProductName:        macOS
ProductVersion:     14.6.1
BuildVersion:       23G93

What I expect to happen:

This should install without warning

What actually happened:

The warning that is displayed above. I'm not quite sure yet I understand this warning correctly, but what I read sounds like switching from setup.py to pyproject.toml could solve this?

What version of Python and Zope/Addons I am using:

See above

This warning and the linked GitHub issue, seems to suggest that pip 25 (soonish) might break editable installs of Zope.

Is that an assessment you guys share? If so, is this the point to switch to a pyproject.toml based workflow for Zope?

[dataflake]

My 25¢:

The Zope ecosystem and Zope itself use buildout almost everywhere. The compatibility story with later setuptools and pip versions is a complete mess at this point. I am getting the feeling both buildout and setuptools are essentially unmaintained. That's why we pin setuptools and pip in all buildout and tox configurations.

Moving this one package to pyproject.toml isn't going to help, there is a rat's tail of literally hundreds of packages in the zopefoundation organization that should then be switched as well. It will just add confusion and a large amount of work. As far as I know there's no good equivalent for the buildout ability to create custom scripts, either.

At this point I think buildout and setuptools the way we use them is a total dead end. But moving off of that is a huge mountain to climb. I don't want to end up with a mishmash of tools. Right now we at least have consistency across all packages.

[dwt]

@dataflake I am not sure why you think that setup tools is a dead tool, looking at it's commit activity it seems surprisingly healthy. Setuptool also seems to embrace the pyproject.toml specs.

Regarding buildout however, your perception of it being quite dead seems correct.

My perception of the python packaging ecosystem is, that it is slowly moving all remaining use cases from setup.py to pyproject.toml, editable installs being one of the latest.

In the end, I would really like to avoid a situation where buildout is the only supported way to install and work on Zope.

[dataflake]

I base my comment about setuptools on the 600 unfixed issues, at least one of which is my own describing a serious issue when working with a universal2 Python on macOS. It's one of the reasons why creating and testing macOS wheels for our compiled binary packages which depend on other compiled binary packages is nothing but pain. No answer at all for a simple question in close to two years.

Right now you can avoid issues by doing what we do - pin pip and setuptools. I don't have a better solution until the numerous compatibility issues between buildout, setuptools and pip are fixed. (No, I do not want to become a buildout and/or setuptools developer myself). Patches welcome?

[dwt]

If I understand this documentation correctly, it is probably a good start to add a pyproject.toml file with this content to the Zope projects, to advertise to build tools that it is still using setuptools.

[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"

This documentation also has additional information on how to adapt setuptools based projects to pyproject.toml.

Would be worth a try if this already solves the warning.

[dwt]

Yep, that fixes the warning. That could be a low effort first step in adapting Zope to the modern world of python building?

[dataflake]

As I mentioned before, this is a change that really needs to go into all packages, so this isn't the correct place. I will move this to the meta repository for inclusion into the project file templates.

[icemac]

Adding these three lines to a pyproject.toml should be possible here in meta/config. @dwt I'll not be unhappy if you'd come up with a PR.

[dwt]

@icemac @dataflake I'd love some review by you two.