Closed eirrgang closed 2 years ago
Thanks for opening the discussion here! You raise a number of good issues here.
we should get away from versioneer and setup.py.
I agree mostly with this notion. Versioneer has been such a great tool for forcing semantic versioning that finding replacements that are not a dependency in of themselves has been a slow process.
As for setup.py, this has been such a main stay of Python for so long, I don't think too many people are aware that other options exist. pip install
and python setup.py
are so ubiquitous with "install python package" that we'll need to be careful about pulling people away from them.
I think the final commit (PEP 621) can wait for next year, when there is hopefully more support in setuptools and other packaging tools.
Setuptools has been the larger reason I haven't pushed for this yet. setuptools.setup
is the main workhorse of most python packages I know of for install purposes. They only started supporting pyptoject.toml
in 61.0.0, and they also say that "For the time being, pip still might require a setup.py file to support editable installs." on their site https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html. As of this writing, version 61.0.0 has only been out for 4 months. I've wanted to give people a bit more time to update their systems before pushing people towards pyproject.toml.
That said, what better way to push for it than updating the cookiecutter? Its probably a safe time to start working on it, but this will also have to come with migration docs, or at least pointers to them. We'll also want to push for helpful errors to steer people off setup.py to the pyproject.toml.
I agree mostly with this notion. Versioneer has been such a great tool for forcing semantic versioning that finding replacements that are not a dependency in of themselves has been a slow process.
The author of versioneer recommends versioningit
as a pep 517/518 alternative. I am finding it to be an effective replacement.
As for setup.py, this has been such a main stay of Python for so long, I don't think too many people are aware that other options exist.
setup.py
is no longer mentioned on https://packaging.python.org/en/latest/tutorials/packaging-projects/, so I think new Python packagers will be increasingly confused by references to it.
pip install
andpython setup.py
are so ubiquitous with "install python package" that we'll need to be careful about pulling people away from them.
I don't propose to pull away from pip
. But distutils-based installation is deprecated and slated for removal in Python 3.12. We should try to make sure this project template is up-to-date well before it has Py 3.12 users.
I think the final commit (PEP 621) can wait for next year, when there is hopefully more support in setuptools and other packaging tools.
Setuptools has been the larger reason I haven't pushed for this yet.
setuptools.setup
is the main workhorse of most python packages I know of for install purposes. They only started supportingpyptoject.toml
in 61.0.0, and they also say that "For the time being, pip still might require a setup.py file to support editable installs." on their site https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html. As of this writing, version 61.0.0 has only been out for 4 months. I've wanted to give people a bit more time to update their systems before pushing people towards pyproject.toml.
setuptools has had PEP 517 support for (edited) setup.cfg-only builds since 40.9.0 (requiring pyproject.toml). It is the migration from setup.cfg
to pyproject.toml
-only builds (and beta support for PEP 621) that was added in setuptools 61. I agree that it makes sense to wait a bit longer before embracing that change.
My main proposal here is to continue to use setuptools, but to use the PEP 517 approach. It looks to me like that puts the required setuptools version at 39.2.0 (for getting the version from an attr
). Requiring 42.0 allows use of license_files
and data_files
.
That said, what better way to push for it than updating the cookiecutter? Its probably a safe time to start working on it, but this will also have to come with migration docs, or at least pointers to them. We'll also want to push for helpful errors to steer people off setup.py to the pyproject.toml.
I am working on updated workshop material for http://education.molssi.org/python-package-best-practices/. I see now that there are several sections of the README that I would need to update in https://github.com/eirrgang/cookiecutter-cms/tree/mei-pyproject-toml.
I don't see any existing documentation on updating projects from updates to this cookiecutter. Is that the use case you have in mind regarding "migration docs"?
I'll separate the PEP 517 and PEP 621 changes into separate branches for clarity. I'll also separate the versioneer->versioningit migration, to keep things simpler and clearer.
pyproject.toml
and migrating most of setup.py
to setup.cfg
. Corresponding workshop material updates are proposed at https://github.com/MolSSI-Education/python-package-best-practices/pull/80setup.py
as it is no longer necessary. Only minor revisions to workshop material are necessary. See https://github.com/MolSSI-Education/python-package-best-practices/pull/79I am pro these changes, I think that we should aim to be adherent to python packaging best practices, which is to move away from setup.py. I understand that it may confuse people used to it, but I think if someone was to make a new package today, using setup.py
for all the metadata is the wrong choice.
Some additional discussion took place offline on Friday.
Key points
find_packages
may have evolving behavior with respect to the package/data
and package/test
directory (are we sure we want a __init__.py
in the test directory?).pip install -e .
without internet access. Extra instructions may be warranted to make sure that the PEP517 build system requirements are available locally so that pip install -e .
can function in an isolated environment.I think we've addressed with with the 3 PRs from @eirrgang and the follow up with #157. If so, does everyone feel good for closing this?
Seems good to me.
On Fri, Aug 19, 2022, 20:35 Levi Naden @.***> wrote:
I think we've addressed with with the 3 PRs from @eirrgang https://github.com/eirrgang and the follow up with #157 https://github.com/MolSSI/cookiecutter-cms/pull/157. If so, does everyone feel good for closing this?
— Reply to this email directly, view it on GitHub https://github.com/MolSSI/cookiecutter-cms/issues/149#issuecomment-1220923950, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABILPWANHMW4TVPUILB5LXTVZ7AW3ANCNFSM54I32FEQ . You are receiving this because you were mentioned.Message ID: @.***>
With distutils deprecated and slated for removal as early as Python 3.12, we should get away from versioneer and setup.py.
We should also try to be more consistent with documents like https://packaging.python.org/en/latest/tutorials/packaging-projects/ so that users and workshop participants find consistent information when they search around online. (https://packaging.python.org/en/latest/guides/distributing-packages-using-setuptools/ is a bit behind https://setuptools.pypa.io/en/latest/userguide/index.html, to which it defers, and doesn't really include any pypi-specific info not already in the above-linked tutorial.)
https://github.com/eirrgang/cookiecutter-cms/tree/mei-pyproject-toml illustrates possible updates to the cookiecutter template.
I think the final commit (PEP 621) can wait for next year, when there is hopefully more support in setuptools and other packaging tools.
The first three commits could go in as a single PR, or could be broken up. First, I establish basic usage of pyproject.toml. I remove setup.py entirely in the third commit with a migration from versioneer to versioningit.
I think my usage of versioningit is consistent with the previous usage of versioneer. There are some possible alterations, though. For instance, if we don't want _version.py to be written in the source directory, we can opt not to use the write feature, but this may complicate editable installs and/or sdist builds.