Try replacing `setup.*` with `pyproject.toml`

[Mr0grog]

Direct usage of setup.py (that is calling python setup.py some_command to do builds and such) has been deprecated and no longer supported for quote a while, and I’ve been meaning to try to move everything to pyproject.toml (the new format for declaring package metadata). It seems like pyproject.toml supports all the nice things now, so this is my experimental attempt to switch to it and drop setup.py and setup.cfg entirely.

(This seems like a good time to do this, since there are some other semi-breaking changes on tap, like rethinking how rate limit errors are handled.)

This doesn’t quite work yet — the main sticking point is Versioneer. When pip or build try and make an isolated environment, it’s missing the runtime requirements for the package and can’t import the version from the code.

This might be fixable by:

• Making a separate _build_version.py file that pulls in the version info the same as __init__.py does but without requiring any other dependencies.
• Bringing back a minimal version of setup.py just to run Versioneer. (But it seems like this shouldn’t be necessary since Versioneer 0.29 claims support for pyproject.toml-only build systems: https://github.com/python-versioneer/python-versioneer/releases/tag/0.29)
• Not doing isolated builds? Same as today, I guess, but disappointing.

Need to experiment here.

Obviously this will need CI changes and maybe Dependabot config, too.

Also Needs testing to make sure builds based on this are still installable and workable on Python 3.6 (assuming a recent version of pip is being used to install).

[danielballan]

I was just thinking of dipping a toe back in here, starting with this. May I toss some commits here on the versioneer aspect?

[Mr0grog]

Dan!! Absolutely, please do add some commits

[Mr0grog]

I was just reading through the PRs that went into v0.29 of Versioneer and https://github.com/python-versioneer/python-versioneer/pull/371 makes it sound like maybe pyproject.toml-only is only supported for the pdm backend, and not setuptools, but 🤷

[danielballan]

Hello! It's been too long!

If I recall correctly, we based this repo on a template and set of recommendations that I used to work from for most of my projects. In May, I got together with Henry Schreiner for week, and we produced updated recommendations. We cover four build backends, including setuptools, but I've been moving my projects over to hatchling and the hatch-vcs extension in place of versioneer.

Its builds are isolated. I've been happy with the git integration. In addition to versioning, it uses .gitignore as a base determination of what files should be including in the package, supplanting MANFIEST.in.

If that sounds reasonable I'll push some commits to try that on here.

[Mr0grog]

@danielballan that sounds pretty reasonable to me! (I think last time I was playing with newer Python tools, I liked PDM more than Hatch, but that was mainly because I didn’t think Hatch managed venvs, and it looks like that’s either changed or I understood it wrong back then. I also liked that it did PEP 582, but that got killed and I am sad.)

One question about Python versions, though:

We currently support Python 3.6, and I think Hatch/Hatchling are 3.7+ (I guess the latest Hatchling looks like just 3.8+). I think the build-system deps are only needed during builds, so this isn’t a big deal (since I have been building from a newer version), but will this be a problem if a Python 3.6 user winds up installing from an sdist instead of a wheel?

IIRC we support 3.6 here because Google Colab took a long time to update, but I just checked and it looks like Colab is on Python 3.10 now. So it’s probably OK to change our minimum Python version. Just want to flag it as a breaking change to avoid if it’s not too hard.

[danielballan]

Good note. Python 3.6 reached end-of-life (no more security patches) in 2021. Python 3.7 reached end-of-life in June 2023. I've been in the habit of dropping Python versions from the test matrix once they are end-of-lifed. Naturally, users stuck on all versions can resort to using older releases of wayback.

Would that policy create a problem for known users?

[Mr0grog]

Would that policy create a problem for known users?

Nah, I think that’s ok here. The only holdup for known users (mainly @edsu and @ericnost) was Google Colab, and that’s no longer an issue.

[danielballan]

In the ~5 years it's been since I showed my face here, my commit bit went away (which is very reasonable). I started a PR into this branch over on my fork: https://github.com/edgi-govdata-archiving/wayback/pull/145

[Mr0grog]

Do you want it back? I’m happy to add you as a committer/maintainer/admin (depending on how responsible you want to be 😉).

[danielballan]

Thanks. Let’s give it a couple months and see if I manage to stay active. 🙂

[Mr0grog]

Yowza, this dropped 3,025 lines of code! Granted that was mostly Versioneer, but nice to see it go. 👋