Migrate away from Poetry

[gotmax23]

I'd like to switch from Poetry to another build system such as setuptools or hatchling for the antsibull* projects. Poetry has caused us a lot of problems. Poetry's dependency resolution is very slow and causes problems with our multiple supported versions of antsibull-core and multiple Python versions. It works differently than pip and may install very old dependency versions that don't work with newer pythons, as it only considers dependencies that are compatible with the minimum python version. Its dev dependency specifications are not understood by other tools.

Hatchling supports PEP 621 metadata which is interoperable with other tools, and dev dependencies can easily be specified as extras. It has a pluggable architecture.

Only pretty new setuptools versions support PEP 621 metadata an, so I prefer to use the "old" setup.cfg configuration.

I'm fine with switching to either hatchling or setuptools. The other popular build backend, flit, doesn't properly include all licenses texts in wheels, so we'd be violating our own license terms by uploading wheels to PyPI :(.

I propose an incremental approach for each antsibull project:

1. Copy the dev depenments.txt (like in https://github.com/ansible-community/antsibull/pull/483) so they can be used without Poetry.
2. Use a test runner such as nox (similar to tox, but its configuration is a Python file and it's easier to use outside venvs) and adjust the project's CI to use that instead of {lint,test}-*.sh and poetry install. This would use the dev dependencies specified in requirements.txt. This can be done at the same time as #1. It would solve the problems we've been in having in CI with Poetry.
3. Drop {lint,test}-*.sh, and adopt hatchling or setuptools at an appropriate place in the release cycle. Adjust any dev documentation that references poetry. This step would remove the dev dependency duplication between pyproject.toml and requirements.txt. I'd prefer not to push this to stable antsibull-docs v1 or antsibull-core v1. We can get it in to v2 and the 0.x.x antsibull and antsibull-changelog, though.

[felixfontein]

@gotmax23 I've been looking at hatch a lot this weekend. One thing I'm confused about is editable installs support (https://github.com/pypa/hatch/issues/588), which seems to not yet exist - but that's something I'm using a lot during development. Poetry allows to do this by adding something like

# For development, we install dependent projects under our control in dev mode:
antsibull-core = { path = "../antsibull-core/", develop = true }

to the dev dependencies. (Having an explicit path in there isn't perfect, but at least it works.)

There are apparently plans to kind of support this (https://github.com/pypa/hatch/pull/589#issuecomment-1313127955), but a) since there is no clear definition on how it will look like, and b) more importantly it isn't anywhere close to being there (AFAICT), it's nothing that helps us now.

I guess just using hatchling (without hatch itself) should work though, but then I'm wondering why not just use setuptools...

[gotmax23]

One thing I'm confused about is editable installs support

I'm not sure about hatch's environment management (I prefer to use nox to run test/linter tools and manually manage venvs), but it's always possible to use pip install -e to install packages in editable mode.

but then I'm wondering why not just use setuptools...

hatch has PEP 621 support, a nice CLI interface, and a pluggable design. setuptools has implemented PEP 621 support, but it's only available in newer versions that don't support Python 3.6 and aren't available on some Linux distributions (this is relevant for distribution packagers and not pip install which will install the latest version). hatch is a newer, leaner project, and I've wanted to try it out :). It seems to be gaining some traction in the Python community.

https://github.com/ansible-community/antsibull-docs-parser/pull/2 (WIP) configures that project to use hatchling and nox. We can replicate this for other antsibull projects if you'd like.

[felixfontein]

:tada: