Move to pyproject.toml and bump to Python 3.10

komish commented 10 months ago

Couple of things happening here. Most of this is coming from a side-effect of noticing that Python 3.12 doesn't play nicely with our script installation.

setup-python tasks at v2 were bumped to v4. We already used v4 in many places, but some places didn't have v4.
setup.py is to be removed (after this merges, see note below) in favor of pyproject.toml. It seems setup.py-based installations are deprecated. I've elected not to introduce alternative tooling to manage packaging, but we can look into it in the future.
The installation of our local package, which referenced setup.py, has been updated to use pip.
A few dependencies (docker, requests, pyyaml) have been updated to support Python 3.12, which threw errors given our previous pinned versions. We should scrutinize CI failures as API changes in these could cause issues.
I've removed an extraneous workflow (seems like it was for generating a token on-demand) that didn't seem to be called anywhere.

NOTE: setup.py still exists in this PR because the in-repo workflows need it, but it can be removed once this merges.

komish commented 10 months ago

A note to reviewers: To see the effects of these changes, you mostly want to look at the CI PRs themselves, where you should see the pyproject.toml get picked up over the setup.py script. E.g.

https://github.com/openshift-helm-charts/sandbox/actions/runs/6883588742/job/18724448113#step:4:192

mgoerens commented 10 months ago

It seems setup.py-based installations are deprecated

@komish Just curious, could you please expand on this a little bit ? And share if you found any resources that have made you come to this conclusion ?

komish commented 10 months ago

It seems setup.py-based installations are deprecated

@komish Just curious, could you please expand on this a little bit ? And share if you found any resources that have made you come to this conclusion ?

@mgoerens Sure thing. Here's how that particular "rabbit hole" unfolded. When using Python 3.12, I got this error trying to install our module:

 ❱ python setup.py install
Traceback (most recent call last):
  File "/path/to/openshift-helm-charts/development/scripts/setup.py", line 1, in <module>
    import setuptools
ModuleNotFoundError: No module named 'setuptools'
(venv.tools)

That struck me as odd, but just tells me that my virtual environment didn't have setuptools available (which was previously installed for us at venv creation IIRC). Fine, I could install it - but I also came across this link (I thought it was through some stack trace at the command-line, but now I can't reproduce it) - https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html.

The PyPA documentation also describes setup.py based installations as legacy: https://pip.pypa.io/en/stable/reference/build-system/

And finally, The Setuptools documentation describes how to replace it with a pyproject.toml: https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html

According to that doc, it would seem we could also move some of our setup.cfg to that file, but I'll don't see it necessary quite yet. I can definitely look to move that direction in the future.

That's how I ended up looking at what to do to replace the setup.py script. I also came across a ton of other tools in the same space (E.g. poetry), but opted to keep things close to what we had for now. At least until we have enough problems in our dev environments with it to justify the move to new tooling.

openshift-helm-charts / development

Move to pyproject.toml and bump to Python 3.10 #299