jenstroeger / python-package-template

An opinionated Python package/application template repository, with SLSA and SBOM support built in, enabled for security scanners, code linters, typing, testing and code coverage monitoring, and release automation for reproducible builds.
MIT License
34 stars 11 forks source link

fix(ci): also update setuptools when setting up the virtual environment #793

Closed jenstroeger closed 1 week ago

jenstroeger commented 1 week ago

So, here’s the thing. Python 3.10.14 and 3.11.9 create a venv like so[^1]

~ > python3.10 -m venv test
~ > . test/bin/activate
(test) ~ > pip list
Package    Version
---------- -------
pip        23.0.1
setuptools 65.5.0

The problem is that setuptools v65.5.0 and below are subject to https://github.com/advisories/GHSA-r9hx-vwmv-q579, and should be — like pip — updated to the latest version when we set up the venv. If not, there’s a good chance that make audit will trigger if no other package happens to update the setuptools package…

In this package we’ve been lucky because of an indirect dependency (e.g. here) and our eager update strategy:

Collecting setuptools>=30.3.0 (from pytest-doctestplus==1.2.1->package==2.13.3)
  Downloading setuptools-72.1.0-py3-none-any.whl.metadata (6.6 kB)

Other packages/repos derived from this one, however, have their own dependencies and did not update setuptools (anymore) and, thus, the package audit triggered and failed CI.

[^1]: Python 3.12 does not install setuptools by default.

jenstroeger commented 1 week ago

Addendum: the last missing piece of the puzzle is this line in the venv goal: https://github.com/jenstroeger/python-package-template/blob/e851a5864f2431da4e3e53f32044f1cb6db09c02/Makefile#L74 Note the --upgrade-deps cmd line argument (docs) which updates both pip and setuptools for the newly created venv.

However, in CI we do not create a venv and instead run make setup directly and, thus, setuptools have not yet been updated beyond Python’s default.

To verify:

~ > python3.10 -m venv --upgrade-deps test
~ > . test/bin/activate
(test) ~ > pip list
Package    Version
---------- -------
pip        24.2
setuptools 74.1.1

This is why the CI issue does not reproduce locally… 🤦🏻‍♂️