Open chadrik opened 6 months ago
Thank you for working on this! It's interesting to see how this feature works out for our CI.
In this particular case, I'm not sure how the benefit of single sourcing the build matrix stacks up against the costs in complexity and runtime overhead. I'm a bit worried about having all jobs block on that initial job computing the matrix, when they could otherwise start running in parallel right away.
Would it make sense to pre-compute the build matrix in pre-commit?
I'm a bit worried about having all jobs block on that initial job computing the matrix, when they could otherwise start running in parallel right away.
I understand the concern, but to be fair the job completes in 7s, so it's not a major delay.
Would it make sense to pre-compute the build matrix in pre-commit?
The easiest thing to do would be to add a pre-commit task that writes nox -l --json
to a file that gets committed in the repo. It could obviously be setup to only run if noxfile.py
changes. Would you prefer that approach?
Hi, thanks for this awesome project!
This PR establishes a single source of truth for python versions and tests matrix, driven by nox.
I had this idea for my own projects as I've often been frustrated with duplicated data between pre-commit, nox, and gitlab. I wanted to implement this in this project for a few reasons:
nox
docs about this.Changes
These are the differences in behavior from before:
if: matrix.python-version != '3.7'
withinci.yml
because that is already in thenoxfile.py
asif (python, tox_version) != ("3.7", "latest")
, and the goal of this PR is to create a single source of truth. In order to do this, I had to change the dimensions of thestrategy:matrix
from[os, python-version]
to[os, python-version, tox-version]
. This may actually improve run time as more jobs run in parallelnox -s 'tests(tox_version=latest)'
will only runtest_tox_to_nox.py
by default. This was done to make the local test and CI behave the same, which reduces developer confusion as well as special-case code inci.yml
. To override, just pass-- tests
Other thoughts
The most difficult part of this project was discovering the right incantations for
jq
to mutate the data fromnox -l --json
(though it is fairly readable once it's done). If this idea catches on, I foresee a need for a CLI tool (either part of nox or separate) which can reduce the complexity and boilerplate by preparing nox session data in ways that can be more easily consumed by Gitlab, Github, pre-commit, etc.