Closed ryneeverett closed 2 years ago
I saw the tox.ini in the repo and ran tox to run the tests. First, it only uses old versions of Py3 which I don't have, so I hacked in a py39-jira200. Having added that, loading the tests fails with #829, meaning no tests run.
Looking at the github actions, it seems like tox isn't used in CI, and CI doesn't even run python setup.py test like tox does. So maybe a place to start is bringing those two things in line?
I don't mind including support for tox if people find it helpful but I don't see much value in it and if it doesn't work I'd be inclined to remove it. I don't see any reason to test the full matrix locally. I'd rather let CI notify me of the occasions that a version-specific test failure occurs. I also don't believe it's advantageous to run tox in CI because we'd lose parallelism and the ability to easily view the matrix of checkmarks in CI.
I'd also note that I intentionally moved away from python setup.py test
in part because it's deprecated.
Also, I noticed that CI is only testing on aarch64 and ppc4le. For a mostly-Python tool, that seems a bit esoteric, and especially so since it does not include the common x86 architecture. Maybe CI would run more quickly if the full matrix ran on x86 and only the latest python version / jira version ran on the other architectures?
Whoops. We should definitely be testing x86 too. I'm not necessarily opposed to a smaller matrix but I'm not sure CI time is really a problem. For the purposes of push-and-pray debugging I'm not sure it makes a difference because you're going to get feedback that CI failed as soon as one job fails.
I suspect it takes longer to start jobs on platforms for which there is less availability? At any rate, if the project is still well within its actions budget then that's good news.
I assume that github automatically starts jobs on platforms with higher availability first but I haven't confirmed it.
My interpretation of the new bait-and-switch actions budget is that only Ralph can see it since we're under his personal namespace, so I have no idea. If it becomes an issue I'd be inclined to consider switching if there's a CI service with a better free tier.
Originally posted by @djmitche in https://github.com/ralphbean/bugwarrior/issues/831#issuecomment-894682505