Reduce CI usage

[kurtmckee]

This PR reduces the total usage in CI (that is, the sum of time across all jobs that run).

Currently, CI usage is ~30 minutes. Without a cache hit, that number is dropped to ~15 minutes. With a cache hit, that number drops to ~10 minutes.

The CI duration (the amount of time for the entire workflow to run while you're staring at a wall clock) stays roughly the same (~5 minutes).

This is achieved by reducing the number of test runners to just one of each OS testing all Python versions simultaneously. (The setup-python action now supports setting up multiple Python versions!) This reduces the duplication of activities that are unrelated to the actual testing. For example, checking out the repo and installing pandoc.

In addition, time is saved by building a wheel once, rather than building a .tar.gz that each Python version-specific pip invocation quietly converts to a wheel in the background.

Finally, a virtual environment is created and cached, and the tox environments are cached as well. This reduces the setup time required to actually launch the tests.

Note: The cache of the tox environment may get rebuilt by tox when a Python version has a hotfix bump, like 3.10.x to 3.10.y. A possible improvement to this would be to write the Python versions to a file and include that in the call to hashFiles().

[nedbat]

Thanks, I wish I understood the tradeoffs here better. For scriv it won't matter much, because there aren't version-specific concerns, but it's easier for me to use similar patterns across projects. I definitely would like to know more about how to speed up CI runs in general. I'll have to study what you did here to understand.

[kurtmckee]

Sure thing! My hope is that the individual commits help isolate the changes so they're easier to interpret and reproduce. My goal was simply "reduce total CI usage, and try to keep the workflow duration roughly equivalent if possible".

If you'd like to hop on a conference call to discuss the approach and techniques here, just shoot me an email and we can coordinate a good time. 👍

[kurtmckee]

Hi Ned! It looks like this PR has gotten into a stuck state. If you're leaning towards rejecting it, it's fine to close.

If you're still interested in learning about what the PR does, let me know; I can be available for a conference call to discuss these changes, and the techniques it's using.

And finally, if you'd like me to break this up into some smaller chunks that are easier to digest, let me know. For example, caching virtual environments will be an easy win.

[nedbat]

@kurtmckee I'm coming back to this. I think I will mostly take what you have here, as a learning experience and to use a different pattern than my other projects. You've taught me about tox labels if nothing else! :)

[nedbat]

I rebased onto main in https://github.com/nedbat/scriv/tree/nedbat/rebase-83. One small change: the tox docs suggested using .pkg for wheel_build_env, so I switched to that. Happy to hear a reason not to though!

[kurtmckee]

Nice! I didn't realize there was a recommendation for the wheel_build_env, and have no reason to choose one name over the other. .pkg sounds good to me.

Do you want me to rebase this work on main, or are you at a point where your branch is ready to merge?

[nedbat]

I can merge my branch if you like instead of asking you to redo the rebasing. BTW: thanks for the changelog entries, but in my opinion, these sorts of internal developer-oriented details don't need a mention.

[kurtmckee]

I can rebase, and simultaneously excise the changelog fragments and migrate to .pkg. I think I can tackle that in the next day or two. 👍

[kurtmckee]

@nedbat I think this addresses the issues you identified. I'll double-check the results of the CI run in the morning.

[nedbat]

Looks good, thanks!

[kurtmckee]

I was just sitting down to review, but you beat me to it! :grinning: