Migration to tox

[webknjaz]

I'd like to do $sbj to the extent possible. It's hard to complete such a change in long-existing projects for a number of reasons, since it's a bit complicated when it comes to C-extensions. But I think the ecosystem is in a better place today than it was when I last considered it (like tox devenv, for example). I've also gained some experience with tox+Cython elsewhere, which makes me think that using tox (a Python-aware workflow tool) is generally better than GNU/make (which is generic and is just a dumb command runner mostly).

We can keep tox invocations wrapped with make if someone wants that, but in general, I consider make impractical in the Python world. It originally exists in aiohttp-adjacent projects because of CPython Core Devs' prior habit.

@bdraco @Dreamsorcerer I'll be counting on your DX feedback in related PRs, specifically around working with C-extensions. But the scafolding I'm hoping to mostly source from my other projects, as it's pretty well-composed already. The only thing I didn't really hash out is the build options (python vs. compiled extension). So it'd be the only thing that will need extra thought.

[bdraco]

tox always seemed like another layer of complexity to debug when there is a large dep chain. I don’t think we have that problem here.

another concern is it tends to slow things down. I wonder if https://github.com/tox-dev/tox-uv if ready for prime time

would like to be mindful of the inverse relationship between CI run time/test invocation complexity and contributor participation

[webknjaz]

I've been using it for like a decade and sometimes contributed to it. Makefile also adds a layer of complexity, and virtualenvs do too for that matter.

tox-uv is probably good, but I haven't had time to play with uv. Though, since Hynek says it's good, I think there's no reason to be conservative about it. FWIW, tox caches the venvs so it only influences the first run and anything that invalidates the cache. And tox 4 was heavily refactored by Bernát and improved a number of things.

Tox's main idea is to actually keep the CI and dev env experience in sync as much as possible. Still, with the tox devenv feature, it'll give you an old-good virtualenv with everything pre-installed, for experiments. So you shouldn't lose anything in experience.

Additionally, I have an interesting approach with heavily-pinned DIY pip-tools-managed lockfiles that ties well into the infra.

[webknjaz]

I'm adding this to the sprint with the understanding that this is difficult and should be worked on in small portions. Somebody will need to pair/consult with me to understand the exact vision I have in mind.

[webknjaz]

Linking some of my points from the past: https://github.com/aio-libs/aiohttp/pull/3419#issuecomment-444662312