replace all uses of distutils

[1fish2]

Python 3.10 deprecates distutils. It'll be removed in Python 3.12.

So with the move to support Python 3.10 in #135 , we should replace distutils in all vivarium projects. setuptools embeds a forked (and updated?) copy of distutils but there are compatibility issues and it's moving away from being a CLI tool to being just a library. setup.py is not dead but apparently we should not run it directly. It's a mess.

(Presumably wcEcoli will stay on Python 3.8.7 and distutils. I did try updating it to setuptools but reverted that due to https://github.com/CovertLab/wcEcoli/issues/1113.)

PEP 517 defines how to describe a project in a small, declarative pyproject.toml file.

Modern tools to consider:

• Flit. Experts recommend Flit and it should be easier and more robust to use and maintain. Flit handles the normal cases but Flit does not support Cython code so it won't work for vivarium-ecoli. It makes sense for us to learn one modern tool and use it in all vivarium projects.
• build.
• Pipenv (there's also a Pipenv guide on Real Python)
• Poetry
• Bento
• Meson is a build tool like pyinvoke and Ruby rake, not a packaging tool, but apparently Numpy & Scipy are moving to [Meson]. Maybe it includes Python packaging features? Anyway, don't switch to numpy.distutils.
• ...

Some of these tools can also obviate the Makefile.

[eli-schwartz]

All the tools on your list are distutils/setuptools alternatives, except for:

• build which is not actually a build system -- it is a CLI for PEP 517, and thus can replace running setup.py directly.
• pipenv which is a program for managing virtualenvs and has nothing to do with build systems, not even the partial connection build has

flit's design goal is to only support the simple cases of pure python and it philosophically rejects all forms of code building, including e.g. embedding version numbers into module/_version.py.

poetry doesn't officially support compiled extensions. It has an undocumented feature to do so, but it involves importing setuptools and using setuptools Extension objects, which then get injected back into poetry. Using it means that poetry builds wheels by generating a setup.py file and running python setup.py build which... you are not supposed to do. :D

bento is dead since 2014, and got mentioned in the SciPy blog post (they used to have bento as an alternative build system): https://labs.quansight.org/blog/2021/07/moving-scipy-to-meson/

• Meson is a build tool like pyinvoke and Ruby rake, not a packaging tool

In case it wasn't already obvious, I am biased in favor of Meson. :D

Speaking of "a packaging tool"... As a build tool, it is less like pyinvoke and more like cmake, autotools, etc. And its core features include installing the results (to the python site-packages directory). For creating wheels specifically, you can use e.g. https://thiblahute.gitlab.io/mesonpep517/ to add wheel metadata on top of Meson, and make it pip-installable. In the long term, Meson does intend to support a builtin PEP 517 packaging backend, but we are not there yet.