Ensure clean sdist and wheel builds

I've finally found some time to have a look at building a clean PyPI sdist and corresponding wheels for lgpio and rgpio. There's quite a bit to unpack here, and a couple of potentially controversial decisions so I'll go through things one by one:

Python 2.x dropped

I've got no means of "cleanly" building anything for Python 2.x (I'll come onto what a "clean" build means specifically in this context a bit later on), even in container environments at this point so the first commit here (5e06193d811bc9c3c741d33a7ccaaa2b614dd663) just drops all the Python 2.x stuff. If that's still desired I can revert that change and make a best guess at what a Python 2.x compatible build would look like in setup.py but it will be just that: a guess. Someone else would have to verify it works the way it's intended to.

src symlink

One of the issues with the current packaging is that the C source is "above" the Python source in the hierarchy. Various bits of the Python packaging mechanisms expect all source to be "below" the setup.py. Thankfully this is fairly easy to fake with a "src" symlink pointing to "..", which is what the second commit (8c66cad5c31b5ce0fe0e881f6cdffcbbc488204b) introduces. This is only present in the PY_LGPIO dir as PY_RGPIO doesn't require it (no C source to include).

PYTHON var in Makefile

PR #5 also intends to introduce this, but does so in a rather more invasive way (and also doesn't solve the issue of "clean" builds or static linking). The third commit here (19d287a1fa46daa2e0b527ebcd3e4ef43bb0a8a8) simply replaces the remaining python3 references in the Makefile with a PYTHON variable.

static builds

I don't want to alter the current default build setup, producing a shared object and a dynamically linked python library, as it works extremely well for the Debian/Ubuntu packaging (which I'm also responsible for). However, I do take the point that wheels uploaded to PyPI should almost certainly include a statically linked lgpio library as pip cannot ask the wider system to install things like shared libraries.

To that end, the fourth commit (46304f60618ca22f96ed4601bca6d49aa21ce2ff) extends the lgpio setup.py to accept an environment variable hint (PYPI=1) which will cause it to build a statically linked binary. This commit also introduces a generated MANIFEST.in which includes everything that should be included in the source distribution.

The upshot is that the lgpio and rgpio modules can now be built with the pypa/build tool. This tool first builds a source distribution (tar.gz) in a clean venv, then creates a separate venv, and builds the binary wheel from the built source distribution in that clean venv. This ensures all build dependencies are correct, and also that the source distribution definitely includes everything necessary for the binary build.

Testing

The normal build procedure (make; sudo make install) should work exactly the same as it always has, producing dynamically linked artifacts.

If you wish to test building statically linked artifacts with pypa/build, please be aware that the version in Ubuntu 22.04 currently has issues (I've uploaded an SRU to fix this but that'll take a while for approval). In the meantime, the following procedure should work on Ubuntu 22.04 (using python3-virtualenv as a workaround for the aforementioned issue):

$ sudo apt update
$ sudo apt install python3-dev python3-setuptools python3-build python3-virtualenv build-essential swig
$ git clone https://github.com/waveform80/lg
$ cd lg
$ git checkout pypi
$ cd PY_LGPIO
$ PYPI=1 python3 -m build .
$ ls dist/

Assuming this passes scrutiny, and there are no strong objections, I'll be using a similar procedure to the above (with the aid of the deadsnakes PPA and some armhf/arm64 containers) to generate statically linked wheels for Python 3.9 through Python 3.12 for upload to PyPI. This should hopefully resolve the current pain around installing lgpio in a virtualenv (i.e. #24) across the vast majority of installs.

If there's demand for earlier Python versions I should be able to produce wheels back to Python 3.7 but earlier than that will be quite tricky.

joan2937 / lg