Support for C Extension Modules

[garryod]

It would be nice if the skeleton supported C extension modules; These differ from simple python projects in at least:

• They require a setup.py which configures the extension modules for building
• The dev container requires additional tooling (e.g. gcc)
• Wheels must be built against various OSs & architectures
• Additional tooling is required to check python stubs (e.g. flake8-pyi)

I have no clue how this should be structured, so thought it would be good to open up the discussion.

[coretl]

At the moment, I'm keen to keep skeleton as the lowest common denominator, something that works for the vast majority of pure Python modules and libraries, but that is extensible to allow things like C extensions. I don't think we have enough examples with C extensions to derive a "C extension batteries included" skeleton. I also think we don't have the effort to support something like this.

That said, I'd like to be able to add C extension support to a skeleton adopted module with the minimum amount of effort, and be able to point to some working C extension building modules in a How-To guide in skeleton-cli, so if you adopt skeleton into any C extension building modules and find that you need to change what's in the base skeleton too much rather than just adding bits, then we can change things to make it fit better.

How does that sound?

[coretl]

I'd like to see https://github.com/dls-controls/pythonSoftIOC adopt skeleton at some point. Are there some of your modules that have already adopted skeleton that have C extensions in them?

[garryod]

https://github.com/DiamondLightSource/numcertain has been developed in parallel to skeleton with reference to it, but there is still a lot that differs; I plan to bring it as close to skeleton as possible in the coming few weeks.

Keeping skeleton as simple as possible seems like a good call to me, as does inclusion of a section on building C Extensions in the How-To Guide - with reference to existing projects.

[garryod]

From in-person discussion with @coretl @gilesknap & @GDYendell:

Unanimous agreement that support for C extension modules should be left out of the python3-pip-skeleton project, but the conversion process should be documented as part of the developer guide