Move away from setuptools

[virtuald]

Their implementation of PEP660 editable installations in setuptools 64 breaks a lot of my workflows. For now, I've pinned to setuptools < 64. Eventually that will be a bad idea (when python 3.12 gets rid of distutils?), so we should use something that isn't setuptools.

Notionally, we want to support building multiple projects from a single monorepo, so I think we'd have to move to our own custom build system anyways. Wrapping distutils was always a hack, but it's a really convenient hack!

Ideas welcome. I'm leaning towards something that wraps a real build system (like cmake or meson), but open to suggestions.

[auscompgeek]

Maybe pants? Dunno how well it'll work with the flow we need.

[virtuald]

I am definitely leaning towards meson these days. I put together a small demonstration for how we could dynamically generate source files and compile them from a single source (which is effectively what robotpy-build does).

meson.build:

project('gendemo', 'c')

keyval = import('keyval')

gen = find_program('./gen.py')
gensrc = find_program('./gensrc.py')

# determine which things need to be generated
l = configure_file(
    input: ['gen.list'],    # could be pyproject.toml
    output: 'out.list',
    command: [gen, '@INPUT0@', '@OUTPUT0@']
    # use depfile to indicate which files were read to generate the list? Might just be pyproject.toml
)

kv = keyval.load(l)
foreach k, v: kv

    # in some ways it might be useful to have a per-generated source configure_file that contains key/values to
    # be passed to the custom target (input headers, depfile, etc)... or this could be a configure_file instead of
    # a custom_target

    tgt = custom_target('gen-' + k,
        output: [k + '.c'], # could use str.split() on v?
        command: [gensrc, k, '@OUTPUT0@'],
    )

    # could use load kv to list outputs, etc?
    # - the rpy-include/{CLASS}.hpp files still present a problem here

    executable(k, tgt, install: true)
endforeach

gen.list:

a
b

gen.py:

#!/usr/bin/env python3

import sys

infile = sys.argv[1]
outfile = sys.argv[2]

with open(infile, "r") as rfp:
    with open(outfile, "w") as wfp:
        for line in rfp:
            line = line.strip()
            if line:
                wfp.write(f"{line}=1\n")

gensrc.py

#!/usr/bin/env python3

import sys

name = sys.argv[1]
outfile = sys.argv[2]

with open(outfile, "w") as fp:
    print(
    f"""
#include <stdio.h>
int main() {{
    printf("generated {name}\\n");
    return 0;
}}
""", file=fp
    )

The challenge would be making it so that this logic lived in robotpy-build, but could be included in the target project's meson.build (and maybe support some customization too?). I've used symlinks for this in the past, but that was when the build was executed by a support script. Not sure if meson-python would support such a thing, and likely would be problematic on Windows.