Discovering .pyx files with pyproject.toml

[MatthewRalston]

OS version

Arch 6.0.12

Python version

3.10.1

Pip version

22.3.1

Guide link

https://setuptools.pypa.io/en/latest/userguide/ext_modules.html

Problem description

Migration from setup.py to pyproject.toml has impacted my Cython extension packaging process.

I am using the guide above, which says pyproject.toml automagically discovers .pyx files. I am unwilling to Cythonize during build time and check the .c files into vcs directly. I'd prefer some help with the correct discovery of .pyx files.

Currently, python -m build produces a WHEEL file without my .pyx files. I can confirm that the sdist also does not include the .pyx files.

My pyproject.toml

[build-system]
requires = ["setuptools>=61.0", "wheel", "numpy>=1.22.0", "Cython>=0.29.21"]
build-backend = "setuptools.build_meta"

[project]
name = "kmerdb"
version = "0.7.2"
description = "Yet another kmer library for Python"
readme = "README.md"
authors = [{name="Matt Ralston <mralston.development@gmail.com>", email="mralston.development@gmail.com"}]
license = { file = "LICENSE.txt" }
dependencies = [
         'biopython>=1.79',
         'cython>=0.29.26',
         'distlib>=0.3.4',
]
requires-python = ">=3.8.0"

[tool.setuptools]
include-package-data = true
packages = ['kmerdb']

[tool.setuptools.package-data]
kmerdb = ["CITATION.txt"]

[project.scripts]
kmerdb = "kmerdb:cli"
#[options.entry_points]
#console_scripts = {kmerdb= "kmerdb:cli"}

[abravalheri]

Hi @MatthewRalston , I suspect that in order for the .pyx files to be included in the sdist (and therefore be available when building the wheel) you need to either use a plugin for the automatic inclusion (e.g. setuptools-scm) or configure MANIFEST.in manually.

Does it work when you try one of these 2 alternatives?

[abravalheri]

Hi @MatthewRalston , I suspect that in order for the .pyx files to be included in the sdist (and therefore be available when building the wheel) you need to either use a plugin for the automatic inclusion (e.g. setuptools-scm or configure MANIFEST.in manually.

Does it work when you try one of these 2 alternatives?

[abravalheri]

Just to clarify, you still need setup.py to define your ext_modules. The only thing that setuptools will automatically do is convert .pyx to .c first or (in the case the .pyx files are not present, use the pre-existing .c ones.

You can put all the declarative configuration in pyproject.toml and leave only the ext_modules part in setup.py

[MatthewRalston]

That mostly solved this problem. Now auditwheel and twine are complaining about the .whl archive validity from the build toolchain. I'll make a new issue.