9.0.2 requires FFMpeg headers to clean source

[jonassmedegaard]

Since the release of v9.0.2, doing a python setup.py clean resolves FFMpeg headers and fails if unable to locate them. This project should never cleanup that other project, so seemingly has no need for those headers during its own cleanup.

[jlaine]

Hi @jonassmedegaard as you probably noticed, the setup.py has been trimmed way down as it was horrendously complex to maintain. The result is that regardless of the operation, the FFmpeg header detection code is run. At what step of the Debian packaging are you running "python setup.py clean"?

Do you maybe have another Python package I can draw inspiration from?

[jonassmedegaard]

Problem seems worse: Even when header detection succeeds, the cleanup process leaves behind generated *.c files.

I run cleanup process twice: Initially outside of the clean build chroot to ensure that only clean source is included with the build, and then again inside the chroot as part of standard Debian packaging routines that first cleans then builds then installs.

Sorry, I am not strong on Python, so not sure which packages to point at. I guess you'd want something using cython - I don't know which are exemplary structured, but here is the full list of packages in Debian build-depending on cython3: abydosabydos adios aplpy astropy astropy-regions astroscrappy asyncpg atropos azure-uamqp-python basemap bcolz binoculars bitshuffle blueman borgbackup breezy brian brltty buildbot camitk capstone ceph cftime chemps2 compizconfig-python compreffor compyle conda-package-handling cyarray cypari2 cysignals cyvcf2 dioptas dipy dune-grid emperor epigrass facet-analyser fast5 fenics-basix fiona fonttools fpylll freeart freesas frozenlist gammapy gdcm gensim gnome-keysign gpyfft grpc gudhi guiqwt gwcs h5py harvest-tools healpy htseq imexam indexed-gzip invesalius jack-mixer joblib khmer kivy labgrid libedlib libgpuarray libimobiledevice libplist libseccomp logbook lxml macs magic-wormhole magic-wormhole-mailbox-server mayavi2 mdanalysis mdtraj memory-allocator meson mia minimap2 mlpack mlpy montage mpi4py mpi4py-fft music netcdf4-python nifti2dicom nipy nipype numcodecs numpy numpy-stl obitools opendrop openems openmm opentsne pairtools pandas paq paraview parsnp peewee petsc4py photutils plyvel pmix pplpy primecountpy printrun py-stringmatching py-ubjson pybik pybj pycangjie pyclipper pycorrfit pyemd pyepr pyfai pyfftw pyfuse3 pygame pygame-sdl2 pygccjit pygrib pykdtree pyliblo pymca pymssql pynfft pyode pyranges pyregion pyresample pyrle pyscanfcs pysph pystemd pystemmer pysurfer pytables pytaglib python-acora python-admesh python-aiohttp python-alignlib python-autobahn python-av python-biom-format python-bx python-cartopy python-cassandra-driver python-cobra python-cogent python-cutadapt python-cwcwidth python-cymem python-cython-blis python-cytoolz python-daphne python-datrie python-django-channels python-djvulibre python-dnaio python-enet python-esmre python-ete3 python-fabio python-falcon python-fastbencode python-feather-format python-fisx python-geopandas python-geotiepoints python-gevent python-gsd python-gssapi python-hdmedians python-hidapi python-httptools python-hug python-intbitset python-libdiscid python-libzim python-line-profiler python-llfuse python-ltfatpy python-lupa python-memprof python-mne python-msgpack python-multidict python-murmurhash python-ncls python-pcl python-pomegranate python-pot python-preshed python-pybedtools python-pyproj python-pysam python-pyspike python-pyvcf python-questplus python-rencode python-rocksdb python-rtmidi python-sfml python-shapely python-skbio python-sqt python-stack-data python-suitesparse-graphblas python-tesserocr python-thinc python-thriftpy python-tinyalign python-tinycss python-vispy python-wheezy.template python-wordcloud python-wsaccel pywavelets pyxdameraulevenshtein pyyaml pyzmq pyzoltan q2-alignment q2-cutadapt q2-demux q2-diversity-lib q2-emperor q2-feature-classifier q2-feature-table q2-fragment-insertion q2-metadata q2-quality-control q2-quality-filter q2-taxa q2-types qiskit-aer qiskit-terra qutip rasterio rdma-core renpy reproject ruamel.yaml ruamel.yaml.clib rust-cbindgen s3ql sagemath sagetex scikit-build scikit-learn scikit-misc scipy setools sfepy siconos silx skimage slepc4py slixmpp sorted-nearest specutils sphinx-automodapi statsmodels termbox theano tiddit tiledb-py tombo umis unifrac uvloop vedo vtk-dicom vtk7 xmms2 xpra xrstools yarl yt zaqar adios aplpy astropy astropy-regions astroscrappy asyncpg atropos azure-uamqp-python basemap bcolz binoculars bitshuffle blueman borgbackup breezy brian brltty buildbot camitk capstone ceph cftime chemps2 compizconfig-python compreffor compyle conda-package-handling cyarray cypari2 cysignals cyvcf2 dioptas dipy dune-grid emperor epigrass facet-analyser fast5 fenics-basix fiona fonttools fpylll freeart freesas frozenlist gammapy gdcm gensim gnome-keysign gpyfft grpc gudhi guiqwt gwcs h5py harvest-tools healpy htseq imexam indexed-gzip invesalius jack-mixer joblib khmer kivy labgrid libedlib libgpuarray libimobiledevice libplist libseccomp logbook lxml macs magic-wormhole magic-wormhole-mailbox-server mayavi2 mdanalysis mdtraj memory-allocator meson mia minimap2 mlpack mlpy montage mpi4py mpi4py-fft music netcdf4-python nifti2dicom nipy nipype numcodecs numpy numpy-stl obitools opendrop openems openmm opentsne pairtools pandas paq paraview parsnp peewee petsc4py photutils plyvel pmix pplpy primecountpy printrun py-stringmatching py-ubjson pybik pybj pycangjie pyclipper pycorrfit pyemd pyepr pyfai pyfftw pyfuse3 pygame pygame-sdl2 pygccjit pygrib pykdtree pyliblo pymca pymssql pynfft pyode pyranges pyregion pyresample pyrle pyscanfcs pysph pystemd pystemmer pysurfer pytables pytaglib python-acora python-admesh python-aiohttp python-alignlib python-autobahn python-av python-biom-format python-bx python-cartopy python-cassandra-driver python-cobra python-cogent python-cutadapt python-cwcwidth python-cymem python-cython-blis python-cytoolz python-daphne python-datrie python-django-channels python-djvulibre python-dnaio python-enet python-esmre python-ete3 python-fabio python-falcon python-fastbencode python-feather-format python-fisx python-geopandas python-geotiepoints python-gevent python-gsd python-gssapi python-hdmedians python-hidapi python-httptools python-hug python-intbitset python-libdiscid python-libzim python-line-profiler python-llfuse python-ltfatpy python-lupa python-memprof python-mne python-msgpack python-multidict python-murmurhash python-ncls python-pcl python-pomegranate python-pot python-preshed python-pybedtools python-pyproj python-pysam python-pyspike python-pyvcf python-questplus python-rencode python-rocksdb python-rtmidi python-sfml python-shapely python-skbio python-sqt python-stack-data python-suitesparse-graphblas python-tesserocr python-thinc python-thriftpy python-tinyalign python-tinycss python-vispy python-wheezy.template python-wordcloud python-wsaccel pywavelets pyxdameraulevenshtein pyyaml pyzmq pyzoltan q2-alignment q2-cutadapt q2-demux q2-diversity-lib q2-emperor q2-feature-classifier q2-feature-table q2-fragment-insertion q2-metadata q2-quality-control q2-quality-filter q2-taxa q2-types qiskit-aer qiskit-terra qutip rasterio rdma-core renpy reproject ruamel.yaml ruamel.yaml.clib rust-cbindgen s3ql sagemath sagetex scikit-build scikit-learn scikit-misc scipy setools sfepy siconos silx skimage slepc4py slixmpp sorted-nearest specutils sphinx-automodapi statsmodels termbox theano tiddit tiledb-py tombo umis unifrac uvloop vedo vtk-dicom vtk7 xmms2 xpra xrstools yarl yt zaqar

[jonassmedegaard]

(sorry for the long post - I blindly assumed CommonMark-compliant Markdown here, i.e. that the list would be wrapped)

[jonassmedegaard]

I succeeded in building, with these two workarounds applied:

1. ignore failure for your cleanup process (i.e. append || true to the standard command
2. manually delete top directory src/ afterwards

So this issue is no longer urgent for me, and if you want a quickfix then that'd be to include my step 2 above with your code.

[jonassmedegaard]

maybe meshio does something good. I really don't know, just looked a bit for packages in Debian including a pyproject.toml file and not being architecture-independent...

[jonassmedegaard]

Another potential inspiration might be aiohttp that both use modern pyproject.toml file and cython3.