Travis / macos / pip is complains about fabio-0.13.0.tar.gz

[jonwright]

This just popped up in my CI for ImageD11. Is it because I have the --no-use-pep517 or is there something more sinister going on? Seems to be a new feature of pip ...

$ python3 -m pip install --upgrade pip setuptools
$ python3 -m pip install --no-use-pep517 --editable .
DEPRECATION: Configuring installation scheme with distutils config files is deprecated and will no longer work in the near future. If you are using a Homebrew or Linuxbrew Python, please see discussion at https://github.com/Homebrew/homebrew-core/issues/76621
Obtaining file:///Users/travis/build/jonwright/ImageD11
  Preparing metadata (setup.py) ... -\/done
Requirement already satisfied: six in /usr/local/Cellar/protobuf/3.11.4/libexec/lib/python3.7/site-packages (from ImageD11==1.9.8) (1.14.0)
Requirement already satisfied: numpy in /usr/local/lib/python3.7/site-packages (from ImageD11==1.9.8) (1.18.1)
Requirement already satisfied: setuptools in /usr/local/lib/python3.7/site-packages (from ImageD11==1.9.8) (62.1.0)
Collecting fabio
  Downloading fabio-0.13.0.tar.gz (1.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.5/1.5 MB 20.5 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... -error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [16 lines of output]
      INFO:fabio.setup:Use setuptools, cython is missing
      INFO:fabio.setup:Use setuptools.setup
      error: Multiple top-level packages discovered in a flat-layout: ['fabio', 'package'].

      To avoid accidental inclusion of unwanted files or directories,
      setuptools will not proceed with this build.

      If you are trying to create a single distribution with multiple packages
      on purpose, you should not rely on automatic discovery.
      Instead, consider the following options:

      1. set up custom discovery (`find` directive with `include` or `exclude`)
      2. use a `src-layout`
      3. explicitly set `py_modules` or `packages` with a list of names

      To find more information, look for "package discovery" on setuptools docs.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
The command "python3 -m pip install --no-use-pep517 --editable ." failed and exited with 1 during .
Your build has been stopped.

[kif]

Hi Jon, This is a known issue ... python 3.12 won't ship distutils any more, in couple of years from now.

Since we depend on numpy.distutils which depends on distutils, we are currently waiting until numpy and scipy find a solution for building extensions in a modular way. Scipy is considering to move to Meson/Ninja, for numpy it is less clear.

The only alternative I see is removing the modularized build from extensions and recreate a monolitic setup.py with thousands of LOC, which looks to me not only as a regression but also a burden to maintains.

If I remember well, limiting the version of setuptools <60 mitigates the problem for now (i.e. not future proof).

[kif]

I am surprised wheels were not used.

To reproduce the bug:

python3 -m venv testfabio
source testfabio/bin/activate
pip install --upgrade pip setuptools
pip install --no-use-pep517 fabio --no-binary :all:

if setuptools is limited to version <60, it goes one step further and crashes with cython not (yet) installed.

Since the cython build dependency is described in the pyproject.toml (pep518) I suspect that the --no-use-pep517 disables the read of this file. Why is this option needed ?

So I see two options for your bug:

1. use pep517/518 (and adapt your project accordingly)
2. manage the build-time requirements of your dependencies manually, i.e. pip install cython in your CI configuration file.

I'll upgrade the requirements.txt to ensure version<60 of setuptools is used.

[jonwright]

Thanks! The pep517 business was to allow editable installs, waiting for 660 I think. Maybe better to drop those for now. The CI is using xcode 11.2 which says python 3.7, so maybe upgrading that would get a wheel instead.