`Backend 'setuptools.build_meta:__legacy__' is not available`

[wingsofovnia]

On some wheels, I am getting Backend 'setuptools.build_meta:__legacy__' is not available.

ERROR: /private/var/tmp/_bazel_user/1da975a732c2a8c8a86b734b8c4ba60b/external/rules_pycross~~lock_repos~pip/_lock/BUILD.bazel:5:8: Building thrift-0.20.0.tar.gz failed: (Exit 1): wheel_builder failed: error executing WheelBuild command (from target @@rules_pycross~~lock_repos~pip//_lock:_build_thrift@0.20.0) bazel-out/darwin_arm64-opt-exec-ST-4fe0721bc43e/bin/external/rules_pycross~/pycross/private/tools/wheel_builder --sdist ... (remaining 15 arguments skipped)

Traceback (most recent call last):
  File "/private/var/tmp/_bazel_user/1da975a732c2a8c8a86b734b8c4ba60b/sandbox/darwin-sandbox/59/execroot/_main/bazel-out/darwin_arm64-opt-exec-ST-4fe0721bc43e/bin/external/rules_pycross~/pycross/private/tools/wheel_builder.runfiles/_main/../rules_pycross~/pycross/private/tools/wheel_builder.py", line 668, in build_wheel
    wheel_file = builder.build(
  File "/private/var/tmp/_bazel_user/1da975a732c2a8c8a86b734b8c4ba60b/sandbox/darwin-sandbox/59/execroot/_main/bazel-out/darwin_arm64-opt-exec-ST-4fe0721bc43e/bin/external/rules_pycross~/pycross/private/tools/wheel_builder.runfiles/rules_pycross~~pycross~rules_pycross_internal/deps/build@1.0.3/site-packages/build/__init__.py", line 298, in build
    return self._call_backend(f'build_{distribution}', output_directory, config_settings, **kwargs)
  File "/private/var/tmp/_bazel_user/1da975a732c2a8c8a86b734b8c4ba60b/sandbox/darwin-sandbox/59/execroot/_main/bazel-out/darwin_arm64-opt-exec-ST-4fe0721bc43e/bin/external/rules_pycross~/pycross/private/tools/wheel_builder.runfiles/rules_pycross~~pycross~rules_pycross_internal/deps/build@1.0.3/site-packages/build/__init__.py", line 344, in _call_backend
    with self._handle_backend(hook_name):
  File "/private/var/tmp/_bazel_user/1da975a732c2a8c8a86b734b8c4ba60b/execroot/_main/external/rules_python~~python~python_3_10_aarch64-apple-darwin/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/private/var/tmp/_bazel_user/1da975a732c2a8c8a86b734b8c4ba60b/sandbox/darwin-sandbox/59/execroot/_main/bazel-out/darwin_arm64-opt-exec-ST-4fe0721bc43e/bin/external/rules_pycross~/pycross/private/tools/wheel_builder.runfiles/rules_pycross~~pycross~rules_pycross_internal/deps/build@1.0.3/site-packages/build/__init__.py", line 354, in _handle_backend
    raise BuildBackendException(
build._exceptions.BuildBackendException: Backend 'setuptools.build_meta:__legacy__' is not available.

ERROR Backend 'setuptools.build_meta:__legacy__' is not available.

Here is the config I am using:

pyproject.toml

[project]
name = "test"
version = "0.0.0"
authors = []
requires-python = ">=3.10,<3.13"
dependencies = [
    "flask==3.0.2",
    "opentelemetry-api==1.24.0",
    "opentelemetry-exporter-jaeger==1.21.0",
    "opentelemetry-sdk==1.24.0",
]
[tool.pdm]
distribution = false

[build-system]
requires = ["pdm-pep517>=1.0.0"]
build-backend = "pdm.pep517.api"

MODULE.bazel

bazel_dep(name = "rules_python", version = "0.32.2")
bazel_dep(name = "rules_pycross", version = "0.5.4")

python_version = "3.10"
python = use_extension("@rules_python//python/extensions:python.bzl", "python")
python.toolchain(is_default = True, python_version = python_version)
use_repo(
    python,
    "python_%s" % python_version.replace(".", "_"),
    "python_%s_host" % python_version.replace(".", "_"),
    "python_versions",
)

pycross = use_extension("@rules_pycross//pycross/extensions:pycross.bzl", "pycross")
pycross.configure_environments(
    platforms = [
        "aarch64-apple-darwin",
        "x86_64-unknown-linux-gnu",
    ],
    python_versions = [python_version],
)
pycross.configure_interpreter(
    python_defs_file = "@python_%s//:defs.bzl" % python_version.replace(".", "_"),
    python_interpreter_target = "@python_%s_host//:python" % python_version.replace(".", "_"),
)
pycross.configure_toolchains()

lock_import = use_extension("@rules_pycross//pycross/extensions:lock_import.bzl", "lock_import")
lock_import.import_pdm(
    default_alias_single_version = True,
    lock_file = "//:pdm.lock",
    project_file = "//:pyproject.toml",
    repo = "pip",
)

lock_repos = use_extension("@rules_pycross//pycross/extensions:lock_repos.bzl", "lock_repos")
use_repo(lock_repos, "pip")

I assume it uses build.pypa.io, isn't it? Interestedly enough, pip-run build -- -m build works with the same pyproject.toml:

> python --version
Python 3.10.14

> python -m build --version
build 1.0.3 (/Users/user/Downloads/pip-tools-venv/lib/python3.10/site-packages/build)

> (pip-tools-venv)  ~/Development/bazel-pycross-pdm-example  python -m build --wheel
* Creating virtualenv isolated environment...
* Installing packages in isolated environment... (pdm-pep517>=1.0.0)
* Getting build dependencies for wheel...
* Building wheel...
 - Adding app/BUILD.bazel
 - Adding app/__init__.py
 - Adding app/app.py
 - Adding test-0.0.0.dist-info/WHEEL
/private/var/folders/pj/jqnlngf95nlg3qd7l5js_wb40000gr/T/build-env-1k2p7r2x/lib/python3.10/site-packages/pdm/pep517/base.py:384: PDMWarning: 'license' is missing
  if meta.license_expression:
 - Adding test-0.0.0.dist-info/METADATA
/private/var/folders/pj/jqnlngf95nlg3qd7l5js_wb40000gr/T/build-env-1k2p7r2x/lib/python3.10/site-packages/pdm/pep517/wheel.py:244: PDMWarning: No license files are matched with glob patterns ['LICENSES/*', 'LICEN[CS]E*', 'COPYING*', 'NOTICE*', 'AUTHORS*'].
  for license_file in self.find_license_files():
Successfully built test-0.0.0-py3-none-any.whl

I also tried setting legacy build-system in pyproject.toml:

[build-system]
requires = ["setuptools>=40.8.0"]
build-backend = "setuptools.build_meta:__legacy__"

Which didn't help. Any ideas what causes it and how to workaround the issue?

[jvolkman]

The lock files don't include build dependencies unfortunately, so you will need to specify them yourself. See here for example.

[wingsofovnia]

Thank you. It did work. It there a way perhaps to globally add build deps instead of patching individual packages one by one?

[jvolkman]

Yeah, it's been a TODO of mine to add some sort of default build dependencies list. That would solve most of these cases, I think.

[wingsofovnia]

I am wondering if it would be easier to peek into pyproject.toml for build-system#requires. It is already provided to lock_import.import_pdm via project_file attr. What do you think?

Ideally, we would like to have their hashes w.r.t. to reproducible builds but neither lock_import.package has them, so similar (albeit not ideal) setup is achievable.

[jvolkman]

Unless I am misunderstanding, that would just be the build system for the local project (e.g., the monorepo if you're using one), and not the external package that you're trying to build. Each package on pypi might define its own build system. The vast majority I think still use setuptools, so defaulting to that would solve a lot of problems people run into.

[wingsofovnia]

You are right. I have little experience with Python but things clarified after reading up.

This can be closed unless you want to keep it open until a default build dependencies list or similar is implemented.

[ewianda]

@jvolkman, if you give me some pointers, I can work on this. I have a basic understanding how the annotation system works.

[jvolkman]

I think we just want a default_build_dependencies list somewhere. If build_dependencies are specified at a package level, they'd override the default list (no merging or anything fancy).