search

python-poetry / poetry

Python packaging and dependency management made easy
https://python-poetry.org
MIT License
30.29k stars 2.23k forks source link

pip works, but poetry not #9518

Closed n-bes closed 11 hours ago

n-bes commented 4 days ago

Description

Minimal reproducible example

Dockerfile:

FROM debian:12
ENV DEBIAN_FRONTEND="noninteractive" \
    POETRY_VERSION="1.8.3" \
    POETRY_HOME="/opt/poetry"
RUN apt-get update -y && \
    apt-get install -y \
        gcc \
        curl \
        python3-venv \
        python3-pip \
        python3-dev
ENV PATH=/root/.cargo/bin:/opt/poetry/bin:$PATH
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- --default-toolchain stable -y
RUN curl -sSL https://install.python-poetry.org | python3 - \
    && poetry config virtualenvs.create false
COPY pyproject.toml .
RUN python3 -m venv /venv && . /venv/bin/activate && \
    poetry lock
COPY poetry.toml .

poetry.toml:

[installer]
no-binary = [
    "annotated-types",
    "pydantic",
    "pydantic-core",
    "typing-extensions",
]

pyproject.toml:

[tool.poetry]
name = "hello-pydantic"
version = "0.1.0"
description = ""
authors = ["Your Name <you@example.com>"]

[tool.poetry.dependencies]
python = ">=3.10,<3.13"
pydantic="2.5.2"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

Steps with pip

$ docker build .
$ docker run --rm -it <image_name>
$ source venv/bin/activate
$ pip3 install --no-binary ":all:" pydantic=="2.5.2"
....
Successfully installed annotated-types-0.7.0 pydantic-2.5.2 pydantic-core-2.14.5 typing-extensions-4.12.2

Steps with poetry

$ docker build .
$ docker run --rm -it <image_name>
$ source venv/bin/activate
$ poetry install --no-root

Installing dependencies from lock file

Package operations: 4 installs, 0 updates, 0 removals

  - Installing typing-extensions (4.12.2)
  - Installing annotated-types (0.7.0)
  - Installing pydantic-core (2.14.5): Failed

  ChefBuildError

  Backend subprocess exited when trying to invoke build_wheel

  Running `maturin pep517 build-wheel -i /usr/bin/python3.11 --compatibility off`
      Updating crates.io index
   Downloading crates ...
    Downloaded windows-targets v0.48.1
    Downloaded tinyvec_macros v0.1.1
    Downloaded equivalent v1.0.1
    Downloaded unicode-bidi v0.3.13
    Downloaded strum v0.25.0
    Downloaded form_urlencoded v1.2.0
    Downloaded rustversion v1.0.13
    Downloaded unindent v0.2.3
    Downloaded percent-encoding v2.3.0
    Downloaded target-lexicon v0.12.9
    Downloaded cfg-if v1.0.0
    Downloaded static_assertions v1.1.0
    Downloaded scopeguard v1.1.0
    Downloaded pyo3-macros v0.20.0
    Downloaded redox_syscall v0.3.5
    Downloaded enum_dispatch v0.3.12
    Downloaded bitflags v1.3.2
    Downloaded autocfg v1.1.0
    Downloaded strum_macros v0.25.3
    Downloaded heck v0.4.1
    Downloaded memoffset v0.9.0
    Downloaded itoa v1.0.8
    Downloaded pyo3-build-config v0.20.0
    Downloaded num-integer v0.1.45
    Downloaded version_check v0.9.4
    Downloaded indoc v2.0.4
    Downloaded lexical-parse-integer v0.8.6
    Downloaded wasi v0.11.0+wasi-snapshot-preview1
    Downloaded getrandom v0.2.10
    Downloaded lexical-core v0.8.5
    Downloaded speedate v0.13.0
    Downloaded once_cell v1.18.0
    Downloaded quote v1.0.29
    Downloaded lock_api v0.4.10
    Downloaded parking_lot v0.12.1
    Downloaded smallvec v1.11.1
    Downloaded num-traits v0.2.16
    Downloaded uuid v1.5.0
    Downloaded serde_derive v1.0.190
    Downloaded parking_lot_core v0.9.8
    Downloaded lexical-write-integer v0.8.5
    Downloaded unicode-ident v1.0.10
    Downloaded pyo3-macros-backend v0.20.0
    Downloaded proc-macro2 v1.0.69
    Downloaded ahash v0.8.6
    Downloaded python3-dll-a v0.2.9
    Downloaded url v2.4.1
    Downloaded tinyvec v1.6.0
    Downloaded ryu v1.0.14
    Downloaded pyo3-ffi v0.20.0
    Downloaded indexmap v2.0.0
    Downloaded cc v1.0.79
    Downloaded jiter v0.0.4
    Downloaded memchr v2.6.3
    Downloaded base64 v0.21.5
    Downloaded serde v1.0.190
    Downloaded num-bigint v0.4.4
    Downloaded lexical-util v0.8.5
    Downloaded lexical-write-float v0.8.5
    Downloaded hashbrown v0.14.0
    Downloaded unicode-normalization v0.1.22
    Downloaded zerocopy v0.7.20
    Downloaded serde_json v1.0.108
    Downloaded aho-corasick v1.0.2
    Downloaded lexical-parse-float v0.8.5
    Downloaded regex v1.10.2
    Downloaded syn v2.0.38
    Downloaded idna v0.4.0
    Downloaded regex-syntax v0.8.2
    Downloaded windows_x86_64_gnullvm v0.48.0
    Downloaded windows_aarch64_gnullvm v0.48.0
    Downloaded pyo3 v0.20.0
    Downloaded zerocopy-derive v0.7.20
    Downloaded regex-automata v0.4.3
    Downloaded windows_x86_64_msvc v0.48.0
    Downloaded libc v0.2.147
    Downloaded windows_aarch64_msvc v0.48.0
    Downloaded windows_x86_64_gnu v0.48.0
    Downloaded windows_i686_msvc v0.48.0
    Downloaded windows_i686_gnu v0.48.0
  📦 Including license file "/tmp/tmp1kuh06dt/pydantic_core-2.14.5/LICENSE"
  🍹 Building a mixed python/rust project
  🔗 Found pyo3 bindings
  🐍 Found CPython 3.11 at /usr/bin/python3.11
  📡 Using build options features, bindings from pyproject.toml
     Compiling autocfg v1.1.0
     Compiling proc-macro2 v1.0.69
     Compiling target-lexicon v0.12.9
     Compiling unicode-ident v1.0.10
     Compiling quote v1.0.29
     Compiling python3-dll-a v0.2.9
     Compiling once_cell v1.18.0
     Compiling libc v0.2.147
     Compiling heck v0.4.1
     Compiling static_assertions v1.1.0
     Compiling version_check v0.9.4
     Compiling cfg-if v1.0.0
     Compiling rustversion v1.0.13
     Compiling parking_lot_core v0.9.8
     Compiling lexical-util v0.8.5
     Compiling scopeguard v1.1.0
     Compiling tinyvec_macros v0.1.1
     Compiling smallvec v1.11.1
     Compiling tinyvec v1.6.0
     Compiling memchr v2.6.3
     Compiling serde v1.0.190
     Compiling indoc v2.0.4
     Compiling hashbrown v0.14.0
     Compiling unicode-bidi v0.3.13
     Compiling ahash v0.8.6
     Compiling num-traits v0.2.16
     Compiling lock_api v0.4.10
     Compiling num-integer v0.1.45
     Compiling num-bigint v0.4.4
     Compiling memoffset v0.9.0
     Compiling aho-corasick v1.0.2
     Compiling pyo3-build-config v0.20.0
     Compiling percent-encoding v2.3.0
     Compiling regex-syntax v0.8.2
     Compiling zerocopy v0.7.20
     Compiling syn v2.0.38
     Compiling lexical-parse-integer v0.8.6
     Compiling lexical-write-integer v0.8.5
     Compiling parking_lot v0.12.1
     Compiling getrandom v0.2.10
     Compiling serde_json v1.0.108
     Compiling unicode-normalization v0.1.22
     Compiling lexical-write-float v0.8.5
     Compiling lexical-parse-float v0.8.5
     Compiling unindent v0.2.3
     Compiling equivalent v1.0.1
     Compiling indexmap v2.0.0
     Compiling form_urlencoded v1.2.0
     Compiling itoa v1.0.8
     Compiling ryu v1.0.14
     Compiling uuid v1.5.0
     Compiling idna v0.4.0
     Compiling base64 v0.21.5
     Compiling lexical-core v0.8.5
     Compiling url v2.4.1
     Compiling pyo3-ffi v0.20.0
     Compiling pyo3 v0.20.0
     Compiling pydantic-core v2.14.5 (/tmp/tmp1kuh06dt/pydantic_core-2.14.5)
  error: failed to run custom build command for `pydantic-core v2.14.5 (/tmp/tmp1kuh06dt/pydantic_core-2.14.5)`

  Caused by:
    process didn't exit successfully: `/tmp/tmp1kuh06dt/pydantic_core-2.14.5/target/release/build/pydantic-core-5e5a4926f685a323/build-script-build` (exit status: 101)
    --- stdout
    cargo:rustc-cfg=Py_3_6
    cargo:rustc-cfg=Py_3_7
    cargo:rustc-cfg=Py_3_8
    cargo:rustc-cfg=Py_3_9
    cargo:rustc-cfg=Py_3_10
    cargo:rustc-cfg=Py_3_11
    cargo:rerun-if-changed=python/pydantic_core/core_schema.py
    cargo:rerun-if-changed=generate_self_schema.py

    --- stderr
    Traceback (most recent call last):
      File "/tmp/tmp1kuh06dt/pydantic_core-2.14.5/generate_self_schema.py", line 17, in <module>
        from typing_extensions import TypedDict, get_args, get_origin, is_typeddict
    ModuleNotFoundError: No module named 'typing_extensions'
    thread 'main' panicked at build.rs:29:9:
    generate_self_schema.py failed with exit status: 1
    note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
  warning: build failed, waiting for other jobs to finish...
  💥 maturin failed
    Caused by: Failed to build a native library through cargo
    Caused by: Cargo build finished with "exit status: 101": `env -u CARGO PYO3_ENVIRONMENT_SIGNATURE="cpython-3.11-64bit" PYO3_PYTHON="/usr/bin/python3.11" PYTHON_SYS_EXECUTABLE="/usr/bin/python3.11" "cargo" "rustc" "--features" "pyo3/extension-module" "--message-format" "json-render-diagnostics" "--manifest-path" "/tmp/tmp1kuh06dt/pydantic_core-2.14.5/Cargo.toml" "--release" "--lib" "--crate-type" "cdylib"`
  Error: command ['maturin', 'pep517', 'build-wheel', '-i', '/usr/bin/python3.11', '--compatibility', 'off'] returned non-zero exit status 1

  at /opt/poetry/venv/lib/python3.11/site-packages/poetry/installation/chef.py:164 in _prepare
      160│
      161│                 error = ChefBuildError("\n\n".join(message_parts))
      162│
      163│             if error is not None:
    → 164│                 raise error from None
      165│
      166│             return path
      167│
      168│     def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:

Note: This error originates from the build backend, and is likely not a problem with poetry but with pydantic-core (2.14.5) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "pydantic-core (==2.14.5)"'.

Workarounds

Install pydantic from binary or without virtual environment

Poetry Installation Method

install.python-poetry.org

Operating System

Debian 12 (bookworm)

Poetry Version

1.8.3

Poetry Configuration

ache-dir = "/root/.cache/pypoetry"
experimental.system-git-client = false
installer.max-workers = null
installer.modern-installation = true
installer.no-binary = "[annotated-types, pydantic, pydantic-core, typing-extensions]"
installer.parallel = true
keyring.enabled = true
solver.lazy-wheel = true
virtualenvs.create = false
virtualenvs.in-project = null
virtualenvs.options.always-copy = false
virtualenvs.options.no-pip = false
virtualenvs.options.no-setuptools = false
virtualenvs.options.system-site-packages = false
virtualenvs.path = "{cache-dir}/virtualenvs"  # /root/.cache/pypoetry/virtualenvs
virtualenvs.prefer-active-python = false
virtualenvs.prompt = "{project_name}-py{python_version}"
warnings.export = true

Python Sysconfig

No response

Example pyproject.toml

No response

Poetry Runtime Logs

See description
dimbleby commented 4 days ago

As the error message says, this is not likely to be a poetry bug.

n-bes commented 4 days ago

Looks like typing_extensions module not passed to build environment. I think that it is combination of poetry and virtual-environment. I will duplicate the issue to pydantic too.

dimbleby commented 4 days ago

Looks like typing_extensions module not passed to build environment

Unlikely. Looks more like pydantic-core has found a way to run a python that is not the one in the build environment.

dimbleby commented 2 days ago

you can take poetry out of the equation by cloning pydantic-core, pip-installing build, and then python -m build - resulting in the same error.

please close

n-bes commented 11 hours ago

See: https://github.com/pydantic/pydantic-core/issues/1352