uv compatibility with existing pyproject.toml managed by poetry

[fleimgruber]

Steps to Reproduce

1. Add this section in pyproject.toml (this is done by default when poetry init is used):

   [build-system]
   requires = ["poetry-core"]
   build-backend = "poetry.core.masonry.api"

2. rye sync

Expected Result

rye syncs without an error.

Actual Result

Reusing already existing virtualenv
Generating production lockfile: C:\Users\LeimgruberF\dev\hyperride\requirements.lock
error: Failed to build editables
  Caused by: Failed to build editable: file:///C:/Users/LeimgruberF/dev/hyperride
  Caused by: Build backend failed to build wheel through `build_editable()` with exit code: 1
--- stdout:

--- stderr:
Traceback (most recent call last):
  File "<string>", line 11, in <module>
  File "C:\Users\LeimgruberF\AppData\Local\uv\cache\.tmpnKihhy\.venv\Lib\site-packages\poetry\core\masonry\api.py", line 82, in build_editable
    return WheelBuilder.make_in(
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\LeimgruberF\AppData\Local\uv\cache\.tmpnKihhy\.venv\Lib\site-packages\poetry\core\masonry\builders\wheel.py", line 88, in make_in
    wb.build(target_dir=directory)
  File "C:\Users\LeimgruberF\AppData\Local\uv\cache\.tmpnKihhy\.venv\Lib\site-packages\poetry\core\masonry\builders\wheel.py", line 118, in build
    self._add_pth(zip_file)
  File "C:\Users\LeimgruberF\AppData\Local\uv\cache\.tmpnKihhy\.venv\Lib\site-packages\poetry\core\masonry\builders\wheel.py", line 147, in _add_pth
    for include in self._module.includes:
                   ^^^^^^^^^^^^
  File "C:\Users\LeimgruberF\.rye\py\cpython@3.11.8\Lib\functools.py", line 1001, in __get__
    val = self.func(instance)
          ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\LeimgruberF\AppData\Local\uv\cache\.tmpnKihhy\.venv\Lib\site-packages\poetry\core\masonry\builders\builder.py", line 97, in _module
    return Module(
           ^^^^^^^
  File "C:\Users\LeimgruberF\AppData\Local\uv\cache\.tmpnKihhy\.venv\Lib\site-packages\poetry\core\masonry\utils\module.py", line 65, in __init__
    raise ModuleOrPackageNotFound(
poetry.core.masonry.utils.module.ModuleOrPackageNotFound: No file/folder found for package hyperride
---
error: could not write production lockfile for project

Caused by:
    Failed to run uv compile C:\Users\LEIMGR~1\AppData\Local\Temp\.tmpgIWp9Q\requirements.txt. uv exited with status: exit code: 2

Version Info

rye 0.31.0 commit: 0.31.0 (72e239b8f 2024-03-22) platform: windows (x86_64) self-python: cpython@3.12.1 symlink support: true uv enabled: true

Stacktrace

No response

[bluss]

Maybe more details need to be filled in for this issue. Are you using poetry's build system for a rye managed package, or for a dependency? Could you find a minimal reproducing example for this problem? (I've tried to reproduce but did not find the same problems, so it would help.)

Rye does not by itself support poetry to build wheels for rye managed projects - I say that just because it's not an option to rye init --build-system, which has these alternatives right now: hatchling setuptools flit pdm maturin

[fleimgruber]

Thanks for checking back.

Are you using poetry's build system for a rye managed package, or for a dependency?

I am trying to use rye on a pyproject.toml that contains [tool.poetry] sections because other developers working on it use poetry. I do not use poetry's build system directly or intentionally. As mentioned in OP, the [build-system] section there comes from e.g. poetry init and might be relied on by other developers. When I remove that section, rye sync works as expected. This issue is about achieving compatibility with that section being present.

I just reproduced this on NixOS with this pyproject.toml (traceback is equivalent to the one in OP):

[project]
name = "rye-poetry-test"
version = "0.1.0"
description = ""
readme = "README.md"

[tool.poetry]
name = "rye-poetry-test"
version = "0.1.0"
description = ""
authors = [
    "John Doe <john@doe.com>",
]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.11"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

rye --version

rye 0.33.0
commit: unknown (54405065a 2024-03-29)
platform: linux (x86_64)
self-python: cpython@3.12.2
symlink support: true
uv enabled: true

[bluss]

The minimal example fails because there is no package to install, it can be fixed like this:

mkdir -p src/rye_poetry_test touch src/rye_poetry_test/__init__.py

However that's probably not representative of the issue. What's the case for the actual poetry project, is it a package that can be built as an editable wheel? Or is poetry configuring it some other way.

I would say it's not expected that rye can manage foreign packages, that's not in the design, at least especially not if they are not buildable with standard tools (This refers to how relevant PEPs say packages defining build-system etc should be built and installed). (This is just IMO)

[fleimgruber]

The minimal example fails because there is no package to install, it can be fixed like this:

Thanks for this alternative workaround, but the project does not use the src/ layout and I do not want to add any files.

I also do not want to use the other workaround (i.e. removing the [build-system] section) mentioned in https://github.com/astral-sh/rye/issues/954#issuecomment-2028014001 as I would be changing that file for other developers who might rely on it.

However that's probably not representative of the issue. What's the case for the actual poetry project, is it a package that can be built as an editable wheel? Or is poetry configuring it some other way.

I intended this issue as an exploration of how much compatibility we can achieve with existing poetry, ideally without any changes to a given project. To answer your question: my use case is to develop the source package, not to create wheels.

I realize I might have misnamed this issue as I am not concerned with the effects or the usage of the poetry build system, it just so happened that I was able to avoid the error by removing that section from pyproject.toml (see workaround above), hence I suspected it being problematic and thus my idea of establishing compatibility with that "problematic" part. I am going to rename the issue accordingly.

[jayqi]

EDIT: I did not read this thread carefully and made some mistakes. My originally comment is probably not useful discussion, and I've removed it so that it's not a distraction or picked up by search engines.

[dimbleby]

Basically, poetry-core only works with Poetry as a frontend, and is not compatible with any other Python packaging tool

this is untrue and misunderstands the relationship between front ends and backends, as you can verify with commands like:

$ git clone git@github.com:python-poetry/poetry.git  # or any other poetry-using project
$ cd poetry
$ pip install .  # or uv pip install .

[jayqi]

Sorry about that, you are right. I am guilty of not carefully reading this thread and made some wrong assumptions about what was happening.

It is true that Poetry is not yet PEP-621-compliant, which means that the stuff under the [project] table in the pyproject.toml is not supported by poetry-core (and is probably ignored). However, the stuff under [tool.poetry] is sufficient for poetry-core to build the package (with any PEP 517 frontend), so this fact isn't relevant to the errors being shown here.

You're also right that the way I described how frontends is not fully accurate, and I've conflated some concepts from frontends with package metadata management. On a closer look, I don't think frontends are relevant.

I've edited my comment with strikethroughs so that they don't confuse any future readers.