Error running rye sync with universal=true on Windows when using uv

dluo-sig commented 3 weeks ago

Steps to Reproduce

In order for this to occur, it seems like it needs the combination of:

Be on Windows
use-uv=true in .rye/config.toml
universal=true in pyproject.toml

My pyproject.toml

[project]
name = "dagster-hybrid"
version = "0.1.0"
description = "..."
authors = [
    { name = "...", email = "..." }
]
dependencies = [
    "dagster>=1.8.0",
    "dagster-cloud>=1.7.14",
    "dagster-shell>=0.23.14",
    "dagster-snowflake>=0.23.14",
    "dagster-dbt>=0.23.14",
    "dbt-snowflake>=1.8.3",
    "dagster-embedded-elt>=0.23.14",
    "adlfs>=2024.4.1",
    "dlt[snowflake]>=0.5.1",
    "openpyxl>=3.1.5",
    "sqlalchemy>=2.0.31",
    "pyodbc>=5.1.0",
    "pyarrow>=17.0.0",
    "pandas>=2.2.2",
]
readme = "README.md"
requires-python = ">= 3.8"

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.rye]
managed = true
universal = true
dev-dependencies = [
    "dagit>=1.7.14",
    "dagster-webserver>=1.7.14",
    "pytest>=8.2.2",
]

[tool.hatch.metadata]
allow-direct-references = true

[tool.hatch.build.targets.wheel]
packages = ["src/orchestration"]

Expected Result

rye sync should not error out.

Actual Result

λ rye sync
Initializing new virtualenv in C:\git\dagster-hybrid\.venv
Python version: cpython@3.12.3
Generating production lockfile: C:\git\dagster-hybrid\requirements.lock
Generating dev lockfile: C:\git\dagster-hybrid\requirements-dev.lock
Installing dependencies
Resolved 179 packages in 125ms
   Built dagster-hybrid @ file:///C:/git/dagster-hybrid
error: Failed to prepare distributions
  Caused by: Failed to fetch wheel: minimal-snowplow-tracker==0.0.2
  Caused by: Failed to build: `minimal-snowplow-tracker==0.0.2`
  Caused by: Build backend failed to build wheel through `build_wheel()` with exit code: 1
--- stdout:
running bdist_wheel
running build
running build_py
copying snowplow_tracker\emitters.py -> build\lib\snowplow_tracker
copying snowplow_tracker\payload.py -> build\lib\snowplow_tracker
copying snowplow_tracker\self_describing_json.py -> build\lib\snowplow_tracker
copying snowplow_tracker\subject.py -> build\lib\snowplow_tracker
copying snowplow_tracker\timestamp.py -> build\lib\snowplow_tracker
copying snowplow_tracker\tracker.py -> build\lib\snowplow_tracker
copying snowplow_tracker\_version.py -> build\lib\snowplow_tracker
copying snowplow_tracker\__init__.py -> build\lib\snowplow_tracker
copying snowplow_tracker\test\__init__.py -> build\lib\snowplow_tracker\test
installing to build\bdist.win-amd64\wheel
running install
running install_lib
running install_egg_info
running egg_info
writing minimal_snowplow_tracker.egg-info\PKG-INFO
writing dependency_links to minimal_snowplow_tracker.egg-info\dependency_links.txt
writing requirements to minimal_snowplow_tracker.egg-info\requires.txt
writing top-level names to minimal_snowplow_tracker.egg-info\top_level.txt
reading manifest file 'minimal_snowplow_tracker.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
adding license file 'LICENSE.txt'
writing manifest file 'minimal_snowplow_tracker.egg-info\SOURCES.txt'
removing 'build\bdist.win-amd64\wheel\.\minimal_snowplow_tracker-0.0.2-py3.12.egg-info' (and everything under it)
Copying minimal_snowplow_tracker.egg-info to build\bdist.win-amd64\wheel\.\minimal_snowplow_tracker-0.0.2-py3.12.egg-info
--- stderr:
warning: no files found matching '*.txt' under directory 'docs'
error: [Errno 2] No such file or directory: 'build\\bdist.win-amd64\\wheel\\.\\minimal_snowplow_tracker-0.0.2-py3.12.egg-info\\dependency_links.txt'
---
error: Installation of dependencies failed in venv at C:\git\dagster-hybrid\.venv. uv exited with status: exit code: 2

Version Info

rye 0.38.0 commit: 0.38.0 (3e3c8540f 2024-08-02) platform: windows (x86_64) self-python: cpython@3.12.3 symlink support: true uv enabled: true

Stacktrace

No response

charliermarsh commented 3 weeks ago

What do you get when you run pip install --use-pep517 minimal-snowplow-tracker==0.0.2?

charliermarsh commented 3 weeks ago

I haven't been able to reproduce this yet unfortunately.

dluo-sig commented 3 weeks ago

That seems to be fine.

(.venv) λ pip install --use-pep517 minimal-snowplow-tracker==0.0.2
Collecting minimal-snowplow-tracker==0.0.2
  Using cached minimal_snowplow_tracker-0.0.2-py3-none-any.whl
Collecting requests<3.0,>=2.2.1 (from minimal-snowplow-tracker==0.0.2)
  Using cached requests-2.32.3-py3-none-any.whl.metadata (4.6 kB)
Collecting six<2.0,>=1.9.0 (from minimal-snowplow-tracker==0.0.2)
  Using cached six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0,>=2.2.1->minimal-snowplow-tracker==0.0.2)
  Using cached charset_normalizer-3.3.2-cp312-cp312-win_amd64.whl.metadata (34 kB)
Collecting idna<4,>=2.5 (from requests<3.0,>=2.2.1->minimal-snowplow-tracker==0.0.2)
  Using cached idna-3.7-py3-none-any.whl.metadata (9.9 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0,>=2.2.1->minimal-snowplow-tracker==0.0.2)
  Using cached urllib3-2.2.2-py3-none-any.whl.metadata (6.4 kB)
Collecting certifi>=2017.4.17 (from requests<3.0,>=2.2.1->minimal-snowplow-tracker==0.0.2)
  Using cached certifi-2024.7.4-py3-none-any.whl.metadata (2.2 kB)
Using cached requests-2.32.3-py3-none-any.whl (64 kB)
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Using cached certifi-2024.7.4-py3-none-any.whl (162 kB)
Using cached charset_normalizer-3.3.2-cp312-cp312-win_amd64.whl (100 kB)
Using cached idna-3.7-py3-none-any.whl (66 kB)
Using cached urllib3-2.2.2-py3-none-any.whl (121 kB)
Installing collected packages: urllib3, six, idna, charset-normalizer, certifi, requests, minimal-snowplow-tracker
Successfully installed certifi-2024.7.4 charset-normalizer-3.3.2 idna-3.7 minimal-snowplow-tracker-0.0.2 requests-2.32.3 six-1.16.0 urllib3-2.2.2

[notice] A new release of pip is available: 24.1.2 -> 24.2
[notice] To update, run: python.exe -m pip install --upgrade pip

dluo-sig commented 3 weeks ago

Now I'm not sure if universal=true is required to repro. I'm having the same issue when removing that line, though I could've sworn it had worked without it previously. I think somehow it skipped the minimal-snowplow-tracker build step somehow last time.

charliermarsh commented 3 weeks ago

Yeah in the initial issue summary, it looks like minimal-snowplow-tracker==0.0.2 is being built from source. But in pip install --use-pep517 minimal-snowplow-tracker==0.0.2, you're using a cached wheel (minimal_snowplow_tracker-0.0.2-py3-none-any.whl).

dluo-sig commented 3 weeks ago

Is that something that's pulled from pypi? Not sure why the behavior would change.

charliermarsh commented 3 weeks ago

I'll try to reproduce this on my Windows machine.

dluo-sig commented 3 weeks ago

Oh, I don't have this issue on the same computer running Ubuntu from WSL2, so I think it should be Windows specific.

dluo-sig commented 2 weeks ago

So after running with pip-tools, switching back to uv works. But I did notice this difference in the lock file generated, not sure if it might be related. I don't think there should be any difference when universal=true. That said, fresh sync with new lock file still fails with same error.

λ git diff
diff --git a/requirements-dev.lock b/requirements-dev.lock
index 035ed06..ddab9f2 100644
--- a/requirements-dev.lock
+++ b/requirements-dev.lock
@@ -428,7 +428,7 @@ six==1.16.0
     # via python-dateutil
 sling==1.2.15
     # via dagster-embedded-elt
-sling-linux-amd64==1.2.15
+sling-windows-amd64==1.2.15
     # via sling
 smmap==5.0.1
     # via gitdb
diff --git a/requirements.lock b/requirements.lock
index 3cabcbd..58c5aea 100644
--- a/requirements.lock
+++ b/requirements.lock
@@ -381,7 +381,7 @@ six==1.16.0
     # via python-dateutil
 sling==1.2.15
     # via dagster-embedded-elt
-sling-linux-amd64==1.2.15
+sling-windows-amd64==1.2.15
     # via sling
 smmap==5.0.1
     # via gitdb

astral-sh / rye