`uv`'s tool installation feature breaks on Wine

alexdelorenzo commented 1 month ago

Reproduce

Steps 1 - 7 should run successfully, step 8 results in an error:

Using Linux, install Wine from your package manager or compile Wine 9.12
Run winecfg, set Windows version to Windows 10 or 11 if it isn't already set
Run wget "https://www.python.org/ftp/python/3.12.4/python-3.12.4-amd64.exe"
Run wine python-3.12.4-amd64.exe
Follow Python's setup, install it to the system and not just for the user
Run wine python -m ensurepip
Run wine python -m pip install uv
Run wine python -m uv tool install wheel, see output below

Output

$ wine python -m uv tool install wheel
warning: `uv tool install` is experimental and may change without warning.
⠙ Resolving dependencies...
⠙ wheel==0.43.0
Resolved 1 package in 16ms
wheel      ------------------------------ 65.78 kB/65.78 kB
error: Failed to prepare distributions
  Caused by: Failed to fetch wheel: wheel==0.43.0
  Caused by: Failed to read from the distribution cache
  Caused by: Request not supported. (os error 50)

Expected output

Run on Windows 11:

$ python -m uv tool install wheel
Resolved 1 package in 6ms
Installed 1 package in 1.95s
 + wheel==0.43.0

Installed scripts:
  - wheel

Notes

uv version is 0.2.25 (d1010228b 2024-07-15)
This might be related to bug #1491 on Windows, the results are similar but the os error is 5, not 50. I cannot get uv to install tools no matter how many times I run it, unlike the reporter in #1491 is able to.
This also breaks Rye on Wine and can be reproduced with the following:

Run winecfg, set Windows version to Windows 10 or 11 if it isn't already set
Run wget "https://github.com/astral-sh/rye/releases/latest/download/rye-x86_64-windows.exe"
Run wine rye-x86_64-windows.exe
Follow Rye's setup script using the default options, it will install successfully
Run wine rye install wheel:

⠙ Resolving dependencies...
⠙ wheel==0.43.0
Resolved 1 package in 6ms
wheel      ------------------------------ 64.39 kB/65.78 kB
error: Failed to prepare distributions
  Caused by: Failed to fetch wheel: wheel==0.43.0
  Caused by: Failed to read from the distribution cache
  Caused by: Request not supported. (os error 50)
error: Installation of wheel failed in venv at C:\users\user\.rye\tools\wheel. uv exited with status: exit code: 2

charliermarsh commented 1 month ago

Interesting, but Rye doesn't use the uv tool feature, I don't think?

alexdelorenzo commented 1 month ago

Interesting, but Rye doesn't use the uv tool feature, I don't think?

I haven't run it in a venv to test, the issue might be with uv's general package installation feature and not just its tool feature.

zanieb commented 1 month ago

This doesn't look related to tool installs, just something with our cache?

Are you getting a Windows version of uv? I imagine this is some weird interaction of our Windows-specific file system code on a machine that's actually using a Linux file system? Can you share the output with RUST_LOG=trace and the -v flag.

alexdelorenzo commented 1 month ago

This doesn't look related to tool installs, just something with our cache?

Are you getting a Windows version of uv? I imagine this is some weird interaction of our Windows-specific file system code on a machine that's actually using a Linux file system? Can you share the output with RUST_LOG=trace and the -v flag.

I downloaded this archive and ran the uv.exe in it with Wine and got the same result.

$ wine ./uv.exe tool install wheel
warning: `uv tool install` is experimental and may change without warning.
⠋ Resolving dependencies...
⠙ wheel==0.43.0
Resolved 1 package in 99ms
wheel      ------------------------------ 65.78 kB/65.78 kB
error: Failed to prepare distributions
  Caused by: Failed to fetch wheel: wheel==0.43.0
  Caused by: Failed to read from the distribution cache
  Caused by: Request not supported. (os error 50)

Here's the output when the env var and -v flag are set:

Click for log

``` DEBUG uv 0.2.25 warning: `uv tool install` is experimental and may change without warning. DEBUG Searching for Python interpreter in managed installations, system path, or `py` launcher DEBUG Searching for managed installations at `C:\users\user\AppData\Roaming\uv\data\python` TRACE Searching PATH for executables: python3.exe, python.exe TRACE Checking `PATH` directory for interpreters: C:\Program Files\Python312\Scripts\ TRACE Checking `PATH` directory for interpreters: C:\Program Files\Python312\ TRACE Found possible Python executable: C:\Program Files\Python312\python.exe TRACE Cached interpreter info for Python 3.12.4, skipping probing: C:\Program Files\Python312\python.exe DEBUG Found cpython 3.12.4 at `C:\Program Files\Python312\python.exe` (search path) DEBUG Using request timeout of 30s DEBUG Using request timeout of 30s TRACE Checking lock for `C:\users\user\AppData\Roaming\uv\data\tools` DEBUG Acquired lock for `C:\users\user\AppData\Roaming\uv\data\tools` TRACE Cached interpreter info for Python 3.12.4, skipping probing: C:\users\user\AppData\Roaming\uv\data\tools\wheel\Scripts\python.exe DEBUG Using existing environment for tool `wheel`: C:\users\user\AppData\Roaming\uv\data\tools\wheel DEBUG At least one requirement is not satisfied: wheel DEBUG Using request timeout of 30s DEBUG Solving with installed Python version: 3.12.4 DEBUG Adding direct dependency: wheel* INFO add_decision: root @ 0a0.dev0 TRACE Fetching metadata for wheel from https://pypi.org/simple/wheel/ TRACE cached request https://pypi.org/simple/wheel/ is storable because its response has a 'public' cache-control directive DEBUG Found fresh response for: https://pypi.org/simple/wheel/ TRACE Received package metadata for: wheel DEBUG Searching for a compatible version of wheel (*) TRACE selecting candidate for package wheel with range Range { segments: [(Unbounded, Unbounded)] } with 75 remote versions TRACE selecting candidate for package wheel with range Range { segments: [(Unbounded, Unbounded)] } with 75 remote versions TRACE found candidate for package PackageName("wheel") with range Range { segments: [(Unbounded, Unbounded)] } after 1 steps: "0.43.0" version TRACE found candidate for package PackageName("wheel") with range Range { segments: [(Unbounded, Unbounded)] } after 1 steps: "0.43.0" version DEBUG Selecting: wheel==0.43.0 (wheel-0.43.0-py3-none-any.whl) TRACE cached request https://files.pythonhosted.org/packages/7d/cd/d7460c9a869b16c3dd4e1e403cce337df165368c71d6af229a74699622ce/wheel-0.43.0-py3-none-any.whl.metadata is storable because its response has a 'public' cache-control directive DEBUG Found fresh response for: https://files.pythonhosted.org/packages/7d/cd/d7460c9a869b16c3dd4e1e403cce337df165368c71d6af229a74699622ce/wheel-0.43.0-py3-none-any.whl.metadata TRACE Received built distribution metadata for: wheel==0.43.0 INFO add_decision: wheel @ 0.43.0 DEBUG Tried 1 versions: wheel 1 DEBUG Split took 0.005s Resolved 1 package in 10ms DEBUG Identified uncached requirement: wheel==0.43.0 TRACE No cache entry exists for \\?\C:\users\user\AppData\Local\uv\cache\wheels-v1\pypi\wheel\wheel-0.43.0-py3-none-any.http DEBUG No cache entry for: https://files.pythonhosted.org/packages/7d/cd/d7460c9a869b16c3dd4e1e403cce337df165368c71d6af229a74699622ce/wheel-0.43.0-py3-none-any.whl TRACE Sending fresh GET request for https://files.pythonhosted.org/packages/7d/cd/d7460c9a869b16c3dd4e1e403cce337df165368c71d6af229a74699622ce/wheel-0.43.0-py3-none-any.whl TRACE Handling request for https://files.pythonhosted.org/packages/7d/cd/d7460c9a869b16c3dd4e1e403cce337df165368c71d6af229a74699622ce/wheel-0.43.0-py3-none-any.whl TRACE Request for https://files.pythonhosted.org/packages/7d/cd/d7460c9a869b16c3dd4e1e403cce337df165368c71d6af229a74699622ce/wheel-0.43.0-py3-none-any.whl is unauthenticated, checking cache TRACE No credentials in cache for URL https://files.pythonhosted.org/packages/7d/cd/d7460c9a869b16c3dd4e1e403cce337df165368c71d6af229a74699622ce/wheel-0.43.0-py3-none-any.whl TRACE Attempting unauthenticated request for https://files.pythonhosted.org/packages/7d/cd/d7460c9a869b16c3dd4e1e403cce337df165368c71d6af229a74699622ce/wheel-0.43.0-py3-none-any.whl TRACE checkout waiting for idle connection: ("https", files.pythonhosted.org) DEBUG starting new connection: https://files.pythonhosted.org/ TRACE Http::connect; scheme=Some("https"), host=Some("files.pythonhosted.org"), port=None DEBUG resolving host="files.pythonhosted.org" DEBUG connecting to [2a04:4e42:8e::223]:443 DEBUG connected to [2a04:4e42:8e::223]:443 TRACE http1 handshake complete, spawning background dispatcher task TRACE checkout dropped for ("https", files.pythonhosted.org) TRACE cached request https://files.pythonhosted.org/packages/7d/cd/d7460c9a869b16c3dd4e1e403cce337df165368c71d6af229a74699622ce/wheel-0.43.0-py3-none-any.whl is storable because its response has a 'public' cache-control directive TRACE put; add idle connection for ("https", files.pythonhosted.org) DEBUG pooling idle connection for ("https", files.pythonhosted.org) error: Failed to prepare distributions Caused by: Failed to fetch wheel: wheel==0.43.0 Caused by: Failed to read from the distribution cache Caused by: Request not supported. (os error 50) ```

zanieb commented 1 month ago

Thanks for the logs! Unfortunately there are a ton of possible places we can throw a cache read error in this code path.

However, the error code 50 looks associated with taking locks — which is essential for safe operation. See https://docs.oracle.com/en-us/iaas/Content/File/Troubleshooting/applicationfailstoacquirelock.htm

Could you explore those two options?

charliermarsh commented 1 month ago

I've never tested on Wine, maybe our fs ops are just making bad assumptions :)

alexdelorenzo commented 1 month ago

Did some digging and this may be a Wine issue at first glance.

When using Rye with uv or pip-tools as backends, and then running wine rye install wheel, Wine emits this message:

$ wine rye install wheel
0134:fixme:heap:GetNumaHighestNodeNumber semi-stub: 00007FFFFE1FF9C0
0134:fixme:file:CopyFileExW LPPROGRESS_ROUTINE is not supported
0134:fixme:file:CopyFileExW LPPROGRESS_ROUTINE is not supported
0134:fixme:kernelbase:AppPolicyGetProcessTerminationMethod FFFFFFFFFFFFFFFA, 00007FFFFE1FFE80
017c:fixme:heap:GetNumaHighestNodeNumber semi-stub: 00007FFFFE1FF9C0
017c:fixme:file:NtLockFile I/O completion on lock not implemented yet
⠋ Resolving dependencies...
⠙ wheel==0.43.0
Resolved 1 package in 104ms
wheel      ------------------------------ 65.78 kB/65.78 kB
error: Failed to prepare distributions
  Caused by: Failed to fetch wheel: wheel==0.43.0
  Caused by: Failed to read from the distribution cache
  Caused by: Request not supported. (os error 50)
017c:fixme:kernelbase:AppPolicyGetProcessTerminationMethod FFFFFFFFFFFFFFFA, 00007FFFFE1FFE80
31m1merror: Installation of wheel failed in venv at C:\users\alex\.rye\tools\wheel. uv exited with status: exit code: 2

The line:

fixme:file:NtLockFile I/O completion on lock not implemented yet

Comes from here in Wine.

Sounds like it's related, but I'm not familiar enough with what Rye/uv/etc is doing file system wise to say definitively.

astral-sh / uv