search

astral-sh / uv

An extremely fast Python package and project manager, written in Rust.
https://docs.astral.sh/uv
Apache License 2.0
26.03k stars 762 forks source link

uv pip install inconsistent failure on Windows #1491

Closed Wofiel closed 3 months ago

Wofiel commented 9 months ago

When using uv pip install -r requirements.txt, I will frequently get the following error:

error: Failed to download distributions
  Caused by: Failed to fetch wheel: setproctitle==1.3.3
  Caused by: Failed to read from the distribution cache
  Caused by: failed to rename file from \\?\C:\Users\[____]\AppData\Local\Temp\1\.tmp56ZHLz\.tmpMXOwQn to \\?\C:\Users\[____]\AppData\Local\Temp\1\.tmp56ZHLz\archive-v0\7RXhEx7TcTNKRTlP3vSED
  Caused by: Access is denied. (os error 5)

This appears to be independent of the package, though seemingly happens more often with some than others.

If run with --no-cache, it's almost impossible to install, as at least one thing is likely to fail in the process and progress restarted.

If run without --no-cache, retrying a number of times will eventually succeed (however, it will leave .tmp[_______] folders listed in the error messages in C:\Users\[____]\AppData\Local\uv\cache).

ofek commented 8 months ago

Was it in response to a long path issue?

I'm not a maintainer but I know the answer: that is the behavior of the standard library function to resolve a path. From my understanding it was basically a mistake but nobody can change it now. Instead, it's common to use the dunce crate or the normpath crate when you don't want to resolve symlinks.

edit: there is now also https://doc.rust-lang.org/std/path/fn.absolute.html on nightly

charliermarsh commented 8 months ago

We do use dunce and strip the prefix anywhere that it interfaces outside of standard library filesystem operations.

charliermarsh commented 8 months ago

@mkleinbort - Did you install that package prior to the latest uv? It could be leftover from your virtual environment being in a bad state.

charliermarsh commented 8 months ago

I'm open to changing to use the dunce-canonicalized version everywhere, but note that it doesn't, like, fully solve the problem (if it is a problem). dunce will still return a UNC prefix if the path is longer than 260 characters.

omdaniel commented 8 months ago

The merged PR #2419 has thus far eliminated failures on the handful pip installs I've done since upgrading. Thanks @charliermarsh

mkleinbort-ic commented 8 months ago

@charliermarsh

Not sure - I hit this error just now trying to upgrade to uv==0.1.21 (I'm on 0.1.20)

image

(Running on gitbash on a windows machine)

Running pythom -m pip install uv==0.1.21 succeeded

FredStober commented 6 months ago

Hi, I observed exactly this issue "Caused by: Failed to fetch wheel..." when testing my dev environment on windows with uv==0.1.39.

The problem randomly appears on two different machines. One is running windows directly - and the other is using WSL with a windows mount. So far, the issue disappared after trying it another time.

jbcpollak commented 5 months ago

Ran into this issue with uv == 0.2.11, Python 3.12 and Windows 11 Pro. No specific enterprise extensions installed to Windows but we do have McAfee Antivirus installed.

axel-kah commented 4 months ago

Hi,

the latest version (uv 0.2.15 (bfc342da9 2024-06-24)) still has these issues. We see this on a certain percentage every on our Windows CI runners. They do have CrowdStrike messing with our workloads unfortunately - corporate policy 😢

The common denominator seems to be source distributions which uv tries to build wheels from. I've never seen this error for packages that are available as wheel.

Hypothesis: snakeoil scanners need more time to scan compressed archives (because of the decompression step) and uvs backoff algorithm just needs a little tweaking to accommodate for that.

A few source distribution only packages: fire, odfpy, mapdata, pywinusb

A bunch of error messages for these packages ```shell $ .\hatch.exe run all.py$INTERPRETER_VERSION`:cov Creating environment: all.py3.11 Installing project error: Failed to download and build `pywinusb==0.4.2` Caused by: Failed to write to the distribution cache Caused by: failed to rename file from \\?\C:\Users\gitlabwinrunner\AppData\Local\Temp\.tmp3V6aqK\built-wheels-v3\.tmpsxHt5d\pywinusb-0.4.2 to \\?\C:\Users\gitlabwinrunner\AppData\Local\Temp\.tmp3V6aqK\built-wheels-v3\index\70e6dbc41e17fe7c\pywinusb\0.4.2\w4S16F7Zw7fjYe2QmhqM_\pywinusb-0.4.2.zip Caused by: Access is denied. (os error 5) $ .\hatch.exe run all.py$INTERPRETER_VERSION`:cov Creating environment: all.py3.9 Installing Python distribution: 3.9 Installing project Checking dependencies Syncing dependencies error: Failed to download and build `odfpy==1.4.1` Caused by: Failed to write to the distribution cache Caused by: failed to rename file from \\?\C:\Users\gitlabwinrunner\AppData\Local\Temp\.tmp30woUu\built-wheels-v3\.tmp7d4zhJ\odfpy-1.4.1 to \\?\C:\Users\gitlabwinrunner\AppData\Local\Temp\.tmp30woUu\built-wheels-v3\index\70e6dbc41e17fe7c\odfpy\1.4.1\J27cgvxiwwWDBSM2Tnauu\odfpy-1.4.1.tar.gz Caused by: Access is denied. (os error 5) $ hatch run uv --version Creating environment: default Installing Python distribution: 3.11 Installing project in development mode Checking dependencies Syncing dependencies error: Failed to download and build `mapdata==3.11.1` Caused by: Failed to write to the distribution cache Caused by: failed to rename file from \\?\C:\Users\gitlabwinrunner\AppData\Local\Temp\.tmptBDtnM\built-wheels-v3\.tmpqFOiKl\mapdata-3.11.1 to \\?\C:\Users\gitlabwinrunner\AppData\Local\Temp\.tmptBDtnM\built-wheels-v3\index\70e6dbc41e17fe7c\mapdata\3.11.1\T7WG4FPsti03XVV0KCULn\mapdata-3.11.1.tar.gz Caused by: Access is denied. (os error 5) $ hatch run uv --version Creating environment: default Installing Python distribution: 3.11 Installing project in development mode Checking dependencies Syncing dependencies error: Failed to download and build `fire==0.6.0` Caused by: Failed to write to the distribution cache Caused by: failed to rename file from \\?\C:\Users\gitlabwinrunner\AppData\Local\Temp\.tmpVXpz23\built-wheels-v3\.tmpnCevMt\fire-0.6.0 to \\?\C:\Users\gitlabwinrunner\AppData\Local\Temp\.tmpVXpz23\built-wheels-v3\index\70e6dbc41e17fe7c\fire\0.6.0\pD3XrlBXA8_x3S3TYSW2o\fire-0.6.0.tar.gz Caused by: Access is denied. (os error 5) ```
johannesloibl commented 4 months ago

Never saw this kind of responsiveness in a project, respect! 🥇

zanieb commented 4 months ago

@axel-kah With --verbose logs do you see logs indicating that we are retrying? e.g. "Retrying rename from...."

We have a maximum backoff of 10s which is pretty long. Do you think we should be waiting longer than that?

axel-kah commented 4 months ago

@zanieb I modified the hatch script to install source-only packages with the latest uv and it seems it does not use any retry at all. Maybe that's the root cause.

verbose output ```shell $ hatch run provoke-issue Creating environment: default Installing project in development mode Checking dependencies Syncing dependencies cmd [1] | D:\glb\GwsU8_zK\0\uv.exe --version uv 0.2.15 (bfc342da9 2024-06-24) cmd [2] | D:\glb\GwsU8_zK\0\uv.exe --verbose pip install fire pywinusb nptdms odfpy ipyvuetable DEBUG uv 0.2.15 DEBUG Searching for Python interpreter in system toolchains DEBUG Found cpython 3.11.9 at `d:\glb\GwsU8_zK\0\.hatch_data\env\virtual\foo\Scripts\python.exe` (active virtual environment) DEBUG Using Python 3.11.9 environment at .hatch_data\env\virtual\foo\Scripts\python.exe DEBUG Acquired lock for `.hatch_data\env\virtual\foo` DEBUG At least one requirement is not satisfied: ipyvuetable DEBUG Using request timeout of 30s DEBUG Solving with installed Python version: 3.11.9 DEBUG Adding direct dependency: fire* DEBUG Adding direct dependency: pywinusb* DEBUG Adding direct dependency: nptdms* DEBUG Adding direct dependency: odfpy* DEBUG Adding direct dependency: ipyvuetable* DEBUG No cache entry for: https://artifactory.acme.com/artifactory/api/pypi/pypi-acme-vir/simple/fire/ DEBUG No cache entry for: https://artifactory.acme.com/artifactory/api/pypi/pypi-acme-vir/simple/pywinusb/ DEBUG No cache entry for: https://artifactory.acme.com/artifactory/api/pypi/pypi-acme-vir/simple/nptdms/ DEBUG No cache entry for: https://artifactory.acme.com/artifactory/api/pypi/pypi-acme-vir/simple/odfpy/ DEBUG No cache entry for: https://artifactory.acme.com/artifactory/api/pypi/pypi-acme-vir/simple/ipyvuetable/ DEBUG Acquired lock for `\\?\C:\Users\gitlabwinrunner\AppData\Local\Temp\.tmpanLS7H\built-wheels-v3\index\70e6dbc41e17fe7c\pywinusb\0.4.2` DEBUG No cache entry for: https://artifactory.acme.com/artifactory/api/pypi/pypi-acme-vir/packages/packages/38/b4/ecce4a3a0dac3b1bf5776943530cf4f36406fc9b3f4f3c31c8dcab2249eb/pywinusb-0.4.2.zip#sha256=e2f5e89f7b74239ca4843721a9bda0fc99014750630c189a176ec0e1b35e86df DEBUG Acquired lock for `\\?\C:\Users\gitlabwinrunner\AppData\Local\Temp\.tmpanLS7H\built-wheels-v3\index\70e6dbc41e17fe7c\odfpy\1.4.1` DEBUG No cache entry for: https://artifactory.acme.com/artifactory/api/pypi/pypi-acme-vir/packages/packages/97/73/8ade73f6749177003f7ce3304f524774adda96e6aaab30ea79fd8fda7934/odfpy-1.4.1.tar.gz#sha256=db766a6e59c5103212f3cc92ec8dd50a0f3a02790233ed0b52148b70d3c438ec DEBUG Searching for a compatible version of fire (*) DEBUG Selecting: fire==0.6.0 (fire-0.6.0.tar.gz) DEBUG Acquired lock for `\\?\C:\Users\gitlabwinrunner\AppData\Local\Temp\.tmpanLS7H\built-wheels-v3\index\70e6dbc41e17fe7c\fire\0.6.0` DEBUG No cache entry for: https://artifactory.acme.com/artifactory/api/pypi/pypi-acme-vir/packages/packages/1b/1b/84c63f592ecdfbb3d77d22a8d93c9b92791e4fa35677ad71a7d6449100f8/fire-0.6.0.tar.gz#sha256=54ec5b996ecdd3c0309c800324a0703d6da512241bc73b553db959d98de0aa66 DEBUG Acquired lock for `\\?\C:\Users\gitlabwinrunner\AppData\Local\Temp\.tmpanLS7H\built-wheels-v3\index\70e6dbc41e17fe7c\nptdms\1.9.0` DEBUG No cache entry for: https://artifactory.acme.com/artifactory/api/pypi/pypi-acme-vir/packages/packages/ff/05/8f560020155c1843d664248fb114e33eac0c1b3ad44fce6bfc2b5dd143c2/npTDMS-1.9.0.tar.gz#sha256=0e65c237e9d50b9b8e162b9c34171353a5ea05f4019c99c3e8ebc00722361cbc DEBUG Downloading source distribution: odfpy==1.4.1 DEBUG Downloading source distribution: fire==0.6.0 DEBUG Downloading source distribution: nptdms==1.9.0 DEBUG Acquired lock for `\\?\C:\Users\gitlabwinrunner\AppData\Local\Temp\.tmpanLS7H\built-wheels-v3\index\70e6dbc41e17fe7c\ipyvuetable\0.4.0` DEBUG No cache entry for: https://artifactory.acme.com/artifactory/api/pypi/pypi-acme-vir/packages/packages/7f/dc/aeb142c5ab3563a8600c8d2409212d9ff7f869ff87adea1e1de252032f08/ipyvuetable-0.4.0.tar.gz#sha256=d88e4d87be451b3813e18a999bd3bb5800db38ba59c612e943029fd0804482e2 DEBUG Downloading source distribution: ipyvuetable==0.4.0 DEBUG Downloading source distribution: pywinusb==0.4.2 DEBUG Preparing metadata for: ipyvuetable==0.4.0 DEBUG No static `PKG-INFO` available for: ipyvuetable==0.4.0 (PkgInfo(UnsupportedMetadataVersion("2.1"))) DEBUG No static `pyproject.toml` available for: ipyvuetable==0.4.0 (MissingPyprojectToml) INFO Ignoring empty directory DEBUG Solving with installed Python version: 3.11.9 DEBUG Adding direct dependency: setuptools>=40.8.0 DEBUG No cache entry for: https://artifactory.acme.com/artifactory/api/pypi/pypi-acme-vir/simple/setuptools/ DEBUG Preparing metadata for: fire==0.6.0 DEBUG No static `PKG-INFO` available for: fire==0.6.0 (PkgInfo(UnsupportedMetadataVersion("2.1"))) DEBUG No static `pyproject.toml` available for: fire==0.6.0 (MissingPyprojectToml) INFO Ignoring empty directory DEBUG Preparing metadata for: nptdms==1.9.0 DEBUG No static `PKG-INFO` available for: nptdms==1.9.0 (PkgInfo(UnsupportedMetadataVersion("2.1"))) DEBUG No static `pyproject.toml` available for: nptdms==1.9.0 (PyprojectToml(FieldNotFound("project"))) INFO Ignoring empty directory DEBUG Solving with installed Python version: 3.11.9 DEBUG Adding direct dependency: setuptools* error: Failed to download and build `pywinusb==0.4.2` Caused by: Failed to write to the distribution cache Caused by: failed to rename file from \\?\C:\Users\gitlabwinrunner\AppData\Local\Temp\.tmpanLS7H\built-wheels-v3\.tmpc3E2Z7\pywinusb-0.4.2 to \\?\C:\Users\gitlabwinrunner\AppData\Local\Temp\.tmpanLS7H\built-wheels-v3\index\70e6dbc41e17fe7c\pywinusb\0.4.2\or1SrhePMDCdKnzdyzMki\pywinusb-0.4.2.zip Caused by: Access is denied. (os error 5) ```

I also set RUST_LOG=trace and found only 1 or 2 instances of "Retrying rename from...." - but only during the initialization of the venv by hatch and not when uv installs the source-only packages.

EDIT: same is true for uv 0.2.17 (2eb1e6693 2024-06-26) (you guys are moving fast 😉 )

zanieb commented 4 months ago

Thanks for the details! I'll investigate.

zanieb commented 4 months ago

And I believe I've found a suspicious line :) if you want to give it a try https://github.com/astral-sh/uv/pull/4605

edit: and another suspicious call addressed in https://github.com/astral-sh/uv/pull/4606

thecityofguanyu commented 4 months ago

Just want to share that we still see these failures quite regularly as of uv 0.2.21 (ebfe6d8fc 2024-07-03)

[2024-07-13T08:03:24.537Z] error: Failed to download and build `conan==2.4.1`
[2024-07-13T08:03:24.537Z]   Caused by: Failed to write to the distribution cache
[2024-07-13T08:03:24.537Z]   Caused by: failed to rename file from \\?\D:\Jenkins\workspace\shared\temp\.tmp3Js7SG\built-wheels-v3\.tmpIcR0og\conan-2.4.1 to \\?\D:\Jenkins\workspace\shared\temp\.tmp3Js7SG\built-wheels-v3\index\f85683082b98592e\conan\2.4.1\rxFYsdKad4Hg7UOBh8zRC\conan-2.4.1.tar.gz
[2024-07-13T08:03:24.537Z]   Caused by: Access is denied. (os error 5)
omdaniel commented 4 months ago

Just want to share that we still see these failures quite regularly as of uv 0.2.21 (ebfe6d8fc 2024-07-03)

[2024-07-13T08:03:24.537Z] error: Failed to download and build `conan==2.4.1`
[2024-07-13T08:03:24.537Z]   Caused by: Failed to write to the distribution cache
[2024-07-13T08:03:24.537Z]   Caused by: failed to rename file from \\?\D:\Jenkins\workspace\shared\temp\.tmp3Js7SG\built-wheels-v3\.tmpIcR0og\conan-2.4.1 to \\?\D:\Jenkins\workspace\shared\temp\.tmp3Js7SG\built-wheels-v3\index\f85683082b98592e\conan\2.4.1\rxFYsdKad4Hg7UOBh8zRC\conan-2.4.1.tar.gz
[2024-07-13T08:03:24.537Z]   Caused by: Access is denied. (os error 5)

I wonder in this case if the back off procedure is being used in the case where a temp build environment is being created to build a wheel for a package in isolation. This seems like in the case where an isolated build is needed the back off approach is not being used.

I have on different issue related to the temp build directory tries to run executables where IT blocks any user executable not under "Program Files", and I couldn't figure out how to override the location were the temp build environment is created (still haven't figured it out)

But that issue gave me insight into what may be going on here which may be a control flow were the back off is not being applied when trying to copy the compressed package to the archive

thecityofguanyu commented 4 months ago

I recorded a failure through Process Monitor in case any of that information is useful here. I can't share it directly (because it came from a corporate environment), but I can redact and provide pieces. Just noting in case this for whatever reason cannot be replicated by the codeowners.

One item of note is that I don't think I see a retry? All I can find is a single SetRenameInformationFile with a corresponding ACCESS DENIED. This was from testing with uv 0.2.23 (4bc36c0cb 2024-07-08).

11:27:28.2285886 AM uv.exe  11604   SetRenameInformationFile    C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0    ACCESS DENIED   ReplaceIfExists: True, FileName: C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\index\f85683082b98592e\conan\2.5.0\Ac6VfyuHm-z0Y9-KNpzw5\conan-2.5.0.tar.gz

The command run to generate above was uv pip install -r requirements.txt --reinstall --no-cache.

Our CI runs with --vvv ever since first encountering this error. Anecdotally (logs aren't in front of me), it seemed like it was previously going through a handful of retries before sometimes failing. Now I don't see it doing that anymore.

charliermarsh commented 4 months ago

Yeah my guess is we're missing it somewhere but not sure where... I think @zanieb is planning to take a second look.

thecityofguanyu commented 4 months ago

Looks like @zanieb already pushed a PR and may have found a fix, but should it be useful, here's the IO activity of the thread from https://github.com/astral-sh/uv/issues/1491#issuecomment-2228830850

11:27:27.3794232 AM uv.exe  11604   WriteFile   C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conan\api\subapi\remove.py SUCCESS Offset: 0, Length: 2,176, Priority: Normal
11:27:27.4956009 AM uv.exe  11604   WriteFile   C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conan\cli\commands\download.py SUCCESS Offset: 0, Length: 3,256, Priority: Normal
11:27:27.5824242 AM uv.exe  11604   CreateFile  C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conan\cli\commands\search.py   SUCCESS Desired Access: Generic Write, Read Attributes, Disposition: Create, Options: Synchronous IO Non-Alert, Non-Directory File, Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: 0, OpenResult: Created
11:27:27.5828526 AM uv.exe  11604   QueryNameInformationFile    C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conan\cli\commands\search.py   BUFFER OVERFLOW Name: \Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conan
11:27:27.5828966 AM uv.exe  11604   QueryNameInformationFile    C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conan\cli\commands\search.py   SUCCESS Name: \Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conan\cli\commands\search.py
11:27:27.5829343 AM uv.exe  11604   QueryBasicInformationFile   C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conan\cli\commands\search.py   SUCCESS CreationTime: 7/15/2024 11:27:27 AM, LastAccessTime: 7/15/2024 11:27:27 AM, LastWriteTime: 7/15/2024 11:27:27 AM, ChangeTime: 7/15/2024 11:27:27 AM, FileAttributes: A
11:27:27.6779132 AM uv.exe  11604   WriteFile   C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conan\internal\cache\conan_reference_layout.py SUCCESS Offset: 0, Length: 3,802, Priority: Normal
11:27:27.7193186 AM uv.exe  11604   WriteFile   C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conan\test\assets\genconanfile.py  SUCCESS Offset: 0, Length: 8,192, Priority: Normal
11:27:27.7570835 AM uv.exe  11604   WriteFile   C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conan\tools\apple\xcodedeps.py FAST IO DISALLOWED  Offset: 8,192, Length: 8,047
11:27:27.7571054 AM uv.exe  11604   WriteFile   C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conan\tools\apple\xcodedeps.py SUCCESS Offset: 8,192, Length: 8,047, Priority: Normal
11:27:27.7995744 AM uv.exe  11604   WriteFile   C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conan\tools\cmake\cmakedeps\templates\target_data.py   SUCCESS Offset: 16,384, Length: 4,231
11:27:27.8362180 AM uv.exe  11604   WriteFile   C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conan\tools\files\packager.py  SUCCESS Offset: 0, Length: 4,177, Priority: Normal
11:27:27.8763317 AM uv.exe  11604   WriteFile   C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conan\tools\google\toolchain.py    SUCCESS Offset: 0, Length: 8,113, Priority: Normal
11:27:27.8764108 AM uv.exe  11604   ReadFile    C:\$Extend\$UsnJrnl:$J:$DATA    SUCCESS Offset: 41,971,712, Length: 88, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal
11:27:27.9159855 AM uv.exe  11604   WriteFile   C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conan\tools\premake\premakedeps.py SUCCESS Offset: 0, Length: 8,192, Priority: Normal
11:27:27.9610159 AM uv.exe  11604   CreateFile  C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conans\client\cache\__init__.py    SUCCESS Desired Access: Generic Write, Read Attributes, Disposition: Create, Options: Synchronous IO Non-Alert, Non-Directory File, Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: 0, OpenResult: Created
11:27:27.9615439 AM uv.exe  11604   QueryNameInformationFile    C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conans\client\cache\__init__.py    BUFFER OVERFLOW Name: \Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conan
11:27:27.9615946 AM uv.exe  11604   QueryNameInformationFile    C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conans\client\cache\__init__.py    SUCCESS Name: \Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conans\client\cache\__init__.py
11:27:27.9616186 AM uv.exe  11604   QueryBasicInformationFile   C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conans\client\cache\__init__.py    SUCCESS CreationTime: 7/15/2024 11:27:27 AM, LastAccessTime: 7/15/2024 11:27:27 AM, LastWriteTime: 7/15/2024 11:27:27 AM, ChangeTime: 7/15/2024 11:27:27 AM, FileAttributes: A
11:27:28.0134359 AM uv.exe  11604   CreateFile  C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conans\client\graph\compute_pid.py SUCCESS Desired Access: Generic Write, Read Attributes, Disposition: Create, Options: Synchronous IO Non-Alert, Non-Directory File, Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: 0, OpenResult: Created
11:27:28.0140592 AM uv.exe  11604   QueryNameInformationFile    C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conans\client\graph\compute_pid.py BUFFER OVERFLOW Name: \Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conan
11:27:28.0141104 AM uv.exe  11604   QueryNameInformationFile    C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conans\client\graph\compute_pid.py SUCCESS Name: \Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conans\client\graph\compute_pid.py
11:27:28.0141272 AM uv.exe  11604   QueryBasicInformationFile   C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conans\client\graph\compute_pid.py SUCCESS CreationTime: 7/15/2024 11:27:28 AM, LastAccessTime: 7/15/2024 11:27:28 AM, LastWriteTime: 7/15/2024 11:27:28 AM, ChangeTime: 7/15/2024 11:27:28 AM, FileAttributes: A
11:27:28.0494167 AM uv.exe  11604   WriteFile   C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conans\client\remote_manager.py    SUCCESS Offset: 0, Length: 8,192, Priority: Normal
11:27:28.1010791 AM uv.exe  11604   CreateFile  C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conans\model   SUCCESS Desired Access: Read Data/List Directory, Synchronize, Disposition: Create, Options: Directory, Synchronous IO Non-Alert, Open Reparse Point, Attributes: N, ShareMode: Read, Write, AllocationSize: 0, OpenResult: Created
11:27:28.1013980 AM uv.exe  11604   QueryNameInformationFile    C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conans\model   BUFFER OVERFLOW Name: \Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conan
11:27:28.1014297 AM uv.exe  11604   QueryNameInformationFile    C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conans\model   SUCCESS Name: \Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conans\model
11:27:28.1014421 AM uv.exe  11604   QueryBasicInformationFile   C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conans\model   SUCCESS CreationTime: 7/15/2024 11:27:28 AM, LastAccessTime: 7/15/2024 11:27:28 AM, LastWriteTime: 7/15/2024 11:27:28 AM, ChangeTime: 7/15/2024 11:27:28 AM, FileAttributes: D
11:27:28.1014725 AM uv.exe  11604   CloseFile   C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conans\model   SUCCESS 
11:27:28.1015137 AM uv.exe  11604   IRP_MJ_CLOSE    C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conans\model   SUCCESS 
11:27:28.1323508 AM uv.exe  11604   WriteFile   C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conans\model\profile.py    FAST IO DISALLOWED  Offset: 8,192, Length: 62
11:27:28.1323753 AM uv.exe  11604   WriteFile   C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0\conans\model\profile.py    SUCCESS Offset: 8,192, Length: 62, Priority: Normal
11:27:28.2281959 AM uv.exe  11604   CreateFile  C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0    SUCCESS Desired Access: Read Attributes, Delete, Synchronize, Disposition: Open, Options: Synchronous IO Non-Alert, Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened
11:27:28.2283100 AM uv.exe  11604   QueryAttributeTagFile   C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0    SUCCESS Attributes: D, ReparseTag: 0x0
11:27:28.2283283 AM uv.exe  11604   QueryBasicInformationFile   C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0    SUCCESS CreationTime: 7/15/2024 11:27:27 AM, LastAccessTime: 7/15/2024 11:27:28 AM, LastWriteTime: 7/15/2024 11:27:28 AM, ChangeTime: 7/15/2024 11:27:28 AM, FileAttributes: D
11:27:28.2284142 AM uv.exe  11604   CreateFile  C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\index\f85683082b98592e\conan\2.5.0\Ac6VfyuHm-z0Y9-KNpzw5  SUCCESS Desired Access: Append Data/Add Subdirectory/Create Pipe Instance, Synchronize, Disposition: Open, Options: , Attributes: n/a, ShareMode: Read, Write, AllocationSize: n/a, OpenResult: Opened
11:27:28.2285886 AM uv.exe  11604   SetRenameInformationFile    C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0    ACCESS DENIED   ReplaceIfExists: True, FileName: C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\index\f85683082b98592e\conan\2.5.0\Ac6VfyuHm-z0Y9-KNpzw5\conan-2.5.0.tar.gz
11:27:28.2288963 AM uv.exe  11604   CloseFile   C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\index\f85683082b98592e\conan\2.5.0\Ac6VfyuHm-z0Y9-KNpzw5  SUCCESS 
11:27:28.2289789 AM uv.exe  11604   IRP_MJ_CLOSE    C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\index\f85683082b98592e\conan\2.5.0\Ac6VfyuHm-z0Y9-KNpzw5  SUCCESS 
11:27:28.2290180 AM uv.exe  11604   CloseFile   C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0    SUCCESS 
11:27:28.2290773 AM uv.exe  11604   IRP_MJ_CLOSE    C:\Users\username\AppData\Local\Temp\2\.tmpcUIakH\built-wheels-v3\.tmpQYLAdW\conan-2.5.0    SUCCESS 
11:27:29.1293378 AM uv.exe  11604   Thread Exit     SUCCESS Thread ID: 10608, User Time: 0.0156250, Kernel Time: 0.0312500
charliermarsh commented 4 months ago

Ok more fixes coming in the next release.

axel-kah commented 3 months ago

Ok more fixes coming in the next release.

I stress-tested v0.2.25 on our windows CI by focussing on source distributions only. They all passed and instead of failing because files in the cache can't be renamed, I finally see retrying warnings like these:

WARN Retrying rename from \\?\C:\Users\gitlabwinrunner\AppData\Local\Temp\.tmpTFyAcI\built-wheels-v3\.tmpFqzuFu\pywinusb-0.4.2 
to \\?\C:\Users\gitlabwinrunner\AppData\Local\Temp\.tmpTFyAcI\built-wheels-v3\index\70e6dbc41e17fe7c\pywinusb\0.4.2\seIYCJmH_UpZwtxYdRGAe\pywinusb-0.4.2.zip due to transient error: 
failed to rename file from \\?\C:\Users\gitlabwinrunner\AppData\Local\Temp\.tmpTFyAcI\built-wheels-v3\.tmpFqzuFu\pywinusb-0.4.2 
to \\?\C:\Users\gitlabwinrunner\AppData\Local\Temp\.tmpTFyAcI\built-wheels-v3\index\70e6dbc41e17fe7c\pywinusb\0.4.2\seIYCJmH_UpZwtxYdRGAe\pywinusb-0.4.2.zip

🎉 Thanks a lot for your effort!

zanieb commented 3 months ago

Wonderful! Thanks for following up <3

paveldikov commented 3 months ago

Clearly separate from the root cause here but since it's mentioned here:

I'm open to changing to use the dunce-canonicalized version everywhere, but note that it doesn't, like, fully solve the problem (if it is a problem). dunce will still return a UNC prefix if the path is longer than 260 characters.

@charliermarsh are you still open to this change? We're hitting this problem with pyvenv.cfg (the home path in particular) -- importing dynamic libraries from UNC-prefixed paths fails miserably for at least some Python versions. Happy to open another issue.

charliermarsh commented 3 months ago

Do you mind opening a separate issue with more details? That sounds like a bug (as opposed to a spurious failure).On Jul 24, 2024, at 7:46 AM, Pavel Dikov @.***> wrote: Clearly separate from the root cause here but since it's mentioned here:

I'm open to changing to use the dunce-canonicalized version everywhere, but note that it doesn't, like, fully solve the problem (if it is a problem). dunce will still return a UNC prefix if the path is longer than 260 characters.

@charliermarsh are you still open to this change? We're hitting this problem with pyvenv.cfg (the home path in particular) -- importing dynamic libraries from UNC-prefixed paths fails miserably for at least some Python versions. Happy to open another issue.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

Super1Windcloud commented 1 month ago

There is no doubt that this is a problem caused by hard links, and pnpm often has various problems