astral-sh / uv

An extremely fast Python package and project manager, written in Rust.
https://docs.astral.sh/uv
Apache License 2.0
25.77k stars 750 forks source link

uv pip install inconsistent failure on Windows #1491

Closed Wofiel closed 3 months ago

Wofiel commented 8 months ago

When using uv pip install -r requirements.txt, I will frequently get the following error:

error: Failed to download distributions
  Caused by: Failed to fetch wheel: setproctitle==1.3.3
  Caused by: Failed to read from the distribution cache
  Caused by: failed to rename file from \\?\C:\Users\[____]\AppData\Local\Temp\1\.tmp56ZHLz\.tmpMXOwQn to \\?\C:\Users\[____]\AppData\Local\Temp\1\.tmp56ZHLz\archive-v0\7RXhEx7TcTNKRTlP3vSED
  Caused by: Access is denied. (os error 5)

This appears to be independent of the package, though seemingly happens more often with some than others.

If run with --no-cache, it's almost impossible to install, as at least one thing is likely to fail in the process and progress restarted.

If run without --no-cache, retrying a number of times will eventually succeed (however, it will leave .tmp[_______] folders listed in the error messages in C:\Users\[____]\AppData\Local\uv\cache).

AucaCoyan commented 8 months ago

I did a test on my windows pc and debian on a VM, and did experiment the same results as the issue. On Windows 10, using nushell

"distributions=2.2.1" | save requirements.txt
uv venv
uv pip install -r requirements.txt

gives

error: Failed to download and build: distributions==2.2.1
  Caused by: Failed to build: distributions==2.2.1
  Caused by: Build backend failed to determine metadata through `prepare_metadata_for_build_wheel`:
--- stdout:

--- stderr:
<string>:177: SyntaxWarning: invalid escape sequence '\S'
Traceback (most recent call last):
  File "<string>", line 10, in <module>
  File "C:\Users\aucac\AppData\Local\Temp\.tmpn2KafH\.venv\Lib\site-packages\setuptools\build_meta.py", line 366, in prepare_metadata_for_build_wheel
    self.run_setup()
  File "C:\Users\aucac\AppData\Local\Temp\.tmpn2KafH\.venv\Lib\site-packages\setuptools\build_meta.py", line 480, in run_setup
    super().run_setup(setup_script=setup_script)
  File "C:\Users\aucac\AppData\Local\Temp\.tmpn2KafH\.venv\Lib\site-packages\setuptools\build_meta.py", line 311, in run_setup
    exec(code, locals())
  File "<string>", line 31, in <module>
ModuleNotFoundError: No module named 'numpy'
---

and in Debian is the same (give or take the paths and other stuff).

Then I compared with pip

Collecting distributions==2.2.1 (from -r requirements.txt (line 1))
  Downloading distributions-2.2.1.tar.gz (1.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.5/1.5 MB 6.1 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [21 lines of output]
      <string>:177: SyntaxWarning: invalid escape sequence '\S'
      Traceback (most recent call last):
        File "C:\Users\aucac\scoop\apps\python\current\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
          main()
        File "C:\Users\aucac\scoop\apps\python\current\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\aucac\scoop\apps\python\current\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\aucac\AppData\Local\Temp\pip-build-env-z4nk3ptj\overlay\Lib\site-packages\setuptools\build_meta.py", line 325, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\aucac\AppData\Local\Temp\pip-build-env-z4nk3ptj\overlay\Lib\site-packages\setuptools\build_meta.py", line 295, in _get_build_requires
          self.run_setup()
        File "C:\Users\aucac\AppData\Local\Temp\pip-build-env-z4nk3ptj\overlay\Lib\site-packages\setuptools\build_meta.py", line 480, in run_setup
          super().run_setup(setup_script=setup_script)
        File "C:\Users\aucac\AppData\Local\Temp\pip-build-env-z4nk3ptj\overlay\Lib\site-packages\setuptools\build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 31, in <module>
      ModuleNotFoundError: No module named 'numpy'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

[notice] A new release of pip is available: 23.2.1 -> 24.0
[notice] To update, run: python.exe -m pip install --upgrade pip

And have around the same result. Also, distributions is not updated 2017 Can you copy and paste your requirements.txt dependencies?

Wofiel commented 8 months ago

That appears to be a different error. This isn't about the particular package distributions. This appears to be very specifically just an error somewhere in the renaming of folders.

Just tried using a requirements file off the shelf: https://github.com/google-research/kubric/blob/main/requirements.txt

I've added the full log including retries this time, showing how it's not always the same package, and retries will eventually succeed. (Also generating 300+MB of trash in the leftover .tmp[______] folders in C:\Users\[____]\AppData\Local\uv\cache)

Subsequent installs are fine after the first install (and fast!), as everything is already in the cache.

Contrasted with regular Python pip, which installs first try, with or without cache.

PS C:\dev\uv_test_uv> uv venv; .\.venv\Scripts\activate.ps1; uv pip install -r requirements.txt
Using Python 3.11.6 interpreter at C:\dev\uv_test_uv\.venv\Scripts\python.exe
Creating virtualenv at: .venv
Resolved 136 packages in 31.02s
error: Failed to download distributions
  Caused by: Failed to fetch wheel: regex==2023.12.25
  Caused by: Failed to read from the distribution cache
  Caused by: failed to rename file from \\?\C:\Users\[____]\AppData\Local\uv\cache\.tmpXU7l5Y to \\?\C:\Users\[____]\AppData\Local\uv\cache\archive-v0\IPIdPmnxfhPlAbb4aByib
  Caused by: Access is denied. (os error 5)
(.venv) PS C:\dev\uv_test_uv> uv pip install -r requirements.txt
Resolved 136 packages in 216ms
   Built dill==0.3.1.1
error: Failed to download distributions
  Caused by: Failed to fetch wheel: fastavro==1.9.4
  Caused by: Failed to read from the distribution cache
  Caused by: failed to rename file from \\?\C:\Users\[____]\AppData\Local\uv\cache\.tmpvibAxB to \\?\C:\Users\[____]\AppData\Local\uv\cache\archive-v0\xESJmy5hSHMPWyOAki-mV
  Caused by: Access is denied. (os error 5)
(.venv) PS C:\dev\uv_test_uv> uv pip install -r requirements.txt
Resolved 136 packages in 202ms
   Built crcmod==1.7
error: Failed to download distributions
  Caused by: Failed to unzip wheel: crcmod==1.7
  Caused by: failed to rename file from \\?\C:\Users\[____]\AppData\Local\uv\cache\.tmpuSxG40 to \\?\C:\Users\[____]\AppData\Local\uv\cache\archive-v0\8seE9kMwlOJb0FO7kkUiO
  Caused by: Access is denied. (os error 5)
(.venv) PS C:\dev\uv_test_uv> uv pip install -r requirements.txt
Resolved 136 packages in 168ms
   Built hdfs==2.7.3
   Built docopt==0.6.2
   Built promise==2.3
   Built pyjsparser==2.7.1
   Built cloudml-hypertune==0.1.0.dev6
error: Failed to download distributions
  Caused by: Failed to fetch wheel: rapidfuzz==3.6.1
  Caused by: Failed to read from the distribution cache
  Caused by: failed to rename file from \\?\C:\Users\[____]\AppData\Local\uv\cache\.tmpnchaw9 to \\?\C:\Users\[____]\AppData\Local\uv\cache\archive-v0\fslC-i4Ih5c8itkG3OCU1
  Caused by: Access is denied. (os error 5)
(.venv) PS C:\dev\uv_test_uv> uv pip install -r requirements.txt
Resolved 136 packages in 199ms
   Built google-apitools==0.5.31
error: Failed to download distributions
  Caused by: Failed to fetch wheel: scikit-learn==1.4.1.post1
  Caused by: Failed to read from the distribution cache
  Caused by: failed to rename file from \\?\C:\Users\[____]\AppData\Local\uv\cache\.tmpyPARfj to \\?\C:\Users\[____]\AppData\Local\uv\cache\archive-v0\BTqIbBmeNiUJZ4iFLjvr5
  Caused by: Access is denied. (os error 5)
(.venv) PS C:\dev\uv_test_uv> uv pip install -r requirements.txt
Resolved 136 packages in 187ms
error: Failed to download distributions
  Caused by: Failed to fetch wheel: scikit-learn==1.4.1.post1
  Caused by: Failed to read from the distribution cache
  Caused by: failed to rename file from \\?\C:\Users\[____]\AppData\Local\uv\cache\.tmpQ9zLLc to \\?\C:\Users\[____]\AppData\Local\uv\cache\archive-v0\N6_d31F0MmeWfay8zuoy1
  Caused by: Access is denied. (os error 5)
(.venv) PS C:\dev\uv_test_uv> uv pip install -r requirements.txt
Resolved 136 packages in 198ms
error: Failed to download distributions
  Caused by: Failed to fetch wheel: pyarrow==11.0.0
  Caused by: Failed to read from the distribution cache
  Caused by: failed to rename file from \\?\C:\Users\[____]\AppData\Local\uv\cache\.tmpoDjYqG to \\?\C:\Users\[____]\AppData\Local\uv\cache\archive-v0\77wbDpLZdJXaCe7d8Bgl4
  Caused by: Access is denied. (os error 5)
(.venv) PS C:\dev\uv_test_uv> uv pip install -r requirements.txt
Resolved 136 packages in 180ms
error: Failed to download distributions
  Caused by: Failed to fetch wheel: scipy==1.12.0
  Caused by: Failed to read from the distribution cache
  Caused by: failed to rename file from \\?\C:\Users\[____]\AppData\Local\uv\cache\.tmpJrQ1Eo to \\?\C:\Users\[____]\AppData\Local\uv\cache\archive-v0\oJCgnzumPreaL33Y2A_jb
  Caused by: Access is denied. (os error 5)
(.venv) PS C:\dev\uv_test_uv> uv pip install -r requirements.txt
Resolved 136 packages in 193ms
Downloaded 3 packages in 1m 30s
Installed 136 packages in 41.33s
 + absl-py==1.4.0
 + apache-beam==2.54.0
 + astunparse==1.6.3
 + attrs==23.2.0
 + bidict==0.23.0
 + cachetools==5.3.2
 + certifi==2024.2.2
 + charset-normalizer==3.3.2
 + click==8.1.7
 + cloudml-hypertune==0.1.0.dev6
 + cloudpickle==2.2.1
 + colorama==0.4.6
 + crcmod==1.7
 + dataclasses==0.6
 + deprecated==1.2.14
 + dill==0.3.1.1
 + dm-tree==0.1.8
 + dnspython==2.5.0
 + docopt==0.6.2
 + etils==1.7.0
 + fastavro==1.9.4
 + fasteners==0.19
 + flatbuffers==23.5.26
 + fsspec==2024.2.0
 + gast==0.5.4
 + google-api-core==2.17.1
 + google-apitools==0.5.31
 + google-auth==2.28.0
 + google-auth-httplib2==0.1.1
 + google-auth-oauthlib==1.2.0
 + google-cloud-aiplatform==1.42.0
 + google-cloud-bigquery==3.17.2
 + google-cloud-bigquery-storage==2.24.0
 + google-cloud-bigtable==2.23.0
 + google-cloud-core==2.4.1
 + google-cloud-datastore==2.19.0
 + google-cloud-dlp==3.15.1
 + google-cloud-language==2.13.1
 + google-cloud-pubsub==2.19.4
 + google-cloud-pubsublite==1.9.0
 + google-cloud-recommendations-ai==0.10.8
 + google-cloud-resource-manager==1.12.1
 + google-cloud-spanner==3.42.0
 + google-cloud-storage==2.14.0
 + google-cloud-videointelligence==2.13.1
 + google-cloud-vision==3.7.0
 + google-crc32c==1.5.0
 + google-pasta==0.2.0
 + google-resumable-media==2.7.0
 + googleapis-common-protos==1.62.0
 + grpc-google-iam-v1==0.13.0
 + grpc-interceptor==0.15.4
 + grpcio==1.60.1
 + grpcio-status==1.60.1
 + h5py==3.10.0
 + hdfs==2.7.3
 + httplib2==0.22.0
 + idna==3.6
 + imageio==2.34.0
 + importlib-resources==6.1.1
 + joblib==1.3.2
 + js2py==0.74
 + jsonpickle==3.0.2
 + jsonschema==4.21.1
 + jsonschema-specifications==2023.12.1
 + keras==2.15.0
 + levenshtein==0.25.0
 + libclang==16.0.6
 + markdown==3.5.2
 + markupsafe==2.1.5
 + ml-dtypes==0.2.0
 + munch==4.0.0
 + numpy==1.24.4
 + oauth2client==4.1.3
 + oauthlib==3.2.2
 + objsize==0.7.0
 + opt-einsum==3.3.0
 + orjson==3.9.14
 + overrides==7.7.0
 + packaging==23.2
 + pandas==2.2.0
 + pillow==10.2.0
 + promise==2.3
 + proto-plus==1.23.0
 + protobuf==4.25.3
 + psutil==5.9.8
 + pyarrow==11.0.0
 + pyarrow-hotfix==0.6
 + pyasn1==0.5.1
 + pyasn1-modules==0.3.0
 + pydot==1.4.2
 + pyjsparser==2.7.1
 + pymongo==4.6.1
 + pyparsing==3.1.1
 + pypng==0.20220715.0
 + pyquaternion==0.9.9
 + python-dateutil==2.8.2
 + python-levenshtein==0.25.0
 + pytz==2024.1
 + rapidfuzz==3.6.1
 + referencing==0.33.0
 + regex==2023.12.25
 + requests==2.31.0
 + requests-oauthlib==1.3.1
 + rpds-py==0.18.0
 + rsa==4.9
 + scikit-learn==1.4.1.post1
 + scipy==1.12.0
 + setuptools==69.1.0
 + shapely==2.0.3
 + singledispatchmethod==1.0
 + six==1.16.0
 + sqlparse==0.4.4
 + tensorboard==2.15.2
 + tensorboard-data-server==0.7.2
 + tensorflow==2.15.0
 + tensorflow-datasets==4.9.4
 + tensorflow-estimator==2.15.0
 + tensorflow-intel==2.15.0
 + tensorflow-io-gcs-filesystem==0.31.0
 + tensorflow-metadata==1.13.1
 + termcolor==2.4.0
 + threadpoolctl==3.3.0
 + toml==0.10.2
 + tqdm==4.66.2
 + traitlets==5.14.1
 + trimesh==4.1.3
 + typing-extensions==4.9.0
 + tzdata==2024.1
 + tzlocal==5.2
 + urllib3==2.2.0
(.venv) PS C:\dev\uv_test_uv>
AucaCoyan commented 8 months ago

I see, thanks for the log! Again, I have a different output sadly 😢. I think I am doing something wrong too.

In my case, it didn't found a compatible version of tensorflow. If you are interested, here I leave the details, but I think my case is unrelated to this issue.

```bash (.venv) PS C:\Users\aucac\repos\exp-python\uv> uv pip install -r .\requirements.txt × No solution found when resolving dependencies: ╰─▶ Because only the following versions of tensorflow are available: tensorflow<=2.5.3 tensorflow>=2.6.0,<=2.6.5 tensorflow>=2.7.0,<=2.7.4 tensorflow>=2.8.0,<=2.8.4 tensorflow>=2.9.0,<=2.9.3 tensorflow>=2.10.0,<=2.10.1 tensorflow>=2.11.0,<=2.11.1 tensorflow>=2.12.0,<=2.12.1 tensorflow>=2.13.0,<=2.13.1 tensorflow>=2.14.0,<=2.14.1 tensorflow>=2.15.0 and tensorflow==0.12.0 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<0.12.1 tensorflow>2.5.3,<2.6.0 tensorflow>2.6.5,<2.7.0 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==0.12.1 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<1.0.0 tensorflow>2.5.3,<2.6.0 tensorflow>2.6.5,<2.7.0 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==1.0.0 is unusable because no wheels are available with a matching Python ABI and tensorflow==1.0.1 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<1.1.0 tensorflow>2.5.3,<2.6.0 tensorflow>2.6.5,<2.7.0 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==1.1.0 is unusable because no wheels are available with a matching Python ABI and tensorflow==1.2.0 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<1.2.1 tensorflow>2.5.3,<2.6.0 tensorflow>2.6.5,<2.7.0 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==1.2.1 is unusable because no wheels are available with a matching Python ABI and tensorflow==1.3.0 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<1.4.0 tensorflow>2.5.3,<2.6.0 tensorflow>2.6.5,<2.7.0 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==1.4.0 is unusable because no wheels are available with a matching Python ABI and tensorflow==1.4.1 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<1.5.0 tensorflow>2.5.3,<2.6.0 tensorflow>2.6.5,<2.7.0 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==1.5.0 is unusable because no wheels are available with a matching Python ABI and tensorflow==1.5.1 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<1.6.0 tensorflow>2.5.3,<2.6.0 tensorflow>2.6.5,<2.7.0 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==1.6.0 is unusable because no wheels are available with a matching Python ABI and tensorflow==1.7.0 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<1.7.1 tensorflow>2.5.3,<2.6.0 tensorflow>2.6.5,<2.7.0 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==1.7.1 is unusable because no wheels are available with a matching Python ABI and tensorflow==1.8.0 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<1.9.0 tensorflow>2.5.3,<2.6.0 tensorflow>2.6.5,<2.7.0 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==1.9.0 is unusable because no wheels are available with a matching Python ABI and tensorflow==1.10.0 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<1.10.1 tensorflow>2.5.3,<2.6.0 tensorflow>2.6.5,<2.7.0 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==1.10.1 is unusable because no wheels are available with a matching Python ABI and tensorflow==1.11.0 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<1.12.0 tensorflow>2.5.3,<2.6.0 tensorflow>2.6.5,<2.7.0 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==1.12.0 is unusable because no wheels are available with a matching Python ABI and tensorflow==1.12.2 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<1.12.3 tensorflow>2.5.3,<2.6.0 tensorflow>2.6.5,<2.7.0 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==1.12.3 is unusable because no wheels are available with a matching Python ABI and tensorflow==1.13.1 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<1.13.2 tensorflow>2.5.3,<2.6.0 tensorflow>2.6.5,<2.7.0 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==1.13.2 is unusable because no wheels are available with a matching Python ABI and tensorflow==1.14.0 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<1.15.0 tensorflow>2.5.3,<2.6.0 tensorflow>2.6.5,<2.7.0 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==1.15.0 is unusable because no wheels are available with a matching Python ABI and tensorflow==1.15.2 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<1.15.3 tensorflow>2.5.3,<2.6.0 tensorflow>2.6.5,<2.7.0 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==1.15.3 is unusable because no wheels are available with a matching Python ABI and tensorflow==1.15.4 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<1.15.5 tensorflow>2.5.3,<2.6.0 tensorflow>2.6.5,<2.7.0 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==1.15.5 is unusable because no wheels are available with a matching Python ABI and tensorflow==2.0.0 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<2.0.1 tensorflow>2.5.3,<2.6.0 tensorflow>2.6.5,<2.7.0 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==2.0.1 is unusable because no wheels are available with a matching Python ABI and tensorflow==2.0.2 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<2.0.3 tensorflow>2.5.3,<2.6.0 tensorflow>2.6.5,<2.7.0 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==2.0.3 is unusable because no wheels are available with a matching Python ABI and tensorflow==2.0.4 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<2.1.0 tensorflow>2.5.3,<2.6.0 tensorflow>2.6.5,<2.7.0 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==2.1.0 is unusable because no wheels are available with a matching Python ABI and tensorflow==2.1.1 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<2.1.2 tensorflow>2.5.3,<2.6.0 tensorflow>2.6.5,<2.7.0 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==2.1.2 is unusable because no wheels are available with a matching Python ABI and tensorflow==2.1.3 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<2.1.4 tensorflow>2.5.3,<2.6.0 tensorflow>2.6.5,<2.7.0 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==2.1.4 is unusable because no wheels are available with a matching Python ABI and tensorflow==2.2.0 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<2.2.1 tensorflow>2.5.3,<2.6.0 tensorflow>2.6.5,<2.7.0 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==2.2.1 is unusable because no wheels are available with a matching Python ABI and tensorflow==2.2.2 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<2.2.3 tensorflow>2.5.3,<2.6.0 tensorflow>2.6.5,<2.7.0 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==2.2.3 is unusable because no wheels are available with a matching Python ABI and tensorflow==2.3.0 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<2.3.1 tensorflow>2.5.3,<2.6.0 tensorflow>2.6.5,<2.7.0 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==2.3.1 is unusable because no wheels are available with a matching Python ABI and tensorflow==2.3.2 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<2.3.3 tensorflow>2.5.3,<2.6.0 tensorflow>2.6.5,<2.7.0 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==2.3.3 is unusable because no wheels are available with a matching Python ABI and tensorflow==2.3.4 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<2.4.0 tensorflow>2.5.3,<2.6.0 tensorflow>2.6.5,<2.7.0 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==2.4.0 is unusable because no wheels are available with a matching Python ABI and tensorflow==2.4.1 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<2.4.2 tensorflow>2.5.3,<2.6.0 tensorflow>2.6.5,<2.7.0 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==2.4.2 is unusable because no wheels are available with a matching Python ABI and tensorflow==2.4.3 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<2.4.4 tensorflow>2.5.3,<2.6.0 tensorflow>2.6.5,<2.7.0 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==2.4.4 is unusable because no wheels are available with a matching Python ABI and tensorflow==2.5.0 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<2.5.1 tensorflow>2.5.3,<2.6.0 tensorflow>2.6.5,<2.7.0 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==2.5.1 is unusable because no wheels are available with a matching Python ABI and tensorflow==2.5.2 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<2.5.3 tensorflow>2.5.3,<2.6.0 tensorflow>2.6.5,<2.7.0 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==2.5.3 is unusable because no wheels are available with a matching Python ABI and tensorflow==2.6.0 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<2.6.1 tensorflow>2.6.5,<2.7.0 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==2.6.1 is unusable because no wheels are available with a matching Python ABI and tensorflow==2.6.2 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<2.6.3 tensorflow>2.6.5,<2.7.0 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==2.6.3 is unusable because no wheels are available with a matching Python ABI and tensorflow==2.6.4 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<2.6.5 tensorflow>2.6.5,<2.7.0 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==2.6.5 is unusable because no wheels are available with a matching Python ABI and tensorflow==2.7.0 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<2.7.1 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==2.7.1 is unusable because no wheels are available with a matching Python ABI and tensorflow==2.7.2 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<2.7.3 tensorflow>2.7.4,<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==2.7.3 is unusable because no wheels are available with a matching Python ABI and tensorflow==2.7.4 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<2.8.0 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==2.8.0 is unusable because no wheels are available with a matching Python ABI and tensorflow==2.8.1 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<2.8.2 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==2.8.2 is unusable because no wheels are available with a matching Python ABI and tensorflow==2.8.3 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<2.8.4 tensorflow>2.8.4,<2.9.0 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==2.8.4 is unusable because no wheels are available with a matching Python ABI and tensorflow==2.9.0 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<2.9.1 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==2.9.1 is unusable because no wheels are available with a matching Python ABI and tensorflow==2.9.2 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<2.9.3 tensorflow>2.9.3,<2.10.0 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==2.9.3 is unusable because no wheels are available with a matching Python ABI and tensorflow==2.10.0 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<2.10.1 tensorflow>2.10.1,<2.11.0 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==2.10.1 is unusable because no wheels are available with a matching Python ABI and tensorflow==2.11.0 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<2.11.1 tensorflow>2.11.1,<2.12.0 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==2.11.1 is unusable because no wheels are available with a matching Python ABI and tensorflow==2.12.0 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<2.12.1 tensorflow>2.12.1,<2.13.0 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==2.12.1 is unusable because no wheels are available with a matching Python ABI and tensorflow==2.13.0 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<2.13.1 tensorflow>2.13.1,<2.14.0 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==2.13.1 is unusable because no wheels are available with a matching Python ABI and tensorflow==2.14.0 is unusable because no wheels are available with a matching Python ABI, we can conclude that any of: tensorflow<2.14.1 tensorflow>2.14.1,<2.15.0 cannot be used. And because tensorflow==2.14.1 is unusable because no wheels are available with a matching Python ABI and tensorflow==2.15.0 is unusable because no wheels are available with a matching Python ABI, we can conclude that tensorflow<2.15.0.post1 cannot be used. And because tensorflow==2.15.0.post1 is unusable because no wheels are available with a matching Python ABI and you require tensorflow, we can conclude that the requirements are unsatisfiable. hint: Pre-releases are available for tensorflow in the requested range (e.g., 2.15.0rc1), but pre-releases weren't enabled (try: `--prerelease=allow`) ```

If somebody wants me to try something, just tell! 😄

omdaniel commented 8 months ago

Just to add some more context to this, I get the same error and it will fail on different packages on each attempt if running uv clean in between

(.venv) PS C:\Program Files\envs\naduv> uv pip install ipython

Resolved 16 packages in 1.68s

error: Failed to download distributions
  Caused by: Failed to fetch wheel: prompt-toolkit==3.0.43
  Caused by: Failed to read from the distribution cache
  Caused by: failed to rename file from \\?\C:\Program Files\envs\pipx\venvs\venvs\uv\uvcache\.tmpvKXHjN to \\?\C:\Program Files\envs\pipx\venvs\venvs\uv\uvcache\archive-v0\VF2P3BmAtyIqJo8V9e9Jb
  Caused by: Access is denied. (os error 5)

(.venv) PS C:\Program Files\envs\naduv> uv clean

Clearing cache at: C:\Program Files\envs\pipx\venvs\venvs\uv\uvcache
Removed 369 files (3.6MiB)

(.venv) PS C:\Program Files\envs\n\n\n\naduv> uv pip install ipython

Resolved 16 packages in 1.77s

error: Failed to download distributions
  Caused by: Failed to fetch wheel: pygments==2.17.2
  Caused by: Failed to read from the distribution cache
  Caused by: failed to rename file from \\?\C:\Program Files\envs\pipx\venvs\venvs\uv\uvcache\.tmpA8BLxj to \\?\C:\Program Files\envs\pipx\venvs\venvs\uv\uvcache\archive-v0\xoGgYMWlHY-hrr7FtBrhm
  Caused by: Access is denied. (os error 5)

I suspect something in the temp file naming is randomly causing issues and after I tried 5 times with "uv clean" in-between it finished but it is happening frequently enough that there is a real issue. I am using the latest uv release 0.15 at the time of posting this issue

omdaniel commented 8 months ago

For many packages I can just run uv pip install [package] again without using "uv clean" and it will work without the "os error 5" but with numpy 1.26.4 it triggers every time and I tried it 15 times in a row

charliermarsh commented 8 months ago

I think this is an error when attempting to persist the unzipped wheel to the wheel cache?

omdaniel commented 8 months ago

I think this is an error when attempting to persist the unzipped wheel to the wheel cache?

The contents of the temp folder in $UV_CACHE_DIR are unzipped after the install fails but not sure if the process is asking the os to move the contents before the temp folder is created

charliermarsh commented 8 months ago

The overall approach is: unzip into a temporary folder, then move that folder into the cache to persist it. (This avoids persisting incomplete downloads that may error partway through.)

omdaniel commented 8 months ago

I think this issue may crop up in corporate or more secure environments that have some layer of security running over all processes. I think there is a slight lag between when uv unzips the files where that process is ran in a sandbox and there is a slight delay before the os will allow access to the resulting folder while its scanned, I think simply catching the "os error" and retrying with some timeout would prevent this error on systems that have some security layer that temporarily effects permissions and also why a package like numpy (big and many small files) would consistently trip and medium packages would occasionally go through and occasionally trip.

i.e. you may not be able to re-produce this error in dev without setting up a test box that use one of these paid corporate security software to trigger it. You generally only create windows binaries on tagged releases, if you can create a dev build binary I can test it on my system

Wofiel commented 8 months ago

I think this issue may crop up in corporate or more secure environments that have some layer of security running over all processes.

Ah, this is consistent with the experience in the first post.

omdaniel commented 8 months ago

@Wofiel Our issue may be too niche to get traction (which I don't disagree with at this stage of uv) time to learn rust and attempt a PR

nathanjmcdougall commented 8 months ago

I think this issue may crop up in corporate or more secure environments that have some layer of security running over all processes.

Ah, this is consistent with the experience in the first post.

This is also consistent with my experience.

FWIW, the versions that it tripped up on were debugpy==1.8.1 and markdown-it-py==3.0.0.

mkleinbort-ic commented 8 months ago

I hit this same error today.

trying to run:

 python -m uv pip install --system -r pyproject.toml --prerelease=allow

on windows.

I keep hitting

error: Failed to download and build: {SOME_PACKAGE_MAME}=={VERSION}
  Caused by: Failed to write to the distribution cache
  Caused by: failed to rename file from \\?\C:\Users\{USERNAME}\AppData\Local\Temp\.tmplED7gi\.tmpxkU2s1\{SOME_PACKAGE_MAME}-{VERSION} to \\?\C:\Users\{USERNAME}\AppData\Local\Temp\.tmplED7gi\built-wheels-v0\pypi\{SOME_PACKAGE_MAME}\{VERSION}\wwYdTWRMDuqFPxs7feX1m\{SOME_PACKAGE_MAME}-{VERSION}.tar.gz
  Caused by: Access is denied. (os error 5)

The package that causes the error seems to be random if I re-run the command.

konstin commented 8 months ago

Is anyone of you able to share (here or in private to konsti@astral.sh) what kind of security software or special windows settings you are running, so we can reproduce the failures you are experiencing?

mkleinbort-ic commented 8 months ago

I don't think there is anything special on my setup.

  1. Runing gitbash
  2. On windows 11
  3. running python -m pip install -r requirements.txt works
  4. running python -m uv pip install --system -r requirements.txt fails
charliermarsh commented 8 months ago

Is it possible that the use of the UNC prefix is itself the problem?

If I created a branch that omitted them, would anyone here be willing to cargo build locally and test it?

mkleinbort-ic commented 8 months ago

I could try it in the week - but don't have the tooling set up at the moment.

mkleinbort-ic commented 8 months ago

Do you mean this: "\\?\C:\..."?

charliermarsh commented 8 months ago

Yeah, that prefix on the file paths.

mkleinbort-ic commented 8 months ago

I don't know enough to help - but I definitely can't cd into "\?\C:\" using powershell or bash

in powershel: this works: cd C:\ this does not cd \\?\C:\

PS C:\> cd \\?\C:\
cd : Cannot process argument because the value of argument "path" is not valid. Change the value of the "path"
argument and run the operation again.
At line:1 char:1
+ cd \\?\C:\
+ ~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (:) [Set-Location], PSArgumentException
    + FullyQualifiedErrorId : Argument,Microsoft.PowerShell.Commands.SetLocationCommand
charliermarsh commented 8 months ago

Just curious -- what if you take the output of pwd, and then ls that prefixed with \\?\?

charliermarsh commented 8 months ago

Either way I'll put together a branch to remove these.

mkleinbort-ic commented 8 months ago

image

gregbedwell commented 8 months ago

The \\?\ prefix is just a way around the MAX_PATH (256 char) limitation in Windows API calls. I'd personally be surprised if it had an impact although I couldn't say for sure.

Having dealt with similar issues in other projects where Windows antivirus software is holding file locks after any file system operations, the best solution is usually just to retry the operation on failure a handful of times with a slightly increasing sleep in between attempts unfortunately.

mkleinbort commented 8 months ago

That's interesting - though retrying multiple times sort of defeats the purpose of UV being so fast.

I wonder why pip success but UV fails.

(Note we do use an antivirus)

omdaniel commented 8 months ago

That's interesting - though retrying multiple times sort of defeats the purpose of UV being so fast.

I wonder why pip success but UV fails.

(Note we do use an antivirus)

I think the effect on speed would be imperceptible to the end user. The uv speed up is more to do with dependency resolution and retrying is to deal with likely milliseconds of time that a security layer on Windows locks access to the extracted zip contents before uv can read and copy the data from the tmp folder to the archive

mkleinbort-ic commented 8 months ago

Oh, yes, happy if this is handled by uv

I just don't want to write a bash script on my end to retry the installation till it works

charliermarsh commented 8 months ago

Definitely want to fix this.

charliermarsh commented 8 months ago

Oh interesting, I just noticed that this is always related (except for one comment here) to persisting to archive-v0. But it succeeds if you rerun?

charliermarsh commented 8 months ago

I really need to be able to reproduce this in order to fix it. (The answer might be to retry with a timeout, but even that needs testing, and I need to determine the appropriate durations.) Can anyone share what antivirus they might be running that would trigger this?

charliermarsh commented 8 months ago

Installed Avast but couldn't reproduce.

omdaniel commented 8 months ago

I used watchdog to see what files a being touched and it shows that numpy always fails when it gets to 37 MB libopenblas64 dll unsurprisingly

omdaniel commented 8 months ago

Maybe a mock test can be made that ensures uv pip install is tolerant to a file temporarily being locked for a few milliseconds

mattibo commented 8 months ago

I got the same issue and i use Trellix and Zscaler.

omdaniel commented 8 months ago

Looking through documentation for Trellix, this may be triggered by https://docs.trellix.com/bundle/application-change-control-8.3.x-product-guide-windows/page/GUID-EE9619ED-4BAE-4A2D-9F5B-1D613B197F8C.html where the newly created (unzipped) DLL files are temporarily read/write protected while a determination is being made, ultimately an "allow" decision is made but uv tries to move those files while they are temporarily blocked, this may be highly architecturally dependent as there seems to be some communication between the end user client PC and the ePO policy server and additional latency there may impact the the frequency of this bug.

charliermarsh commented 8 months ago

I will own this.

charliermarsh commented 8 months ago

This will take some trial and error. I'll add some improvements in the next release, and then look to folks in this thread for testing.

mkleinbort-ic commented 8 months ago

Tested on uv==0.1.19 (with the exponential backoff). So far all good - no issues on my setup.

If it breaks again I'll comment here, but it seems to be fixed

charliermarsh commented 8 months ago

I'll optimistically close, but can re-open if anything comes up.

robmcd commented 8 months ago

Upgrading to uv==0.1.20 fixed this for me.

charliermarsh commented 8 months ago

Great, thank you!

mattibo commented 8 months ago

for me most of the packages synced fine with version uv==0.1.20! But i still have issues with one package of i think bigger size (https://squidfunk.github.io/mkdocs-material/).

This leads to following error:

Resolved 1 package in 4ms
DEBUG No cache entry for: https://files.pythonhosted.org/packages/64/dc/9817689e3500b8e3967302201341dda7878f33d4ad089f9deb1126a091d7/mkdocs_material-9.5.13-py3-none-any.whl
DEBUG No credentials found for: https://files.pythonhosted.org/packages/64/dc/9817689e3500b8e3967302201341dda7878f33d4ad089f9deb1126a091d7/mkdocs_material-9.5.13-py3-none-any.whl
WARN Retrying rename from \\?\C:\Users\x\AppData\Local\uv\cache\.tmpXVEqoW to \\?\C:\Users\x\AppData\Local\uv\cache\archive-v0\ivw1_SgSoJofW7-c2dlZ0 due to transient error: failed to rename file from \\?\C:\Users\x\AppData\Local\uv\cache\.tmpXVEqoW to \\?\C:\Users\x\AppData\Local\uv\cache\archive-v0\ivw1_SgSoJofW7-c2dlZ0
WARN Retrying rename from \\?\C:\Users\x\AppData\Local\uv\cache\.tmpXVEqoW to \\?\C:\Users\x\AppData\Local\uv\cache\archive-v0\ivw1_SgSoJofW7-c2dlZ0 due to transient error: failed to rename file from \\?\C:\Users\x\AppData\Local\uv\cache\.tmpXVEqoW to \\?\C:\Users\x\AppData\Local\uv\cache\archive-v0\ivw1_SgSoJofW7-c2dlZ0
WARN Retrying rename from \\?\C:\Users\x\AppData\Local\uv\cache\.tmpXVEqoW to \\?\C:\Users\x\AppData\Local\uv\cache\archive-v0\ivw1_SgSoJofW7-c2dlZ0 due to transient error: failed to rename file from \\?\C:\Users\x\AppData\Local\uv\cache\.tmpXVEqoW to \\?\C:\Users\x\AppData\Local\uv\cache\archive-v0\ivw1_SgSoJofW7-c2dlZ0
WARN Retrying rename from \\?\C:\Users\x\AppData\Local\uv\cache\.tmpXVEqoW to \\?\C:\Users\x\AppData\Local\uv\cache\archive-v0\ivw1_SgSoJofW7-c2dlZ0 due to transient error: failed to rename file from \\?\C:\Users\x\AppData\Local\uv\cache\.tmpXVEqoW to \\?\C:\Users\x\AppData\Local\uv\cache\archive-v0\ivw1_SgSoJofW7-c2dlZ0
WARN Retrying rename from \\?\C:\Users\x\AppData\Local\uv\cache\.tmpXVEqoW to \\?\C:\Users\x\AppData\Local\uv\cache\archive-v0\ivw1_SgSoJofW7-c2dlZ0 due to transient error: failed to rename file from \\?\C:\Users\x\AppData\Local\uv\cache\.tmpXVEqoW to \\?\C:\Users\x\AppData\Local\uv\cache\archive-v0\ivw1_SgSoJofW7-c2dlZ0
WARN Retrying rename from \\?\C:\Users\x\AppData\Local\uv\cache\.tmpXVEqoW to \\?\C:\Users\x\AppData\Local\uv\cache\archive-v0\ivw1_SgSoJofW7-c2dlZ0 due to transient error: failed to rename file from \\?\C:\Users\x\AppData\Local\uv\cache\.tmpXVEqoW to \\?\C:\Users\x\AppData\Local\uv\cache\archive-v0\ivw1_SgSoJofW7-c2dlZ0
WARN Retrying rename from \\?\C:\Users\x\AppData\Local\uv\cache\.tmpXVEqoW to \\?\C:\Users\x\AppData\Local\uv\cache\archive-v0\ivw1_SgSoJofW7-c2dlZ0 due to transient error: failed to rename file from \\?\C:\Users\x\AppData\Local\uv\cache\.tmpXVEqoW to \\?\C:\Users\x\AppData\Local\uv\cache\archive-v0\ivw1_SgSoJofW7-c2dlZ0
WARN Retrying rename from \\?\C:\Users\x\AppData\Local\uv\cache\.tmpXVEqoW to \\?\C:\Users\x\AppData\Local\uv\cache\archive-v0\ivw1_SgSoJofW7-c2dlZ0 due to transient error: failed to rename file from \\?\C:\Users\x\AppData\Local\uv\cache\.tmpXVEqoW to \\?\C:\Users\x\AppData\Local\uv\cache\archive-v0\ivw1_SgSoJofW7-c2dlZ0
WARN Retrying rename from \\?\C:\Users\x\AppData\Local\uv\cache\.tmpXVEqoW to \\?\C:\Users\x\AppData\Local\uv\cache\archive-v0\ivw1_SgSoJofW7-c2dlZ0 due to transient error: failed to rename file from \\?\C:\Users\x\AppData\Local\uv\cache\.tmpXVEqoW to \\?\C:\Users\x\AppData\Local\uv\cache\archive-v0\ivw1_SgSoJofW7-c2dlZ0
WARN Retrying rename from \\?\C:\Users\x\AppData\Local\uv\cache\.tmpXVEqoW to \\?\C:\Users\x\AppData\Local\uv\cache\archive-v0\ivw1_SgSoJofW7-c2dlZ0 due to transient error: failed to rename file from \\?\C:\Users\x\AppData\Local\uv\cache\.tmpXVEqoW to \\?\C:\Users\x\AppData\Local\uv\cache\archive-v0\ivw1_SgSoJofW7-c2dlZ0
WARN Retrying rename from \\?\C:\Users\x\AppData\Local\uv\cache\.tmpXVEqoW to \\?\C:\Users\x\AppData\Local\uv\cache\archive-v0\ivw1_SgSoJofW7-c2dlZ0 due to transient error: failed to rename file from \\?\C:\Users\x\AppData\Local\uv\cache\.tmpXVEqoW to \\?\C:\Users\x\AppData\Local\uv\cache\archive-v0\ivw1_SgSoJofW7-c2dlZ0
WARN Retrying rename from \\?\C:\Users\x\AppData\Local\uv\cache\.tmpXVEqoW to \\?\C:\Users\x\AppData\Local\uv\cache\archive-v0\ivw1_SgSoJofW7-c2dlZ0 due to transient error: failed to rename file from \\?\C:\Users\x\AppData\Local\uv\cache\.tmpXVEqoW to \\?\C:\Users\x\AppData\Local\uv\cache\archive-v0\ivw1_SgSoJofW7-c2dlZ0
WARN Retrying rename from \\?\C:\Users\x\AppData\Local\uv\cache\.tmpXVEqoW to \\?\C:\Users\x\AppData\Local\uv\cache\archive-v0\ivw1_SgSoJofW7-c2dlZ0 due to transient error: failed to rename file from \\?\C:\Users\x\AppData\Local\uv\cache\.tmpXVEqoW to \\?\C:\Users\x\AppData\Local\uv\cache\archive-v0\ivw1_SgSoJofW7-c2dlZ0
WARN Retrying rename from \\?\C:\Users\x\AppData\Local\uv\cache\.tmpXVEqoW to \\?\C:\Users\x\AppData\Local\uv\cache\archive-v0\ivw1_SgSoJofW7-c2dlZ0 due to transient error: failed to rename file from \\?\C:\Users\x\AppData\Local\uv\cache\.tmpXVEqoW to \\?\C:\Users\x\AppData\Local\uv\cache\archive-v0\ivw1_SgSoJofW7-c2dlZ0
WARN Retrying rename from \\?\C:\Users\x\AppData\Local\uv\cache\.tmpXVEqoW to \\?\C:\Users\x\AppData\Local\uv\cache\archive-v0\ivw1_SgSoJofW7-c2dlZ0 due to transient error: failed to rename file from \\?\C:\Users\x\AppData\Local\uv\cache\.tmpXVEqoW to \\?\C:\Users\x\AppData\Local\uv\cache\archive-v0\ivw1_SgSoJofW7-c2dlZ0
WARN Retrying rename from \\?\C:\Users\x\AppData\Local\uv\cache\.tmpXVEqoW to \\?\C:\Users\x\AppData\Local\uv\cache\archive-v0\ivw1_SgSoJofW7-c2dlZ0 due to transient error: failed to rename file from \\?\C:\Users\x\AppData\Local\uv\cache\.tmpXVEqoW to \\?\C:\Users\x\AppData\Local\uv\cache\archive-v0\ivw1_SgSoJofW7-c2dlZ0
error: Failed to download distributions
  Caused by: Failed to fetch wheel: mkdocs-material==9.5.13
  Caused by: Failed to read from the distribution cache
  Caused by: failed to rename file from \\?\C:\Users\x\AppData\Local\uv\cache\.tmpXVEqoW to \\?\C:\Users\x\AppData\Local\uv\cache\archive-v0\ivw1_SgSoJofW7-c2dlZ0
  Caused by: Zugriff verweigert (os error 5)
ofek commented 8 months ago

Did this happen yet? https://github.com/astral-sh/uv/issues/1491#issuecomment-1974827245

charliermarsh commented 8 months ago

Nope.

ofek commented 8 months ago

Ah okay, is there a tracking issue for that?

charliermarsh commented 8 months ago

I can do it, I just haven’t seen evidence that it’s a problem.

charliermarsh commented 8 months ago

To clarify: I understand that device paths can lead to confusing and incorrect behaviors. But we strip the device path prefix everywhere that matters (in my experience). So I'd like to understand why it would matter in the case above, for example.

ofek commented 8 months ago

One compelling reason, to me at least, is confusion for users. Until Rust happened I had quite literally never seen a UNC path in the wild; it was just some abstract concept I read about in the Windows APIs.

cc @mitsuhiko I'd be curious to get your opinion on this

mkleinbort-ic commented 8 months ago

I hit the issue again (on uv 0.1.20)

Running:

python -m uv pip install --python 3.11 --reinstall pylance==0.10.3

Resulted in

error: failed to remove file `C:\Users\x\AppData\Local\Programs\Python\Python311\Lib\site-packages\numpy.libs/libopenblas64__v0.3.23-293-gc2f4bdbb-gcc_10_3_0-2bde3a66a51006b2b53eb373ff767a3f.dll`
  Caused by: Access is denied. (os error 5)

and re-running it resulted in

error: Cannot uninstall package; RECORD file not found at: C:\Users\x\AppData\Local\Programs\Python\Python311\Lib\site-packages\numpy-1.26.4.dist-info\RECORD     
gregbedwell commented 8 months ago

One compelling reason, to me at least, is confusion for users. Until Rust happened I had quite literally never seen a UNC path in the wild; it was just some abstract concept I read about in the Windows APIs.

Note that by removing the UNC prefix you're potentially introducing a regression if the path is longer than 256 characters. In practice I don't know whether this is likely here, but I'd be interested to know the motivating reason for adding them in the first place. Was it in response to a long path issue?

See this page for more info: https://learn.microsoft.com/en-gb/windows/win32/fileio/naming-a-file?redirectedfrom=MSDN#win32-file-namespaces