astral-sh / rye

a Hassle-Free Python Experience
https://rye.astral.sh
MIT License
13.67k stars 467 forks source link

Concurrent bootstrap attempts can fail #960

Open stuhood opened 6 months ago

stuhood commented 6 months ago

Steps to Reproduce

Running multiple rye (sync in particular, but likely any command which uses an interpreter) processes concurrently, some of the processes can fail with:

$ rye sync --no-lock
Bootstrapping rye internals
Downloading cpython@3.12.2
Checking checksum
Unpacking
error: could not sync because bootstrap failed

Caused by:
    0: failed to fetch internal cpython toolchain cpython@3.12
    1: unpacking of downloaded tarball https://github.com/indygreg/python-build-standalone/releases/download/20240224/cpython-3.12.2%2B20240224-x86_64-unknown-linux-gnu-pgo%2Blto-full.tar.zst to '/home/vscode/.rye/py/cpython@3.12.2' failed
    2: failed to unpack `python/build/Modules/Setup` into `/home/vscode/.rye/py/cpython@3.12.2/build/Modules/Setup`
    3: File exists (os error 17)

...while the others succeed.

Expected Result

Unpacked artifacts should be moved to their final location atomically.

Actual Result

See above.

Version Info

$ rye --version
rye 0.31.0
commit: 0.31.0 (72e239b8f 2024-03-22)
platform: linux (x86_64)
self-python: cpython@3.12.2
symlink support: true
uv enabled: true

Stacktrace

No response

stuhood commented 6 months ago

I think that this can also exhibit as:

Bootstrapping rye internals
Found a compatible Python version: cpython@3.12.2
  × Querying Python at
  │ `/home/vscode/.rye/py/cpython@3.12.2/install/bin/python3` failed with
  │ status exit status: 1 with exit status: 1
  │ --- stdout:

  │ --- stderr:
  │ 'import warnings' failed; traceback:
  │ ModuleNotFoundError: No module named 'warnings'
  │ Could not import runpy module
  │ Traceback (most recent call last):
  │   File "<frozen runpy>", line 14, in <module>
  │   File
  │ "/home/vscode/.rye/py/cpython@3.12.2/install/lib/python3.12/importlib/__init__.py",
  │ line 57, in <module>
  │     import warnings
  │ ModuleNotFoundError: No module named 'warnings'
  │ ---
error: error bootstrapping venv

Caused by:
    Failed to create self venv using /home/vscode/.rye/py/cpython@3.12.2/install/bin/python3. uv exited with status: exit status: 1

...perhaps if a Python install was partially extracted when another run tried to use it?