search

pypa / cibuildwheel

🎡 Build Python wheels for all the platforms with minimal configuration.
https://cibuildwheel.pypa.io
Other
1.78k stars 227 forks source link

build[uv] uses wrong Python when testing on Windows #1877

Closed njzjz closed 2 weeks ago

njzjz commented 2 weeks ago

Description

It worked yesterday but failed today.

See below. The cp311-win_amd64 build uses Python 3.7.9 to create the environment.

Building cp311-win_amd64 wheel
CPython 3.11 Windows 64bit
...
Testing wheel...

  + uv venv 'C:\Users\runneradmin\AppData\Local\Temp\cibw-run-90ozl8o3\cp311-win_amd64\venv-test' --python=python
  Using Python 3.7.9 interpreter at: C:\hostedtoolcache\windows\Python\3.7.9\x64\python3.exe
  Creating virtualenv at: C:\Users\runneradmin\AppData\Local\Temp\cibw-run-90ozl8o3\cp311-win_amd64\venv-test
  Activate with: C:\Users\runneradmin\AppData\Local\Temp\cibw-run-90ozl8o3\cp311-win_amd64\venv-test\Scripts\activate
  + where python
  C:\Users\runneradmin\AppData\Local\Temp\cibw-run-90ozl8o3\cp311-win_amd64\venv-test\Scripts\python.exe
  C:\Users\runneradmin\AppData\Local\Temp\cibw-run-90ozl8o3\cp311-win_amd64\build\venv\Scripts\python.exe
  C:\hostedtoolcache\windows\Python\3.7.9\x64\python.exe
  + uv pip install 'C:\Users\runneradmin\AppData\Local\Temp\cibw-run-90ozl8o3\cp311-win_amd64\repaired_wheel\deepmd_kit-3.0.0a1.dev211+g4a27dec7-py37-none-win_amd64.whl[cpu]'
    × No solution found when resolving dependencies:
    ╰─▶ Because only deepmd-kit[cpu]==3.0.0a1.dev211+g4a27dec7 is available and
        the current Python version (3.7.9) does not satisfy Python>=3.8, we can
        conclude that all versions of deepmd-kit[cpu] are incompatible.
        And because you require deepmd-kit[cpu], we can conclude that the
        requirements are unsatisfiable.
                                                              ✕ 6.49s
Error: Command ['uv', 'pip', 'install', 'C:\\Users\\runneradmin\\AppData\\Local\\Temp\\cibw-run-90ozl8o3\\cp311-win_amd64\\repaired_wheel\\deepmd_kit-3.0.0a1.dev211+g4a27dec7-py37-none-win_amd64.whl[cpu]'] failed with code 1.

Build log

https://github.com/deepmodeling/deepmd-kit/actions/runs/9483099069/job/26133743134

CI config

https://github.com/deepmodeling/deepmd-kit/blob/devel/.github/workflows/build_wheel.yml

henryiii commented 2 weeks ago

Could you check the previous version of uv? https://github.com/deepmodeling/deepmd-kit/blob/73dab63fa847acb1acd0aa76700130e10a8e8516/.github/workflows/build_wheel.yml#L78 can be modified to install a specific version. Like https://astral.sh/uv/0.2.6/install.sh.

We specify the Python version at https://github.com/pypa/cibuildwheel/blob/78da7bafcc41b0af99c3fe002a5f9f492505a354/cibuildwheel/windows.py#L521

njzjz commented 2 weeks ago

It works with uv 0.2.6: https://github.com/deepmodeling/deepmd-kit/actions/runs/9488527936/job/26147696043

Testing wheel...

  + uv venv 'C:\Users\runneradmin\AppData\Local\Temp\cibw-run-l0v678gi\cp311-win_amd64\venv-test' --python=python
  Using Python 3.11.9 interpreter at: C:\Users\runneradmin\AppData\Local\Temp\cibw-run-l0v678gi\cp311-win_amd64\build\venv\Scripts\python.exe
  Creating virtualenv at: C:\Users\runneradmin\AppData\Local\Temp\cibw-run-l0v678gi\cp311-win_amd64\venv-test
  Activate with: C:\Users\runneradmin\AppData\Local\Temp\cibw-run-l0v678gi\cp311-win_amd64\venv-test\Scripts\activate
  + where python
  C:\Users\runneradmin\AppData\Local\Temp\cibw-run-l0v678gi\cp311-win_amd64\venv-test\Scripts\python.exe
  C:\Users\runneradmin\AppData\Local\Temp\cibw-run-l0v678gi\cp311-win_amd64\build\venv\Scripts\python.exe
  C:\hostedtoolcache\windows\Python\3.7.9\x64\python.exe
henryiii commented 2 weeks ago

Can you try with the latest one and RUST_LOG set to uv=trace? Our CI is using the latest one and isn't broken.

njzjz commented 2 weeks ago 
Testing wheel...
  + uv venv 'C:\Users\runneradmin\AppData\Local\Temp\cibw-run-x8apk4mi\cp311-win_amd64\venv-test' --python=python
  DEBUG Searching for Python  in search path or `py` launcher output
  TRACE Searching PATH for executables: python3.exe, python.exe
  TRACE Checking `PATH` directory for interpreters: C:\Users\runneradmin\AppData\Local\Temp\cibw-run-x8apk4mi\cp311-win_amd64\build\venv
  TRACE Checking `PATH` directory for interpreters: C:\Users\runneradmin\AppData\Local\Temp\cibw-run-x8apk4mi\cp311-win_amd64\build\venv\Scripts
  TRACE Found possible Python executable: C:\Users\runneradmin\AppData\Local\Temp\cibw-run-x8apk4mi\cp311-win_amd64\build\venv\Scripts\python.exe
  TRACE Cached interpreter info for Python 3.11.9, skipping probing: C:\Users\runneradmin\AppData\Local\Temp\cibw-run-x8apk4mi\cp311-win_amd64\build\venv\Scripts\python.exe
  DEBUG Found CPython 3.11.9 at `C:\Users\runneradmin\AppData\Local\Temp\cibw-run-x8apk4mi\cp311-win_amd64\build\venv\Scripts\python.exe` (search path)
  DEBUG Ignoring Python interpreter at `C:\Users\runneradmin\AppData\Local\Temp\cibw-run-x8apk4mi\cp311-win_amd64\build\venv\Scripts\python.exe`: system interpreter required
  TRACE Checking `PATH` directory for interpreters: C:\Program Files\PowerShell\7
  TRACE Checking `PATH` directory for interpreters: C:\Program Files\MongoDB\Server\5.0\bin
  TRACE Checking `PATH` directory for interpreters: C:\aliyun-cli
  TRACE Checking `PATH` directory for interpreters: C:\vcpkg
  TRACE Checking `PATH` directory for interpreters: C:\cf-cli
  TRACE Checking `PATH` directory for interpreters: C:\Program Files (x86)\NSIS\
  TRACE Checking `PATH` directory for interpreters: C:\tools\zstd
  TRACE Checking `PATH` directory for interpreters: C:\Program Files\Mercurial\
  TRACE Checking `PATH` directory for interpreters: C:\hostedtoolcache\windows\stack\2.15.7\x64
  TRACE Checking `PATH` directory for interpreters: C:\\ghcup\bin
  TRACE Checking `PATH` directory for interpreters: C:\mingw64\bin
  TRACE Checking `PATH` directory for interpreters: C:\Program Files\dotnet
  TRACE Checking `PATH` directory for interpreters: C:\Program Files\MySQL\MySQL Server 5.7\bin
  TRACE Checking `PATH` directory for interpreters: C:\Program Files\R\R-4.4.0\bin\x64
  TRACE Checking `PATH` directory for interpreters: C:\SeleniumWebDrivers\GeckoDriver
  TRACE Checking `PATH` directory for interpreters: C:\SeleniumWebDrivers\EdgeDriver\
  TRACE Checking `PATH` directory for interpreters: C:\SeleniumWebDrivers\ChromeDriver
  TRACE Checking `PATH` directory for interpreters: C:\Program Files (x86)\sbt\bin
  TRACE Checking `PATH` directory for interpreters: C:\Program Files\Git\bin
  TRACE Checking `PATH` directory for interpreters: C:\Program Files (x86)\pipx_bin
  TRACE Checking `PATH` directory for interpreters: C:\npm\prefix
  TRACE Checking `PATH` directory for interpreters: C:\hostedtoolcache\windows\go\1.21.11\x64\bin
  TRACE Checking `PATH` directory for interpreters: C:\hostedtoolcache\windows\Python\3.7.9\x64\Scripts
  TRACE Checking `PATH` directory for interpreters: C:\hostedtoolcache\windows\Python\3.7.9\x64
  TRACE Found possible Python executable: C:\hostedtoolcache\windows\Python\3.7.9\x64\python3.exe
  TRACE Querying interpreter executable at C:\hostedtoolcache\windows\Python\3.7.9\x64\python3.exe
  DEBUG Found CPython 3.7.9 at `C:\hostedtoolcache\windows\Python\3.7.9\x64\python3.exe` (search path)
  Using Python 3.7.9 interpreter at: C:\hostedtoolcache\windows\Python\3.7.9\x64\python3.exe
  Creating virtualenv at: C:\Users\runneradmin\AppData\Local\Temp\cibw-run-x8apk4mi\cp311-win_amd64\venv-test
  Activate with: C:\Users\runneradmin\AppData\Local\Temp\cibw-run-x8apk4mi\cp311-win_amd64\venv-test\Scripts\activate
  + where python
  C:\Users\runneradmin\AppData\Local\Temp\cibw-run-x8apk4mi\cp311-win_amd64\venv-test\Scripts\python.exe
  C:\Users\runneradmin\AppData\Local\Temp\cibw-run-x8apk4mi\cp311-win_amd64\build\venv\Scripts\python.exe
  C:\hostedtoolcache\windows\Python\3.7.9\x64\python.exe
zanieb commented 2 weeks ago

Hi! What's happening here is the Python interpreter at the top of the PATH is inside a virtual environment so we skip it. We don't use interpreters found inside virtual environments to create virtual environments.

zanieb commented 2 weeks ago

And now I see why you were using the --python=python flag, we changed this behavior to be consistent in https://github.com/astral-sh/uv/pull/3728 because it seemed confusing otherwise.

edit: Henry pointed out this pull request isn't the reason, investigating what pull request changed it.

mayeut commented 2 weeks ago

Our CI is using the latest one and isn't broken.

I'm not sure there are tests for tests (except on macOS).

zanieb commented 2 weeks ago

Thanks looks like you found an unexpected change in the latest release https://github.com/astral-sh/uv/pull/4289

I would still strongly recommend not using --python=python — its a weird way to request the interpreter you want and I will probably change the behavior here to be more consistent.

henryiii commented 2 weeks ago

Passing through the proper path in #1881. Could you test the branch, @njzjz?

njzjz commented 2 weeks ago

Passing through the proper path in #1881. Could you test the branch, @njzjz?

Thanks. It works:

Testing wheel...
  + uv venv 'C:\Users\runneradmin\AppData\Local\Temp\cibw-run-d2jic__1\cp311-win_amd64\venv-test' '--python=C:\Users\runneradmin\AppData\Local\pypa\cibuildwheel\Cache\nuget-cpython\python.3.11.9\tools\python.exe'
  DEBUG Checking for Python interpreter at path `C:\Users\runneradmin\AppData\Local\pypa\cibuildwheel\Cache\nuget-cpython\python.3.11.9\tools\python.exe`
  TRACE Cached interpreter info for Python 3.11.9, skipping probing: C:\Users\runneradmin\AppData\Local\pypa\cibuildwheel\Cache\nuget-cpython\python.3.11.9\tools\python.exe
  Using Python 3.11.9 interpreter at: C:\Users\runneradmin\AppData\Local\pypa\cibuildwheel\Cache\nuget-cpython\python.3.11.9\tools\python.exe
  Creating virtualenv at: C:\Users\runneradmin\AppData\Local\Temp\cibw-run-d2jic__1\cp311-win_amd64\venv-test
  Activate with: C:\Users\runneradmin\AppData\Local\Temp\cibw-run-d2jic__1\cp311-win_amd64\venv-test\Scripts\activate