astral-sh / uv

An extremely fast Python package installer and resolver, written in Rust.
https://astral.sh/
Apache License 2.0
14.72k stars 419 forks source link

Unknown operation system: mingw_x86_64_ucrt #3573

Open srnwk opened 1 month ago

srnwk commented 1 month ago

Trying to use uv 0.1.43 on Windows from MSYS2 / with a mingw64 GNU ABI python results in the following error:

> uv venv venv
  × Can't use Python at `C:\MSYS2\ucrt64\bin\python3.exe`
  ╰─▶ Unknown operation system: `mingw_x86_64_ucrt`

The immediate error comes from uv-interpreter/python/get_interpreter_info.py's get_operating_system_and_architecture() not recognizing the mingw_x86_64_ucrt value returned by sysconfig.get_platform().
Also, since mingw python's virtual environments use the linux style paths (bin, not Scripts like MSVC ABI python), just recognizing mingw platforms as windows probably won't be enough to get uv working there.

henryiii commented 1 month ago

If you want to test on gha, it’s pretty easy: https://github.com/scikit-build/scikit-build-core/blob/86f40d949741d8e6ce4d9f55264a6d05224408dd/.github/workflows/ci.yml#L251

zanieb commented 1 month ago

Thanks Henry!

zanieb commented 1 month ago

We should support binsetups on Windows, although I'm making that consistent in #3266 and there may be bugs until that merges. I'll look into the other issue though.

zanieb commented 1 month ago

Hm in CI this fails with

error: Querying Python at `D:\a\_temp\msys64\mingw64\bin\python3.exe` failed with status exit code: 1 with exit code: 1
--- stdout:
--- stderr:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'python'
---

Which is a failure at https://github.com/astral-sh/uv/blob/55aedda37934ee2153312130de5efebbd23efebb/crates/uv-interpreter/src/interpreter.rs#L455 where we can't find our python query module that we inject into the system path.

Will need some further investigation... cc @konstin

samypr100 commented 1 month ago

Was this error from installing uv from an official msys2 package manager build?

e.g. https://packages.msys2.org/package/mingw-w64-ucrt-x86_64-uv?repo=ucrt64

Worth asking to confirm, what environment are you using msys with? What's the output of env | grep MSYSTEM?

srnwk commented 1 month ago

Was this error from installing uv from an official msys2 package manager build?

e.g. https://packages.msys2.org/package/mingw-w64-ucrt-x86_64-uv?repo=ucrt64

Worth asking to confirm, what environment are you using msys with? What's the output of env | grep MSYSTEM?

Yes, using the msys2 package, I think I got the same python path related error as zanieb above. Sorry for not mentioning that. Using the UCRT64 environment:

MSYSTEM=UCRT64
MSYSTEM_CARCH=x86_64
MSYSTEM_CHOST=x86_64-w64-mingw32
MSYSTEM_PREFIX=/ucrt64

Since I first noticed the problem in an automated build that runs outside the UCRT64 environment (but with ucrt64/bin on PATH, for the toolchain and python interpeter), I tried some more things.

It seems like the absence of the MSYSTEM env var in my setup is what side-stepped the python path problem, somehow:

$ uv venv venv
  x Querying Python at `D:\Programs\msys2\ucrt64\bin\python3.exe` failed with status exit code: 1
  | with exit code: 1
  | --- stdout:

  | --- stderr:
  | Traceback (most recent call last):
  |   File "<string>", line 1, in <module>
  | ModuleNotFoundError: No module named 'python'
  | ---
$ export "MSYSTEM="
$ uv venv venv
  x Can't use Python at `D:\Programs\msys2\ucrt64\bin\python3.exe`
  `-> Unknown operation system: `mingw_x86_64_ucrt`
konstin commented 1 month ago

For me it mainly complains that the strings are not escaped, even though the strings on the rust side use all \\:

  x Querying Python at `C:\msys64\usr\bin\python.exe` failed with status exit code: 1 with exit code: 1
  | --- stdout:

  | --- stderr:
  | File "<string>", line 1
  |     import sys; sys.path = ["C:\Users\Konstantin\AppData\Local\uv\cache\.tmp5Wlq3f"] + sys.path; from python.get_interpreter_info import main; main()      
  |                                                                                    ^
  | SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape
  | ---
konstin commented 1 month ago

The msys2 python i've installed wants unix paths instead of windows paths. I had to patch in the following:

        let import_path = "/".to_string()
            + &tempdir
                .path()
                // Remove `\\?\` UNC prefix
                .simplified_display()
                .to_string()
                // Remove the colon after the drive letter
                .replacen(":", "", 1)
                .replace("\\", "/");
        let script = format!(
            r#"import sys; sys.path = ["{}"] + sys.path; from python.get_interpreter_info import main; main()"#,
            import_path.escape_for_python()
        );

to get

  x Can't use Python at `C:\msys64\usr\bin\python.exe`
  `-> Unknown operation system: `msys`
error: process didn't exit successfully: `target\debug\uv.exe venv -p C:\msys64\usr\bin\python.exe` (exit code: 1)

To support this, we need to figure out a way to determine if a python is gnu or msvc before probing the interpreter.

Could someone getting a mingw_x86_64_ucrt share installation instructions so we can reproduce this one too?

srnwk commented 1 month ago

Could someone getting a mingw_x86_64_ucrt share installation instructions so we can reproduce this one too?

I installed the mingw/UCRT64 python:

pacman -S mingw-w64-ucrt-x86_64-python

and used the MSYS2 UCRT64 start menu shortcut to launch the shell: image

zanieb commented 1 month ago

@konstin you can also see it in CI at #3632