astral-sh / uv

An extremely fast Python package and project manager, written in Rust.
https://docs.astral.sh/uv
Apache License 2.0
26.67k stars 774 forks source link

Unknown operation system: mingw_x86_64_ucrt #3573

Open srnwk opened 6 months ago

srnwk commented 6 months ago

Trying to use uv 0.1.43 on Windows from MSYS2 / with a mingw64 GNU ABI python results in the following error:

> uv venv venv
  × Can't use Python at `C:\MSYS2\ucrt64\bin\python3.exe`
  ╰─▶ Unknown operation system: `mingw_x86_64_ucrt`

The immediate error comes from uv-interpreter/python/get_interpreter_info.py's get_operating_system_and_architecture() not recognizing the mingw_x86_64_ucrt value returned by sysconfig.get_platform().
Also, since mingw python's virtual environments use the linux style paths (bin, not Scripts like MSVC ABI python), just recognizing mingw platforms as windows probably won't be enough to get uv working there.

henryiii commented 6 months ago

If you want to test on gha, it’s pretty easy: https://github.com/scikit-build/scikit-build-core/blob/86f40d949741d8e6ce4d9f55264a6d05224408dd/.github/workflows/ci.yml#L251

zanieb commented 6 months ago

Thanks Henry!

zanieb commented 6 months ago

We should support binsetups on Windows, although I'm making that consistent in #3266 and there may be bugs until that merges. I'll look into the other issue though.

zanieb commented 6 months ago

Hm in CI this fails with

error: Querying Python at `D:\a\_temp\msys64\mingw64\bin\python3.exe` failed with status exit code: 1 with exit code: 1
--- stdout:
--- stderr:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'python'
---

Which is a failure at https://github.com/astral-sh/uv/blob/55aedda37934ee2153312130de5efebbd23efebb/crates/uv-interpreter/src/interpreter.rs#L455 where we can't find our python query module that we inject into the system path.

Will need some further investigation... cc @konstin

samypr100 commented 6 months ago

Was this error from installing uv from an official msys2 package manager build?

e.g. https://packages.msys2.org/package/mingw-w64-ucrt-x86_64-uv?repo=ucrt64

Worth asking to confirm, what environment are you using msys with? What's the output of env | grep MSYSTEM?

srnwk commented 6 months ago

Was this error from installing uv from an official msys2 package manager build?

e.g. https://packages.msys2.org/package/mingw-w64-ucrt-x86_64-uv?repo=ucrt64

Worth asking to confirm, what environment are you using msys with? What's the output of env | grep MSYSTEM?

Yes, using the msys2 package, I think I got the same python path related error as zanieb above. Sorry for not mentioning that. Using the UCRT64 environment:

MSYSTEM=UCRT64
MSYSTEM_CARCH=x86_64
MSYSTEM_CHOST=x86_64-w64-mingw32
MSYSTEM_PREFIX=/ucrt64

Since I first noticed the problem in an automated build that runs outside the UCRT64 environment (but with ucrt64/bin on PATH, for the toolchain and python interpeter), I tried some more things.

It seems like the absence of the MSYSTEM env var in my setup is what side-stepped the python path problem, somehow:

$ uv venv venv
  x Querying Python at `D:\Programs\msys2\ucrt64\bin\python3.exe` failed with status exit code: 1
  | with exit code: 1
  | --- stdout:

  | --- stderr:
  | Traceback (most recent call last):
  |   File "<string>", line 1, in <module>
  | ModuleNotFoundError: No module named 'python'
  | ---
$ export "MSYSTEM="
$ uv venv venv
  x Can't use Python at `D:\Programs\msys2\ucrt64\bin\python3.exe`
  `-> Unknown operation system: `mingw_x86_64_ucrt`
konstin commented 6 months ago

For me it mainly complains that the strings are not escaped, even though the strings on the rust side use all \\:

  x Querying Python at `C:\msys64\usr\bin\python.exe` failed with status exit code: 1 with exit code: 1
  | --- stdout:

  | --- stderr:
  | File "<string>", line 1
  |     import sys; sys.path = ["C:\Users\Konstantin\AppData\Local\uv\cache\.tmp5Wlq3f"] + sys.path; from python.get_interpreter_info import main; main()      
  |                                                                                    ^
  | SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape
  | ---
konstin commented 6 months ago

The msys2 python i've installed wants unix paths instead of windows paths. I had to patch in the following:

        let import_path = "/".to_string()
            + &tempdir
                .path()
                // Remove `\\?\` UNC prefix
                .simplified_display()
                .to_string()
                // Remove the colon after the drive letter
                .replacen(":", "", 1)
                .replace("\\", "/");
        let script = format!(
            r#"import sys; sys.path = ["{}"] + sys.path; from python.get_interpreter_info import main; main()"#,
            import_path.escape_for_python()
        );

to get

  x Can't use Python at `C:\msys64\usr\bin\python.exe`
  `-> Unknown operation system: `msys`
error: process didn't exit successfully: `target\debug\uv.exe venv -p C:\msys64\usr\bin\python.exe` (exit code: 1)

To support this, we need to figure out a way to determine if a python is gnu or msvc before probing the interpreter.

Could someone getting a mingw_x86_64_ucrt share installation instructions so we can reproduce this one too?

srnwk commented 6 months ago

Could someone getting a mingw_x86_64_ucrt share installation instructions so we can reproduce this one too?

I installed the mingw/UCRT64 python:

pacman -S mingw-w64-ucrt-x86_64-python

and used the MSYS2 UCRT64 start menu shortcut to launch the shell: image

zanieb commented 6 months ago

@konstin you can also see it in CI at #3632

lazka commented 2 months ago

Had a quick look, and three things:

1) I don't think it's reasonable to expect from uv to support cygwin python, so all those errors about <..>/usr/bin/python* are fine imo, and those interpreters should be ignored.

2) mingw Python doesn't seem to deal properly with UNC paths in sys.path. This could be worked around by using something like dunce::simplified() on the uv side from what I understand. Also needs investigation on the mingw-Python side why UNC paths don't work in sys.path. (edit: filed here now: https://github.com/msys2-contrib/cpython-mingw/issues/176)

3) Once a non-UNC path is used uv gets back {"result": "error", "kind": "unknown_operating_system", "operating_system": "mingw_x86_64_ucrt"} which is due to it parsing the platform string instead of using sys.platform or os.name (it's also missing Windows arm64 support from what I see): https://github.com/astral-sh/uv/blob/d8c41481ec884cc5166bcfadd95674ff7b2e918d/crates/uv-python/python/get_interpreter_info.py#L428