msys2 / MINGW-packages

Package scripts for MinGW-w64 targets to build under MSYS2.
https://packages.msys2.org
BSD 3-Clause "New" or "Revised" License
2.24k stars 1.21k forks source link

python3 -m pip install ninja failure #16013

Open winterz opened 1 year ago

winterz commented 1 year ago

I'm having trouble with pre-commit python package. the issue seems to be with ninja, as shown in this error. I reporting to msys2 project since I'm not experiencing any problems with pre-commit on linux , mac or Windows. or is the problem with scikit? sorry, I'm really not sure where to report this problem

> python3 -m pip install ninja
Collecting ninja
  Using cached ninja-1.10.2.3.tar.gz (26 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: ninja
  Building wheel for ninja (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for ninja (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [21 lines of output]
      Traceback (most recent call last):
        File "/usr/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
          main()
        File "/usr/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/usr/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 261, in build_wheel
          return _build_backend().build_wheel(wheel_directory, config_settings,
        File "/tmp/pip-build-env-tc3o96h8/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 412, in build_wheel
          return self._build_with_temp_dir(['bdist_wheel'], '.whl',
        File "/tmp/pip-build-env-tc3o96h8/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 397, in _build_with_temp_dir
          self.run_setup()
        File "/tmp/pip-build-env-tc3o96h8/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 335, in run_setup
          exec(code, locals())
        File "<string>", line 32, in <module>
        File "/tmp/pip-build-env-tc3o96h8/overlay/lib/python3.10/site-packages/skbuild/setuptools_wrap.py", line 613, in setup
          cmkr = cmaker.CMaker(cmake_executable)
        File "/tmp/pip-build-env-tc3o96h8/overlay/lib/python3.10/site-packages/skbuild/cmaker.py", line 142, in __init__
          self.platform = get_platform()
        File "/tmp/pip-build-env-tc3o96h8/overlay/lib/python3.10/site-packages/skbuild/platform_specifics/platform_factory.py", line 40, in get_platform
          raise RuntimeError(msg)
      RuntimeError: Unsupported platform: mingw64_nt-10.0-19043. Please contact the scikit-build team.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for ninja
Failed to build ninja
ERROR: Could not build wheels for ninja, which is required to install pyproject.toml-based projects
Biswa96 commented 1 year ago

Would you like to provide some information about the real issue or what you are trying to solve? It will help to provide an actual solution without going through XY problem.

winterz commented 1 year ago

First I install pre-commit using: python -m pip install -U pre-commit seems happy

then cd to my git repository and run pre-commit install. so far so good

and then actually try to use pre-commit to do something:

pre-commit run --all-files [INFO] Installing environment for https://github.com/pre-commit/mirrors-clang-format. [INFO] Once installed this environment will be reused. [INFO] This may take a few minutes... An unexpected error has occurred: CalledProcessError: command: ('/home/allen/.cache/pre-commit/repogp_qslvr/py_env-python3.10/bin/python', '-mpip', 'install', '.') return code: 1 expected return code: 0 stdout: Processing /home/allen/.cache/pre-commit/repogp_qslvr Preparing metadata (setup.py): started Preparing metadata (setup.py): finished with status 'done' Collecting clang-format==13.0.0 Using cached clang-format-13.0.0.tar.gz (8.2 kB) Installing build dependencies: started Installing build dependencies: still running... Installing build dependencies: finished with status 'error'

stderr: error: subprocess-exited-with-error

  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [55 lines of output]
      Collecting setuptools>=42
        Using cached setuptools-65.3.0-py3-none-any.whl (1.2 MB)
      Collecting wheel
        Using cached wheel-0.37.1-py2.py3-none-any.whl (35 kB)
      Collecting scikit-build
        Using cached scikit_build-0.15.0-py2.py3-none-any.whl (77 kB)
      Collecting ninja
        Using cached ninja-1.10.2.3.tar.gz (26 kB)
        Installing build dependencies: started
        Installing build dependencies: finished with status 'done'
        Getting requirements to build wheel: started
        Getting requirements to build wheel: finished with status 'done'
        Preparing metadata (pyproject.toml): started
        Preparing metadata (pyproject.toml): finished with status 'done'
      Collecting packaging
        Using cached packaging-21.3-py3-none-any.whl (40 kB)
      Collecting distro
        Using cached distro-1.7.0-py3-none-any.whl (20 kB)
      Collecting pyparsing!=3.0.5,>=2.0.2
        Using cached pyparsing-3.0.9-py3-none-any.whl (98 kB)
      Building wheels for collected packages: ninja
        Building wheel for ninja (pyproject.toml): started
        Building wheel for ninja (pyproject.toml): finished with status 'error'
        error: subprocess-exited-with-error

        × Building wheel for ninja (pyproject.toml) did not run successfully.
        │ exit code: 1
        ╰─> [21 lines of output]
            Traceback (most recent call last):
              File "/home/allen/.cache/pre-commit/repogp_qslvr/py_env-python3.10/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
                main()
              File "/home/allen/.cache/pre-commit/repogp_qslvr/py_env-python3.10/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
                json_out['return_val'] = hook(**hook_input['kwargs'])
              File "/home/allen/.cache/pre-commit/repogp_qslvr/py_env-python3.10/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 261, in build_wheel
                return _build_backend().build_wheel(wheel_directory, config_settings,
              File "/tmp/pip-build-env-9o25i8wx/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 412, in build_wheel
                return self._build_with_temp_dir(['bdist_wheel'], '.whl',
              File "/tmp/pip-build-env-9o25i8wx/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 397, in _build_with_temp_dir
                self.run_setup()
              File "/tmp/pip-build-env-9o25i8wx/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 335, in run_setup
                exec(code, locals())
              File "<string>", line 32, in <module>
              File "/tmp/pip-build-env-9o25i8wx/overlay/lib/python3.10/site-packages/skbuild/setuptools_wrap.py", line 613, in setup
                cmkr = cmaker.CMaker(cmake_executable)
              File "/tmp/pip-build-env-9o25i8wx/overlay/lib/python3.10/site-packages/skbuild/cmaker.py", line 142, in __init__
                self.platform = get_platform()
              File "/tmp/pip-build-env-9o25i8wx/overlay/lib/python3.10/site-packages/skbuild/platform_specifics/platform_factory.py", line 40, in get_platform
                raise RuntimeError(msg)
            RuntimeError: Unsupported platform: mingw64_nt-10.0-19043. Please contact the scikit-build team.
            [end of output]

        note: This error originates from a subprocess, and is likely not a problem with pip.
        ERROR: Failed building wheel for ninja
      Failed to build ninja
      ERROR: Could not build wheels for ninja, which is required to install pyproject.toml-based projects
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Check the log at /home/allen/.cache/pre-commit/pre-commit.log

which then led me to trying python3 -m pip install ninja , as I mentioned in the initial post does python3 -m pip install ninja work for you?

Biswa96 commented 1 year ago

From the OP, it seems that you are using msys2 python. Try to use the mingw one.

  1. Open the blue 'M' icon in start menu.
  2. Install required packages with pacman, for example
pacman -Syyu
pacman -S $MINGW_PACKAGE_PREFIX-python $MINGW_PACKAGE_PREFIX-python-pip $MINGW_PACKAGE_PREFIX-ninja
  1. Now try your command.
  2. Be aware that there are some known issues with mingw python vs official python, see https://www.msys2.org/docs/python/
Here is the output of pip command ``` $ python -m pip install -U pre-commit Collecting pre-commit Downloading pre_commit-2.20.0-py2.py3-none-any.whl (199 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 199.5/199.5 kB 1.5 MB/s eta 0:00:00 Collecting virtualenv>=20.0.8 Downloading virtualenv-20.16.5-py3-none-any.whl (8.8 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.8/8.8 MB 6.9 MB/s eta 0:00:00 Collecting nodeenv>=0.11.1 Downloading nodeenv-1.7.0-py2.py3-none-any.whl (21 kB) Collecting pyyaml>=5.1 Downloading PyYAML-6.0.tar.gz (124 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 125.0/125.0 kB 3.6 MB/s eta 0:00:00 Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Collecting cfgv>=2.0.0 Downloading cfgv-3.3.1-py2.py3-none-any.whl (7.3 kB) Requirement already satisfied: toml in f:/msys64/ucrt64/lib/python3.10/site-packages (from pre-commit) (0.10.2) Collecting identify>=1.0.0 Downloading identify-2.5.5-py2.py3-none-any.whl (98 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 98.7/98.7 kB 2.9 MB/s eta 0:00:00 Requirement already satisfied: setuptools in f:/msys64/ucrt64/lib/python3.10/site-packages (from nodeenv>=0.11.1->pre-commit) (65.3.0) Requirement already satisfied: distlib<1,>=0.3.5 in f:/msys64/ucrt64/lib/python3.10/site-packages (from virtualenv>=20.0.8->pre-commit) (0.3.6) Collecting filelock<4,>=3.4.1 Downloading filelock-3.8.0-py3-none-any.whl (10 kB) Collecting platformdirs<3,>=2.4 Downloading platformdirs-2.5.2-py3-none-any.whl (14 kB) Building wheels for collected packages: pyyaml Building wheel for pyyaml (pyproject.toml) ... done Created wheel for pyyaml: filename=PyYAML-6.0-cp310-cp310-mingw_x86_64_ucrt.whl size=45337 sha256=4764182f927caa1cf2124aeb72cf5ef95728f6095b0fb0cdc8bde0c861c68354 Stored in directory: c:/users/x/appdata/local/pip/cache/wheels/1d/f3/b4/4aea0992adbed14b36ce9c3857d3707c762a4374479230685d Successfully built pyyaml Installing collected packages: pyyaml, platformdirs, nodeenv, identify, filelock, cfgv, virtualenv, pre-commit Successfully installed cfgv-3.3.1 filelock-3.8.0 identify-2.5.5 nodeenv-1.7.0 platformdirs-2.5.2 pre-commit-2.20.0 pyyaml-6.0 virtualenv-20.16.5 ```
winterz commented 1 year ago

First I install pre-commit using: python -m pip install -U pre-commit seems happy

then cd to my git repository and run pre-commit install. so far so good

and then actually try to use pre-commit to do something:

pre-commit run --all-files [INFO] Installing environment for https://github.com/pre-commit/mirrors-clang-format. [INFO] Once installed this environment will be reused. [INFO] This may take a few minutes... An unexpected error has occurred: CalledProcessError: command: ('/home/allen/.cache/pre-commit/repogp_qslvr/py_env-python3.10/bin/python', '-mpip', 'install', '.') return code: 1 expected return code: 0 stdout: Processing /home/allen/.cache/pre-commit/repogp_qslvr Preparing metadata (setup.py): started Preparing metadata (setup.py): finished with status 'done' Collecting clang-format==13.0.0 Using cached clang-format-13.0.0.tar.gz (8.2 kB) Installing build dependencies: started Installing build dependencies: still running... Installing build dependencies: finished with status 'error'

stderr: error: subprocess-exited-with-error

  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [55 lines of output]
      Collecting setuptools>=42
        Using cached setuptools-65.3.0-py3-none-any.whl (1.2 MB)
      Collecting wheel
        Using cached wheel-0.37.1-py2.py3-none-any.whl (35 kB)
      Collecting scikit-build
        Using cached scikit_build-0.15.0-py2.py3-none-any.whl (77 kB)
      Collecting ninja
        Using cached ninja-1.10.2.3.tar.gz (26 kB)
        Installing build dependencies: started
        Installing build dependencies: finished with status 'done'
        Getting requirements to build wheel: started
        Getting requirements to build wheel: finished with status 'done'
        Preparing metadata (pyproject.toml): started
        Preparing metadata (pyproject.toml): finished with status 'done'
      Collecting packaging
        Using cached packaging-21.3-py3-none-any.whl (40 kB)
      Collecting distro
        Using cached distro-1.7.0-py3-none-any.whl (20 kB)
      Collecting pyparsing!=3.0.5,>=2.0.2
        Using cached pyparsing-3.0.9-py3-none-any.whl (98 kB)
      Building wheels for collected packages: ninja
        Building wheel for ninja (pyproject.toml): started
        Building wheel for ninja (pyproject.toml): finished with status 'error'
        error: subprocess-exited-with-error

        × Building wheel for ninja (pyproject.toml) did not run successfully.
        │ exit code: 1
        ╰─> [21 lines of output]
            Traceback (most recent call last):
              File "/home/allen/.cache/pre-commit/repogp_qslvr/py_env-python3.10/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
                main()
              File "/home/allen/.cache/pre-commit/repogp_qslvr/py_env-python3.10/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
                json_out['return_val'] = hook(**hook_input['kwargs'])
              File "/home/allen/.cache/pre-commit/repogp_qslvr/py_env-python3.10/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 261, in build_wheel
                return _build_backend().build_wheel(wheel_directory, config_settings,
              File "/tmp/pip-build-env-9o25i8wx/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 412, in build_wheel
                return self._build_with_temp_dir(['bdist_wheel'], '.whl',
              File "/tmp/pip-build-env-9o25i8wx/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 397, in _build_with_temp_dir
                self.run_setup()
              File "/tmp/pip-build-env-9o25i8wx/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 335, in run_setup
                exec(code, locals())
              File "<string>", line 32, in <module>
              File "/tmp/pip-build-env-9o25i8wx/overlay/lib/python3.10/site-packages/skbuild/setuptools_wrap.py", line 613, in setup
                cmkr = cmaker.CMaker(cmake_executable)
              File "/tmp/pip-build-env-9o25i8wx/overlay/lib/python3.10/site-packages/skbuild/cmaker.py", line 142, in __init__
                self.platform = get_platform()
              File "/tmp/pip-build-env-9o25i8wx/overlay/lib/python3.10/site-packages/skbuild/platform_specifics/platform_factory.py", line 40, in get_platform
                raise RuntimeError(msg)
            RuntimeError: Unsupported platform: mingw64_nt-10.0-19043. Please contact the scikit-build team.
            [end of output]

        note: This error originates from a subprocess, and is likely not a problem with pip.
        ERROR: Failed building wheel for ninja
      Failed to build ninja
      ERROR: Could not build wheels for ninja, which is required to install pyproject.toml-based projects
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Check the log at /home/allen/.cache/pre-commit/pre-commit.log

which then led me to trying python3 -m pip install ninja , as I mentioned in the initial post does python3 -m pip install ninja work for you?

winterz commented 1 year ago

From the OP, it seems that you are using msys2 python. Try to use the mingw one.

1. Open the blue 'M' icon in start menu.

2. Install required packages with pacman, for example
pacman -Syyu
pacman -S $MINGW_PACKAGE_PREFIX-python $MINGW_PACKAGE_PREFIX-python-pip $MINGW_PACKAGE_PREFIX-ninja
3. Now try your command.

4. Be aware that there are some known issues with mingw python vs official python, see https://www.msys2.org/docs/python/

Here is the output of pip command

Yes, that works. Can I get a package for $MINGW_PACKAGE_PREFIX-precommit ? I can open an new issue for a pip pre-commit package if you want.

Biswa96 commented 1 year ago

Yes, that works.

Did you mean that fixes the whole issue of ERROR: Failed building wheel for ninja ?

Can I get a package for $MINGW_PACKAGE_PREFIX-precommit ?

Possibly. But it also depends on other python packages which may require some time.

I can open an new issue for a pip pre-commit package if you want.

If you want please open the issue here https://github.com/msys2/MINGW-packages with package-request tag. This repository is just for the documentation and website.

winterz commented 1 year ago

I run pacman -S $MINGW_PACKAGE_PREFIX-python-pre-commit

and now I see the following problem. what am I doing wrong? do we need an msys python yaml too?

[534]<DESKTOP-PB0VF0C:~/projects/kddockwidgets> pre-commit run --all-files
[INFO] Installing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
An unexpected error has occurred: CalledProcessError: command: ('C:\\msys64\\mingw64\\bin/
python.EXE', '-mpip', 'install', '.')
return code: 1
expected return code: 0
stdout:
    Processing c:/users/allen/.cache/pre-commit/repo25y79o5g
      Installing build dependencies: started
      Installing build dependencies: finished with status 'done'
      Getting requirements to build wheel: started
      Getting requirements to build wheel: finished with status 'done'
      Preparing metadata (pyproject.toml): started
      Preparing metadata (pyproject.toml): finished with status 'done'
    Requirement already satisfied: toml in c:/msys64/mingw64/lib/python3.10/site-packages
(from pre-commit-hooks==3.2.0) (0.10.2)
    Collecting ruamel.yaml>=0.15
      Using cached ruamel.yaml-0.17.21-py3-none-any.whl (109 kB)
    Collecting ruamel.yaml.clib>=0.2.6
      Using cached ruamel.yaml.clib-0.2.6.tar.gz (180 kB)
      Installing build dependencies: started
      Installing build dependencies: finished with status 'done'
      Getting requirements to build wheel: started
      Getting requirements to build wheel: finished with status 'error'

stderr:
      error: subprocess-exited-with-error

      Getting requirements to build wheel did not run successfully.
      exit code: 1

      [2 lines of output]
      sys.argv ['setup.py', 'egg_info']
      test compiling C:/Users/allen/AppData/Local/Temp/tmp_ruamel_d5fbgato/test_ruamel_yam
l.c -> test_ruamel_yaml Exception: --plat-name must be one of ('win32', 'win-amd64', 'win-
arm32', 'win-arm64')
      [end of output]

      note: This error originates from a subprocess, and is likely not a problem with pip.
    error: subprocess-exited-with-error

    Getting requirements to build wheel did not run successfully.
    exit code: 1

    See above for output.

    note: This error originates from a subprocess, and is likely not a problem with pip.

Check the log at C:/Users/allen/.cache/pre-commit/pre-commit.log
Biswa96 commented 1 year ago

It seems that pre-commit tries to install some packages using pip and the compilation failed. If I install those from mingw packages i.e. python-ruamel-yaml and python-ruamel.yaml.clib I get another error

An unexpected error has occurred: AssertionError: BUG: expected environment for python to be healthy immediately after install, please open an issue describing your environment

more info:

virtualenv python version did not match created version:
- actual version: <<error retrieving version from C:/Users/x/.cache/pre-commit/repo6v3tvved/py_env-python3.10/Scripts/python.exe>>
- expected version: 3.10.7.final.0

I am not familiar with python in-depth and can not provide any solution for this issue. Others may help to fix the issue 🙇