Build failure on Windows

clin1234 commented 5 months ago

Description

Running the install command with 3.13 yielded the following

C:\Users\ךינשגכהד\src\pyvex>pip install . --verbose
Using pip 24.1 from C:\Users\ךינשגכהד\scoop\apps\python-alpha\current\Lib\site-packages\pip (python 3.13)
Processing c:\users\ךינשגכהד\src\pyvex
  Running command pip subprocess to install build dependencies
  Using pip 24.1 from C:\Users\ךינשגכהד\scoop\persist\python-alpha\Lib\site-packages\pip (python 3.13)
  Collecting setuptools>=59
    Obtaining dependency information for setuptools>=59 from https://files.pythonhosted.org/packages/55/b3/b3a3415d2debd837106ed417f8681d8af63037fed367fa1b85dbfef081f1/setuptools-70.1.0-py3-none-any.whl.metadata
    Using cached setuptools-70.1.0-py3-none-any.whl.metadata (6.0 kB)
  Collecting wheel
    Obtaining dependency information for wheel from https://files.pythonhosted.org/packages/7d/cd/d7460c9a869b16c3dd4e1e403cce337df165368c71d6af229a74699622ce/wheel-0.43.0-py3-none-any.whl.metadata
    Using cached wheel-0.43.0-py3-none-any.whl.metadata (2.2 kB)
  Collecting cffi>=1.0.3
    Using cached cffi-1.16.0.tar.gz (512 kB)
    Installing build dependencies: started
    Running command pip subprocess to install build dependencies
    Using pip 24.1 from C:\Users\ךינשגכהד\scoop\persist\python-alpha\Lib\site-packages\pip (python 3.13)
    Collecting setuptools>=66.1
      Obtaining dependency information for setuptools>=66.1 from https://files.pythonhosted.org/packages/55/b3/b3a3415d2debd837106ed417f8681d8af63037fed367fa1b85dbfef081f1/setuptools-70.1.0-py3-none-any.whl.metadata
      Using cached setuptools-70.1.0-py3-none-any.whl.metadata (6.0 kB)
    Using cached setuptools-70.1.0-py3-none-any.whl (882 kB)
    Installing collected packages: setuptools
    ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
    poetry 1.9.0.dev0 requires dulwich<0.22.0,>=0.21.2, but you have dulwich 0.22.1 which is incompatible.
    poetry 1.9.0.dev0 requires keyring<25.0.0,>=24.3.1, but you have keyring 25.2.1 which is incompatible.
    Successfully installed setuptools-70.1.0
    Installing build dependencies: finished with status 'done'
    Getting requirements to build wheel: started
    Running command Getting requirements to build wheel
    Traceback (most recent call last):
      File "C:\Users\ךינשגכהד\scoop\persist\python-alpha\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
        main()
        ~~~~^^
      File "C:\Users\ךינשגכהד\scoop\persist\python-alpha\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
        json_out['return_val'] = hook(**hook_input['kwargs'])
                                 ~~~~^^^^^^^^^^^^^^^^^^^^^^^^
      File "C:\Users\ךינשגכהד\scoop\persist\python-alpha\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 118, in get_requires_for_build_wheel
        return hook(config_settings)
      File "C:\Users\ךינשגכהד\AppData\Local\Temp\pip-build-env-35xs_87a\overlay\Lib\site-packages\setuptools\build_meta.py", line 327, in get_requires_for_build_wheel
        return self._get_build_requires(config_settings, requirements=[])
               ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "C:\Users\ךינשגכהד\AppData\Local\Temp\pip-build-env-35xs_87a\overlay\Lib\site-packages\setuptools\build_meta.py", line 297, in _get_build_requires
        self.run_setup()
        ~~~~~~~~~~~~~~^^
      File "C:\Users\ךינשגכהד\AppData\Local\Temp\pip-build-env-35xs_87a\overlay\Lib\site-packages\setuptools\build_meta.py", line 313, in run_setup
        exec(code, locals())
        ~~~~^^^^^^^^^^^^^^^^
      File "<string>", line 126, in <module>
      File "<string>", line 105, in uses_msvc
      File "C:\Users\ךינשגכהד\AppData\Local\Temp\pip-build-env-35xs_87a\overlay\Lib\site-packages\setuptools\_distutils\command\config.py", line 220, in try_compile
        self._compile(body, headers, include_dirs, lang)
        ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "C:\Users\ךינשגכהד\AppData\Local\Temp\pip-build-env-35xs_87a\overlay\Lib\site-packages\setuptools\_distutils\command\config.py", line 132, in _compile
        self.compiler.compile([src], include_dirs=include_dirs)
        ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "C:\Users\ךינשגכהד\AppData\Local\Temp\pip-build-env-35xs_87a\overlay\Lib\site-packages\setuptools\_distutils\_msvccompiler.py", line 343, in compile
        self.initialize()
        ~~~~~~~~~~~~~~~^^
      File "C:\Users\ךינשגכהד\AppData\Local\Temp\pip-build-env-35xs_87a\overlay\Lib\site-packages\setuptools\_distutils\_msvccompiler.py", line 253, in initialize
        vc_env = _get_vc_env(plat_spec)
      File "C:\Users\ךינשגכהד\AppData\Local\Temp\pip-build-env-35xs_87a\overlay\Lib\site-packages\setuptools\msvc.py", line 232, in msvc14_get_vc_env
        return _msvc14_get_vc_env(plat_spec)
      File "C:\Users\ךינשגכהד\AppData\Local\Temp\pip-build-env-35xs_87a\overlay\Lib\site-packages\setuptools\msvc.py", line 189, in _msvc14_get_vc_env
        raise distutils.errors.DistutilsPlatformError("Unable to find vcvarsall.bat")
    distutils.errors.DistutilsPlatformError: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
    error: subprocess-exited-with-error

    × Getting requirements to build wheel did not run successfully.
    │ exit code: 1
    ╰─> See above for output.

    note: This error originates from a subprocess, and is likely not a problem with pip.
    full command: 'C:\Users\ךינשגכהד\scoop\apps\python-alpha\current\python.exe' 'C:\Users\ךינשגכהד\scoop\persist\python-alpha\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py' get_requires_for_build_wheel 'C:\Users\0BAC~1\AppData\Local\Temp\tmpyei06lf0'
    cwd: C:\Users\ךינשגכהד\AppData\Local\Temp\pip-install-y8vid60m\cffi_80d4a805fc3147d8b6f4398a96d91d72
    Getting requirements to build wheel: finished with status 'error'
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  error: subprocess-exited-with-error

  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: 'C:\Users\ךינשגכהד\scoop\apps\python-alpha\current\python.exe' 'C:\Users\ךינשגכהד\scoop\persist\python-alpha\Lib\site-packages\pip\__pip-runner__.py' install --ignore-installed --no-user --prefix 'C:\Users\ךינשגכהד\AppData\Local\Temp\pip-build-env-y8ry2xx5\overlay' --no-warn-script-location -v --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=59' wheel 'cffi >= 1.0.3'
  cwd: [inherit]
  Installing build dependencies ... error
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Steps to reproduce the bug

No response

Environment

No response

Additional context

No response

rhelmot commented 5 months ago

https://visualstudio.microsoft.com/visual-cpp-build-tools/

clin1234 commented 5 months ago

But I have the build tools installed. In fact, I was running it from the Developer Command Prompt for VS 2022

rhelmot commented 5 months ago

Setuptools is reporting that it can't find vcvarsall.bat... I have a memory that microsoft stopped using vcvarsall.bat to set up its build environments, but I don't know why setuptools would be insisting on that file existing. Maybe try upgrading setuptools?

clin1234 commented 5 months ago

setuptools is already at the latest version: 70.1.0

clin1234 commented 5 months ago

~Tracking: https://github.com/pypa/setuptools/issues/4431~

clin1234 commented 5 months ago

Update:

"C:\Program Files\Microsoft Visual Studio\2022\Preview\VC\Tools\MSVC\14.41.33923\bin\HostX86\x64\link.exe" /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:C:\Users\ךינשגכהד\scoop\apps\python-alpha\current\libs /LIBPATH:C:\Users\ךינשגכהד\scoop\apps\python-alpha\current /LIBPATH:C:\Users\ךינשגכהד\scoop\apps\python-alpha\current\PCbuild\amd64 "/LIBPATH:C:\Program Files\Microsoft Visual Studio\2022\Preview\VC\Tools\MSVC\14.41.33923\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.26100.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\\lib\10.0.26100.0\\um\x64" "/LIBPATH:C:\Program Files\Microsoft Visual Studio\2022\Preview\VC\Tools\MSVC\14.41.33923\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.26100.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\\lib\10.0.26100.0\\um\x64" /EXPORT:PyInit__cffi_backend build\temp.win-amd64-cpython-313\Release\src/c/_cffi_backend.obj build\temp.win-amd64-cpython-313\Release\src/c/libffi_x86_x64\ffi.obj build\temp.win-amd64-cpython-313\Release\src/c/libffi_x86_x64\prep_cif.obj build\temp.win-amd64-cpython-313\Release\src/c/libffi_x86_x64\types.obj /OUT:build\lib.win-amd64-cpython-313\_cffi_backend.cp313-win_amd64.pyd /IMPLIB:build\temp.win-amd64-cpython-313\Release\src/c\_cffi_backend.cp313-win_amd64.lib src/c/libffi_x86_x64\win64.obj
               Creating library build\temp.win-amd64-cpython-313\Release\src/c\_cffi_backend.cp313-win_amd64.lib and object build\temp.win-amd64-cpython-313\Release\src/c\_cffi_backend.cp313-win_amd64.exp
            _cffi_backend.obj : error LNK2001: unresolved external symbol _PyErr_WriteUnraisableMsg
            build\lib.win-amd64-cpython-313\_cffi_backend.cp313-win_amd64.pyd : fatal error LNK1120: 1 unresolved externals
            error: command 'C:\\Program Files\\Microsoft Visual Studio\\2022\\Preview\\VC\\Tools\\MSVC\\14.41.33923\\bin\\HostX86\\x64\\link.exe' failed with exit code 1120
            [end of output]

        note: This error originates from a subprocess, and is likely not a problem with pip.
        ERROR: Failed building wheel for cffi
      Failed to build cffi
      ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (cffi)
      [end of output]

Fixed by bumping cffi to https://github.com/python-cffi/cffi/releases/tag/v1.17.0rc1

clin1234 commented 5 months ago

?

twizmwazin commented 5 months ago

If you have a pyvex issue please let us know, this issue appears resolved.

clin1234 commented 5 months ago

If you have a pyvex issue please let us know, this issue appears resolved.

I'm bewildered because the prerelease version of cffi hasn't been published to PyPI yet, as far as I can tell. Without bumping the cffi version in pyproject.toml, on 3.13, building from source will fail

twizmwazin commented 5 months ago

We don't support unreleased python versions. cffi has published their rc to PyPI but it is a pre-release so pip will select the latest stable instead. Once 3.13 is officially released and cffi's support is stable, we can revisit it, but I don't anticipate any changes will be needed on the pyvex side, as pip will automatically select the latest version of cffi anyways.

angr / pyvex