search

fbertola / bgfx-python

Python 3.7+ wrapper for the BGFX library. 🐍
BSD 2-Clause "Simplified" License
122 stars 6 forks source link

Unable to install on windows #19

Open ompadu opened 3 years ago

ompadu commented 3 years ago

edit: Might be related to #8, missing binary wheels for python 3.9 and it tries to build it from source.

original post: Ran the following command pip install bgfx-python==1.0.4 --user in the VS2019 command prompt with the output (replaced my actual user name with <user_name>):

Collecting bgfx-python==1.0.4
  Using cached bgfx-python-1.0.4.tar.gz (93 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting loguru
  Using cached loguru-0.5.3-py3-none-any.whl (57 kB)
Requirement already satisfied: colorama>=0.3.4 in c:\users\<user_name>\appdata\roaming\python\python39\site-packages (from loguru->bgfx-python==1.0.4) (0.4.4)
Collecting win32-setctime>=1.0.0
  Using cached win32_setctime-1.0.3-py3-none-any.whl (3.5 kB)
Building wheels for collected packages: bgfx-python
  Building wheel for bgfx-python (PEP 517) ... error
  ERROR: Command errored out with exit status 1:
   command: 'c:\users\<user_name>\scoop\apps\python\current\python.exe' 'C:\Users\<user_name>\AppData\Roaming\Python\Python39\site-packages\pip\_vendor\pep517\_in_process.py' build_wheel 'C:\Users\<user_name>\AppData\Local\Temp\tmp2tjh_ayy'
       cwd: C:\Users\<user_name>\AppData\Local\Temp\pip-install-_fsm2ab_\bgfx-python_c762d88519934ecdbe890d32471c95a5
  Complete output (152 lines):
  Not searching for unused variables given on the command line.
  -- The C compiler identification is MSVC 19.28.29335.0
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.28.29333/bin/Hostx64/x64/cl.exe - skipped
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- The CXX compiler identification is MSVC 19.28.29335.0
  CMake Warning (dev) at C:/Users/<user_name>/AppData/Local/Temp/pip-build-env-4lzdyvwp/overlay/Lib/site-packages/cmake/data/share/cmake-3.18/Modules/CMakeDetermineCXXCompiler.cmake:157 (if):
    Policy CMP0054 is not set: Only interpret if() arguments as variables or
    keywords when unquoted.  Run "cmake --help-policy CMP0054" for policy
    details.  Use the cmake_policy command to set the policy and suppress this
    warning.

    Quoted variables like "MSVC" will no longer be dereferenced when the policy
    is set to NEW.  Since the policy is not set the OLD behavior will be used.
  Call Stack (most recent call first):
    CMakeLists.txt:4 (ENABLE_LANGUAGE)
  This warning is for project developers.  Use -Wno-dev to suppress it.

  CMake Warning (dev) at C:/Users/<user_name>/AppData/Local/Temp/pip-build-env-4lzdyvwp/overlay/Lib/site-packages/cmake/data/share/cmake-3.18/Modules/CMakeDetermineCXXCompiler.cmake:177 (elseif):
    Policy CMP0054 is not set: Only interpret if() arguments as variables or
    keywords when unquoted.  Run "cmake --help-policy CMP0054" for policy
    details.  Use the cmake_policy command to set the policy and suppress this
    warning.

    Quoted variables like "MSVC" will no longer be dereferenced when the policy
    is set to NEW.  Since the policy is not set the OLD behavior will be used.
  Call Stack (most recent call first):
    CMakeLists.txt:4 (ENABLE_LANGUAGE)
  This warning is for project developers.  Use -Wno-dev to suppress it.

  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.28.29333/bin/Hostx64/x64/cl.exe - skipped
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  -- Configuring done
  -- Generating done
  -- Build files have been written to: C:/Users/<user_name>/AppData/Local/Temp/pip-install-_fsm2ab_/bgfx-python_c762d88519934ecdbe890d32471c95a5/_cmake_test_compile/build
  -- The C compiler identification is MSVC 19.28.29335.0
  -- The CXX compiler identification is MSVC 19.28.29335.0
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.28.29333/bin/Hostx64/x64/cl.exe - skipped
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.28.29333/bin/Hostx64/x64/cl.exe - skipped
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  -- Found Python3: C:/Users/<user_name>/scoop/apps/python/current/libs/python39.lib (found version "3.9.1") found components: Development Development.Module Development.Embed
  -- Found PythonInterp: c:/users/<user_name>/scoop/apps/python/current/python.exe (found version "3.9.1")
  -- Found PythonLibs: c:/users/<user_name>/scoop/apps/python/current/libs/python39.lib (found version "3.9.1")
  -- PYTHON_LIBRARIES = c:/users/<user_name>/scoop/apps/python/current/libs/python39.lib
  -- PYTHON_INCLUDE_DIRS = c:/users/<user_name>/scoop/apps/python/current/Include
  -- BX_DIR = C:/Users/<user_name>/AppData/Local/Temp/pip-install-_fsm2ab_/bgfx-python_c762d88519934ecdbe890d32471c95a5/src/../dependencies/bx
  -- BIMG_DIR = C:/Users/<user_name>/AppData/Local/Temp/pip-install-_fsm2ab_/bgfx-python_c762d88519934ecdbe890d32471c95a5/src/../dependencies/bimg
  -- BGFX_DIR = C:/Users/<user_name>/AppData/Local/Temp/pip-install-_fsm2ab_/bgfx-python_c762d88519934ecdbe890d32471c95a5/src/../dependencies/bgfx
  -- PYBIND11_DIR = C:/Users/<user_name>/AppData/Local/Temp/pip-install-_fsm2ab_/bgfx-python_c762d88519934ecdbe890d32471c95a5/src/../dependencies/pybind11
  -- FindWindowsSDK: Detected Visual Studio 2012 or newer, not using the _xp toolset variant: including SDK versions that drop XP support in search!
  -- Found WindowsSDK: C:/Program Files (x86)/Windows Kits/10
  'cmd' '/c' 'C:/Users/<user_name>/AppData/Local/Temp/pip-install-_fsm2ab_/bgfx-python_c762d88519934ecdbe890d32471c95a5/src/scripts/rename_runtime.bat' 'C:/Users/<user_name>/AppData/Local/Temp/pip-install-_fsm2ab_/bgfx-python_c762d88519934ecdbe890d32471c95a5/src/../dependencies/bgfx/.build/projects/vs2019'
  dir : Cannot find path 'C:\Users\<user_name>\AppData\Local\Temp\pip-install-_fsm2ab_\bgfx-python_c762d88519934ecdbe890d32471c9
  5a5\dependencies\bgfx\.build\projects\' because it does not exist.
  At line:1 char:1
  + dir -Path C:/Users/<user_name>/AppData/Local/Temp/pip-install-_fsm2ab_/bgfx- ...
  + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      + CategoryInfo          : ObjectNotFound: (C:\Users\<user_name>\A...build\projects\:String) [Get-ChildItem], ItemNotFound
     Exception
      + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetChildItemCommand

  Microsoft (R) Build Engine version 16.8.2+25e4d540b for .NET Framework
  Copyright (C) Microsoft Corporation. All rights reserved.

  MSBUILD : error MSB1009: Project file does not exist.
  Switch: bgfx.sln
  -- Performing Test Weak Link MODULE -> SHARED (gnu_ld_ignore) - Failed
  -- Performing Test Weak Link MODULE -> SHARED (osx_dynamic_lookup) - Failed
  -- Performing Test Weak Link MODULE -> SHARED (no_flag) - Failed
  _modinit_prefix:PyInit_
  _modinit_prefix:PyInit_
  -- Configuring done
  CMake Error at CMakeLists.txt:79 (add_library):
    Cannot find source file:

      C:/Users/<user_name>/AppData/Local/Temp/pip-install-_fsm2ab_/bgfx-python_c762d88519934ecdbe890d32471c95a5/dependencies/bgfx/3rdparty/dear-imgui/imgui.cpp

    Tried extensions .c .C .c++ .cc .cpp .cxx .cu .m .M .mm .h .hh .h++ .hm
    .hpp .hxx .in .txx

  CMake Error at CMakeLists.txt:93 (add_library):
    Cannot find source file:

      C:/Users/<user_name>/AppData/Local/Temp/pip-install-_fsm2ab_/bgfx-python_c762d88519934ecdbe890d32471c95a5/dependencies/bgfx/tools/shaderc/shaderc_glsl.cpp

    Tried extensions .c .C .c++ .cc .cpp .cxx .cu .m .M .mm .h .hh .h++ .hm
    .hpp .hxx .in .txx

  CMake Error at CMakeLists.txt:79 (add_library):
    No SOURCES given to target: bgfx_lib

  CMake Error at CMakeLists.txt:93 (add_library):
    No SOURCES given to target: shaderc

  CMake Generate step failed.  Build files cannot be regenerated correctly.
    File "C:\Users\<user_name>\AppData\Local\Temp\pip-build-env-4lzdyvwp\overlay\Lib\site-packages\skbuild\setuptools_wrap.py", line 582, in setup
      env = cmkr.configure(cmake_args,
    File "C:\Users\<user_name>\AppData\Local\Temp\pip-build-env-4lzdyvwp\overlay\Lib\site-packages\skbuild\cmaker.py", line 229, in configure
      raise SKBuildError(

  --------------------------------------------------------------------------------
  -- Trying "Ninja (Visual Studio 15 2017 Win64 v141)" generator
  --------------------------------
  ---------------------------
  ----------------------
  -----------------
  ------------
  -------
  --
  --
  -------
  ------------
  -----------------
  ----------------------
  ---------------------------
  --------------------------------
  -- Trying "Ninja (Visual Studio 15 2017 Win64 v141)" generator - success
  --------------------------------------------------------------------------------

  Configuring Project
    Working directory:
      C:\Users\<user_name>\AppData\Local\Temp\pip-install-_fsm2ab_\bgfx-python_c762d88519934ecdbe890d32471c95a5\_skbuild\win-amd64-3.9\cmake-build
    Command:
      cmake 'C:\Users\<user_name>\AppData\Local\Temp\pip-install-_fsm2ab_\bgfx-python_c762d88519934ecdbe890d32471c95a5\src' -G Ninja '-DCMAKE_INSTALL_PREFIX:PATH=C:\Users\<user_name>\AppData\Local\Temp\pip-install-_fsm2ab_\bgfx-python_c762d88519934ecdbe890d32471c95a5\_skbuild\win-amd64-3.9\cmake-install' '-DPYTHON_EXECUTABLE:FILEPATH=c:\users\<user_name>\scoop\apps\python\current\python.exe' -DPYTHON_VERSION_STRING:STRING=3.9.1 '-DPYTHON_INCLUDE_DIR:PATH=c:\users\<user_name>\scoop\apps\python\current\Include' '-DPYTHON_LIBRARY:FILEPATH=c:\users\<user_name>\scoop\apps\python\current\libs\python39.lib' -DSKBUILD:BOOL=TRUE '-DCMAKE_MODULE_PATH:PATH=C:\Users\<user_name>\AppData\Local\Temp\pip-build-env-4lzdyvwp\overlay\Lib\site-packages\skbuild\resources\cmake' -DCMAKE_BUILD_TYPE:STRING=Release

  Traceback (most recent call last):

  An error occurred while configuring with CMake.
    Command:
      cmake 'C:\Users\<user_name>\AppData\Local\Temp\pip-install-_fsm2ab_\bgfx-python_c762d88519934ecdbe890d32471c95a5\src' -G Ninja '-DCMAKE_INSTALL_PREFIX:PATH=C:\Users\<user_name>\AppData\Local\Temp\pip-install-_fsm2ab_\bgfx-python_c762d88519934ecdbe890d32471c95a5\_skbuild\win-amd64-3.9\cmake-install' '-DPYTHON_EXECUTABLE:FILEPATH=c:\users\<user_name>\scoop\apps\python\current\python.exe' -DPYTHON_VERSION_STRING:STRING=3.9.1 '-DPYTHON_INCLUDE_DIR:PATH=c:\users\<user_name>\scoop\apps\python\current\Include' '-DPYTHON_LIBRARY:FILEPATH=c:\users\<user_name>\scoop\apps\python\current\libs\python39.lib' -DSKBUILD:BOOL=TRUE '-DCMAKE_MODULE_PATH:PATH=C:\Users\<user_name>\AppData\Local\Temp\pip-build-env-4lzdyvwp\overlay\Lib\site-packages\skbuild\resources\cmake' -DCMAKE_BUILD_TYPE:STRING=Release
    Source directory:
      C:\Users\<user_name>\AppData\Local\Temp\pip-install-_fsm2ab_\bgfx-python_c762d88519934ecdbe890d32471c95a5\src
    Working directory:
      C:\Users\<user_name>\AppData\Local\Temp\pip-install-_fsm2ab_\bgfx-python_c762d88519934ecdbe890d32471c95a5\_skbuild\win-amd64-3.9\cmake-build
  Please see CMake's output for more information.
  ----------------------------------------
  ERROR: Failed building wheel for bgfx-python
Failed to build bgfx-python
ERROR: Could not build wheels for bgfx-python which use PEP 517 and cannot be installed directly
fbertola commented 3 years ago

Hello @ompadu ! Yes I faced a major roadblock with pypi not accepting the wheels due to a file size limit. I requested a limit increase but so far no response. In the meantime, try building from source cloning the repository and its submodules.

I'm in the process to rewrite a major portion of the c++ bindings with cppyy. When its ready I'll try to release wheels for Python 3.9 and deprecate 3.6.

Thanks for your patience.

ompadu commented 3 years ago

Thank you for the answer. No problem, I am in no big rush,

JackWitherell commented 3 years ago

Hate to bump this when it's known that something already needs to be fixed, but I wanted to mention that it's not possible to build the repo and it's modules using the readme. Hoping to help someone or possibly get someone up to my current block to look further into it.

Unclear 1

The readme suggests to run pip setup.py install which isn't correct at all, the correct statement should be python setup.py install. This is because setup.py can't be used as a parameter for pip.

Building Dependency 1

On attempting to install, however, I get an error in setup.py stating that there's no module named skbuild. scikit-build is a dependency on setup.py and isn't installed by default.

Unclear 2

After attempting this however, I get issues with skbuild, which states that there's a problem with cmake on my machine. I installed cmake from chocolatey, which wasn't directly suggested in the readme but is an easy mistake to make as the readme states to use chocolatey to install an out of date version of visual studio.

it should be made clear in the readme that the versions of cmake, make and ninja that are installed should be installed through pip, not chocolatey.

Unclear 3

After correcting the issue however, scikit-build makes it clear that there's no generator - hence the readme stating to install visual studio 2017 and to call vcvarsall.bat. However, after installing vs2017 community with chocolatey and attempting to call the vc vars setup script, it's suggested that it doesn't exist. Looking into it, vcvarsall.bat never existed in vs2017 and is a vs2015 only feature. The suggestion for python users is to... use a wheel instead.

In lieu of a functioning wheel, I started trying to find the alternative command for this, which is call "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\Tools\VsDevCmd.bat" -arch=x64 to set up these vars.

This functions properly... until I try building the repo.

Building Dependency 2

Attempting this causes tons of CMake errors due to not having the Windows 8.1 SDK. Yikes. Not a good look.

Building Dependency 3

After installing the Windows 8.1 SDK through the visual studio installer... LibClang can't be found. This is apparently needed to build cppyy. Instead of trying to figure out what wonky dev setup allows LibClang to be pulled out of thin air, I ended up running pip install cppyy to intervene.

Misconfiguration?

After that, I got an error from ninja stating that dependencies/bgfx/.build/win64_vs2017/bin/bimgRelease.lib is needed. The dependencies tree while building doesn't include this but includes one for win32 instead. Either this is a misconfiguration in the CMakeLists file or further clueing is needed to get the 64 bit version of bimgRelease.lib to compile. My fix for this was to change the version that is actually used by cmake in cmakelists.txt (changed this to win32) as I don't know how to force the win64 version to build instead. I think this is due to the 64 bit version being the only one supported for a while, with 32 bit releasing later and becoming the default. https://github.com/fbertola/bgfx-python/blob/f83c43c37362b9462abd4b277c13f24477e3e63e/src/CMakeLists.txt#L38

Blocked

And lastly, this is where I hit a hard wall.

Ninja started failing due to an improperly formatted command generated by FindCppyy.cmake. It's very clear that this cmake file was never meant to be used in its current state, especially since it's located in the experimental folder of the official ROOT repo for Python's rootcling implementation: https://github.com/eric-erki/The-official-repository-for-ROOT/blob/7e47c421a88fdd75b7dce058ece9b4f31135b41b/bindings/pyroot_experimental/cppyy/cppyy-backend/cling/python/cppyy_backend/cmake/FindCppyy.cmake

TL;DR

There's a lot more prerequisites to building, most of which have come out morphing visual studio distributions and bad recommendations on behalf of building this from source. As the only other set of recommended python bindings for BGFX are also practically non-functional, we're left with kinda a hot mess of no real python support in bgfx.

I'd be interested to know where we're at, if anywhere on the proposed rewrite. I think it's a really bad look for BGFX to have this listed on their page and have it be practically nonfunctional.