search

DiligentGraphics / DiligentTools

Utilities built on top of core module
Apache License 2.0
116 stars 71 forks source link

Python Dependency Management #214

Open alexk101 opened 8 months ago

alexk101 commented 8 months ago

Issue

This issue relates to how dependencies in the python portion of this project's build system are managed. This project makes the assumption that as long as there is an available python interpreter, that it is able to manage its dependencies through pip. This, however, isn't true on a number of linux distributions, namely arch linux and NixOS, which manage dependencies centrally through their os package manager. As a result, when attempting to build DiligentTools, the process fails since it is unable to successfully install libclang. Looking at the code causing this problem

https://github.com/DiligentGraphics/DiligentTools/blob/bb613678317149074b7e42fa2ce767fd25ac284f/RenderStateNotation/CMakeLists.txt#L29

we can see that pip is run on every build. Therefore, even if I manually install libclang through my distribution's package manager, it will still fail. I don't really expect platform specific code to accommodate such cases, though that would be nice, but doing a check if the python package even needs to be installed seems pretty sensible, not only in a usability case for such systems, but also for the sake of efficiency. This would at least allow me to manually install the package and continue the build process.

My personal recommendation would be something like this...

execute_process(
    COMMAND ${Python3_EXECUTABLE} -c "import clang"
    RESULT_VARIABLE EXIT_CODE
    OUTPUT_QUIET
)

execute_process(
    COMMAND ${Python3_EXECUTABLE} -c "import jinja2"
    RESULT_VARIABLE EXIT_CODE
    OUTPUT_QUIET
)

I include jinja as well, since it is required, though my build fails on libclang before even getting to jinja. Doing this and then running pip only if these check fail would allow for manual intervention of the user, whereas now, there is simply no way for me to build.

Debug Output

Click me ```cmake -- Fetching DiligentCore ... Target processor: x86_64 CMake generator: Ninja Target platform: Linux 64 D3D11_SUPPORTED: FALSE D3D12_SUPPORTED: FALSE GL_SUPPORTED: TRUE GLES_SUPPORTED: FALSE VULKAN_SUPPORTED: TRUE METAL_SUPPORTED: FALSE -- optimizer enabled CMake Deprecation Warning at build/debug/_deps/diligentcore-src/ThirdParty/SPIRV-Cross/CMakeLists.txt:22 (cmake_minimum_required): Compatibility with CMake < 3.5 will be removed from a future version of CMake. Update the VERSION argument value or use a ... suffix to tell CMake that the project does not need compatibility with older versions. -- SPIRV-Cross: Finding Git version for SPIRV-Cross. -- SPIRV-Cross: Git hash: 42aac916 -- xxHash build type: Debug -- Architecture: x86_64 -- DiligentCore commit hash: b0e438cf CMake Warning (dev) at /usr/share/cmake/Modules/FindOpenGL.cmake:381 (message): Policy CMP0072 is not set: FindOpenGL prefers GLVND by default when available. Run "cmake --help-policy CMP0072" for policy details. Use the cmake_policy command to set the policy and suppress this warning. FindOpenGL found both a legacy GL library: OPENGL_gl_LIBRARY: /usr/lib/libGL.so and GLVND libraries for OpenGL and GLX: OPENGL_opengl_LIBRARY: /usr/lib/libOpenGL.so OPENGL_glx_LIBRARY: /usr/lib/libGLX.so OpenGL_GL_PREFERENCE has not been set to "GLVND" or "LEGACY", so for compatibility with CMake 3.10 and below the legacy GL library will be used. Call Stack (most recent call first): build/debug/_deps/diligentcore-src/Graphics/GraphicsEngineOpenGL/CMakeLists.txt:202 (find_package) This warning is for project developers. Use -Wno-dev to suppress it. CMake Warning (dev) at /usr/share/cmake/Modules/FindOpenGL.cmake:381 (message): Policy CMP0072 is not set: FindOpenGL prefers GLVND by default when available. Run "cmake --help-policy CMP0072" for policy details. Use the cmake_policy command to set the policy and suppress this warning. FindOpenGL found both a legacy GL library: OPENGL_gl_LIBRARY: /usr/lib/libGL.so and GLVND libraries for OpenGL and GLX: OPENGL_opengl_LIBRARY: /usr/lib/libOpenGL.so OPENGL_glx_LIBRARY: /usr/lib/libGLX.so OpenGL_GL_PREFERENCE has not been set to "GLVND" or "LEGACY", so for compatibility with CMake 3.10 and below the legacy GL library will be used. Call Stack (most recent call first): build/debug/_deps/diligentcore-src/Graphics/Archiver/CMakeLists.txt:136 (find_package) This warning is for project developers. Use -Wno-dev to suppress it. -- Fetching DiligentTools ... CMake Deprecation Warning at build/debug/_deps/diligenttools-src/ThirdParty/libpng/CMakeLists.txt:82 (message): The option PNG_EXECUTABLES has been deprecated in favour of PNG_TOOLS CMake Warning (dev) at build/debug/_deps/diligenttools-src/ThirdParty/libpng/CMakeLists.txt:84 (message): Setting PNG_TOOLS to OFF, to stay compatible with PNG_EXECUTABLES This warning is for project developers. Use -Wno-dev to suppress it. -- Symbol prefix: CMake Warning (dev) at /usr/share/cmake/Modules/FindOpenGL.cmake:381 (message): Policy CMP0072 is not set: FindOpenGL prefers GLVND by default when available. Run "cmake --help-policy CMP0072" for policy details. Use the cmake_policy command to set the policy and suppress this warning. FindOpenGL found both a legacy GL library: OPENGL_gl_LIBRARY: /usr/lib/libGL.so and GLVND libraries for OpenGL and GLX: OPENGL_opengl_LIBRARY: /usr/lib/libOpenGL.so OPENGL_glx_LIBRARY: /usr/lib/libGLX.so OpenGL_GL_PREFERENCE has not been set to "GLVND" or "LEGACY", so for compatibility with CMake 3.10 and below the legacy GL library will be used. Call Stack (most recent call first): build/debug/_deps/diligenttools-src/NativeApp/CMakeLists.txt:472 (find_package) This warning is for project developers. Use -Wno-dev to suppress it. error: externally-managed-environment × This environment is externally managed ╰─> To install Python packages system-wide, try 'pacman -S python-xyz', where xyz is the package you are trying to install. If you wish to install a non-Arch-packaged Python package, create a virtual environment using 'python -m venv path/to/venv'. Then use path/to/venv/bin/python and path/to/venv/bin/pip. If you wish to install a non-Arch packaged Python application, it may be easiest to use 'pipx install xyz', which will manage a virtual environment for you. Make sure you have python-pipx installed via pacman. note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages. hint: See PEP 668 for the detailed specification. CMake Error at build/debug/_deps/diligenttools-src/RenderStateNotation/CMakeLists.txt:40 (message): Command '/bin/python3.11;-m;pip;install;libclang==16.0.6' failed with error code 1 } ```

Additional Info