search

mitsuba-renderer / mitsuba3

Mitsuba 3: A Retargetable Forward and Inverse Renderer
https://www.mitsuba-renderer.org/
Other
2.1k stars 246 forks source link

pip install latest commit on main #1302

Closed tomas16 closed 2 months ago

tomas16 commented 2 months ago

Summary

Some fixes were added to mitsuba recently, and I'd like to start using them without waiting for a new official release.

Mitsuba is a dependency of a project I'm working on, so I modified its pip requirements.txt file to include

mitsuba @ git+https://github.com/mitsuba-renderer/mitsuba3.git

During installation, mitsuba fails to build.

This issue is probably related to: https://github.com/mitsuba-renderer/mitsuba3/issues/1147 https://github.com/mitsuba-renderer/mitsuba3/issues/1293

It would be great to fix the pip build, but I'm open to other suggestions for being able to use the latest commit of mitsuba.

System configuration

System information:

OS: macOS 14.6.1 CPU: Intel Core i-9 GPU: AMD Radeon Pro 5500M Python version: 3.11.9 LLVM version: 16 CUDA version: n/a NVidia driver: n/a

Description

Running pip install on the project that has mitsuba @ git+https://github.com/mitsuba-renderer/mitsuba3.git in its requirements file produces the following output:

  Building wheel for mitsuba (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for mitsuba (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [94 lines of output]

      --------------------------------------------------------------------------------
      -- Trying 'Ninja' generator
      --------------------------------
      ---------------------------
      ----------------------
      -----------------
      ------------
      -------
      --
      Not searching for unused variables given on the command line.
      -- The C compiler identification is AppleClang 15.0.0.15000309
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- The CXX compiler identification is AppleClang 15.0.0.15000309
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- Configuring done (4.6s)
      -- Generating done (0.0s)
      -- Build files have been written to: /private/var/folders/vy/6xmdzv1x2d5dxk093pz0lf240000gn/T/pip-install-e69l1qgu/mitsuba_0ae8b92a4ab14ebb9578159b7af30010/_cmake_test_compile/build
      --
      -------
      ------------
      -----------------
      ----------------------
      ---------------------------
      --------------------------------
      -- Trying 'Ninja' generator - success
      --------------------------------------------------------------------------------

      Configuring Project
        Working directory:
          /private/var/folders/vy/6xmdzv1x2d5dxk093pz0lf240000gn/T/pip-install-e69l1qgu/mitsuba_0ae8b92a4ab14ebb9578159b7af30010/_skbuild/macosx-14.0-x86_64-3.11/cmake-build
        Command:
          /private/var/folders/vy/6xmdzv1x2d5dxk093pz0lf240000gn/T/pip-build-env-otx13rj5/overlay/lib/python3.11/site-packages/cmake/data/bin/cmake /private/var/folders/vy/6xmdzv1x2d5dxk093pz0lf240000gn/T/pip-install-e69l1qgu/mitsuba_0ae8b92a4ab14ebb9578159b7af30010 -G Ninja -DCMAKE_MAKE_PROGRAM:FILEPATH=/private/var/folders/vy/6xmdzv1x2d5dxk093pz0lf240000gn/T/pip-build-env-otx13rj5/overlay/lib/python3.11/site-packages/ninja/data/bin/ninja --no-warn-unused-cli -DCMAKE_INSTALL_PREFIX:PATH=/private/var/folders/vy/6xmdzv1x2d5dxk093pz0lf240000gn/T/pip-install-e69l1qgu/mitsuba_0ae8b92a4ab14ebb9578159b7af30010/_skbuild/macosx-14.0-x86_64-3.11/cmake-install -DPYTHON_VERSION_STRING:STRING=3.11.9 -DSKBUILD:INTERNAL=TRUE -DCMAKE_MODULE_PATH:PATH=/private/var/folders/vy/6xmdzv1x2d5dxk093pz0lf240000gn/T/pip-build-env-otx13rj5/overlay/lib/python3.11/site-packages/skbuild/resources/cmake -DPYTHON_EXECUTABLE:PATH=/Users/tomas/mambaforge/envs/py311/bin/python3.11 -DPYTHON_INCLUDE_DIR:PATH=/Users/tomas/mambaforge/envs/py311/include/python3.11 -DPYTHON_LIBRARY:PATH=/Users/tomas/mambaforge/envs/py311/lib/libpython3.11.dylib -DPython_EXECUTABLE:PATH=/Users/tomas/mambaforge/envs/py311/bin/python3.11 -DPython_ROOT_DIR:PATH=/Users/tomas/mambaforge/envs/py311 -DPython_FIND_REGISTRY:STRING=NEVER -DPython_INCLUDE_DIR:PATH=/Users/tomas/mambaforge/envs/py311/include/python3.11 -DPython3_EXECUTABLE:PATH=/Users/tomas/mambaforge/envs/py311/bin/python3.11 -DPython3_ROOT_DIR:PATH=/Users/tomas/mambaforge/envs/py311 -DPython3_FIND_REGISTRY:STRING=NEVER -DPython3_INCLUDE_DIR:PATH=/Users/tomas/mambaforge/envs/py311/include/python3.11 -DCMAKE_MAKE_PROGRAM:FILEPATH=/private/var/folders/vy/6xmdzv1x2d5dxk093pz0lf240000gn/T/pip-build-env-otx13rj5/overlay/lib/python3.11/site-packages/ninja/data/bin/ninja -DCMAKE_INSTALL_LIBDIR=mitsuba -DCMAKE_INSTALL_BINDIR=mitsuba -DCMAKE_INSTALL_INCLUDEDIR=mitsuba/include -DCMAKE_INSTALL_DATAROOTDIR=mitsuba/data -DCMAKE_TOOLCHAIN_FILE= -DMI_DRJIT_CMAKE_DIR:STRING= -DMI_SRGB_COEFF_FILE:STRING= -DMI_PYTHON_STUBS_DIR:STRING= -DCMAKE_OSX_DEPLOYMENT_TARGET:STRING=14.0 -DCMAKE_OSX_ARCHITECTURES:STRING=x86_64 -DCMAKE_BUILD_TYPE:STRING=Release

      Not searching for unused variables given on the command line.
      -- The CXX compiler identification is AppleClang 15.0.0.15000309
      -- The C compiler identification is AppleClang 15.0.0.15000309
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- Mitsuba v3.5.0
      -- Mitsuba: setting portable compilation defaults for scikit-build.
      -- Mitsuba: using libc++.
      -- Found Python: /Users/tomas/mambaforge/envs/py311/bin/python3.11 (found suitable version "3.11.9", minimum required is "3.8") found components: Interpreter Development.Module Development.SABIModule
      '/Users/tomas/mambaforge/envs/py311/bin/python3.11' '-m' 'nanobind' '--cmake_dir'
      /Users/tomas/mambaforge/envs/py311/bin/python3.11: No module named nanobind
      CMake Error at ext/drjit/ext/drjit-core/ext/nanothread/ext/cmake-defaults/CMakeLists.txt:295 (find_package):
        Could not find a package configuration file provided by "nanobind" with any
        of the following names:

          nanobindConfig.cmake
          nanobind-config.cmake

        Add the installation prefix of "nanobind" to CMAKE_PREFIX_PATH or set
        "nanobind_DIR" to a directory containing one of the above files.  If
        "nanobind" provides a separate development package or SDK, be sure it has
        been installed.
      Call Stack (most recent call first):
        CMakeLists.txt:80 (include)

      -- Configuring incomplete, errors occurred!
      Traceback (most recent call last):
        File "/private/var/folders/vy/6xmdzv1x2d5dxk093pz0lf240000gn/T/pip-build-env-otx13rj5/overlay/lib/python3.11/site-packages/skbuild/setuptools_wrap.py", line 660, in setup
          env = cmkr.configure(
                ^^^^^^^^^^^^^^^
        File "/private/var/folders/vy/6xmdzv1x2d5dxk093pz0lf240000gn/T/pip-build-env-otx13rj5/overlay/lib/python3.11/site-packages/skbuild/cmaker.py", line 354, in configure
          raise SKBuildError(msg)

      An error occurred while configuring with CMake.
        Command:
          /private/var/folders/vy/6xmdzv1x2d5dxk093pz0lf240000gn/T/pip-build-env-otx13rj5/overlay/lib/python3.11/site-packages/cmake/data/bin/cmake /private/var/folders/vy/6xmdzv1x2d5dxk093pz0lf240000gn/T/pip-install-e69l1qgu/mitsuba_0ae8b92a4ab14ebb9578159b7af30010 -G Ninja -DCMAKE_MAKE_PROGRAM:FILEPATH=/private/var/folders/vy/6xmdzv1x2d5dxk093pz0lf240000gn/T/pip-build-env-otx13rj5/overlay/lib/python3.11/site-packages/ninja/data/bin/ninja --no-warn-unused-cli -DCMAKE_INSTALL_PREFIX:PATH=/private/var/folders/vy/6xmdzv1x2d5dxk093pz0lf240000gn/T/pip-install-e69l1qgu/mitsuba_0ae8b92a4ab14ebb9578159b7af30010/_skbuild/macosx-14.0-x86_64-3.11/cmake-install -DPYTHON_VERSION_STRING:STRING=3.11.9 -DSKBUILD:INTERNAL=TRUE -DCMAKE_MODULE_PATH:PATH=/private/var/folders/vy/6xmdzv1x2d5dxk093pz0lf240000gn/T/pip-build-env-otx13rj5/overlay/lib/python3.11/site-packages/skbuild/resources/cmake -DPYTHON_EXECUTABLE:PATH=/Users/tomas/mambaforge/envs/py311/bin/python3.11 -DPYTHON_INCLUDE_DIR:PATH=/Users/tomas/mambaforge/envs/py311/include/python3.11 -DPYTHON_LIBRARY:PATH=/Users/tomas/mambaforge/envs/py311/lib/libpython3.11.dylib -DPython_EXECUTABLE:PATH=/Users/tomas/mambaforge/envs/py311/bin/python3.11 -DPython_ROOT_DIR:PATH=/Users/tomas/mambaforge/envs/py311 -DPython_FIND_REGISTRY:STRING=NEVER -DPython_INCLUDE_DIR:PATH=/Users/tomas/mambaforge/envs/py311/include/python3.11 -DPython3_EXECUTABLE:PATH=/Users/tomas/mambaforge/envs/py311/bin/python3.11 -DPython3_ROOT_DIR:PATH=/Users/tomas/mambaforge/envs/py311 -DPython3_FIND_REGISTRY:STRING=NEVER -DPython3_INCLUDE_DIR:PATH=/Users/tomas/mambaforge/envs/py311/include/python3.11 -DCMAKE_MAKE_PROGRAM:FILEPATH=/private/var/folders/vy/6xmdzv1x2d5dxk093pz0lf240000gn/T/pip-build-env-otx13rj5/overlay/lib/python3.11/site-packages/ninja/data/bin/ninja -DCMAKE_INSTALL_LIBDIR=mitsuba -DCMAKE_INSTALL_BINDIR=mitsuba -DCMAKE_INSTALL_INCLUDEDIR=mitsuba/include -DCMAKE_INSTALL_DATAROOTDIR=mitsuba/data -DCMAKE_TOOLCHAIN_FILE= -DMI_DRJIT_CMAKE_DIR:STRING= -DMI_SRGB_COEFF_FILE:STRING= -DMI_PYTHON_STUBS_DIR:STRING= -DCMAKE_OSX_DEPLOYMENT_TARGET:STRING=14.0 -DCMAKE_OSX_ARCHITECTURES:STRING=x86_64 -DCMAKE_BUILD_TYPE:STRING=Release
        Source directory:
          /private/var/folders/vy/6xmdzv1x2d5dxk093pz0lf240000gn/T/pip-install-e69l1qgu/mitsuba_0ae8b92a4ab14ebb9578159b7af30010
        Working directory:
          /private/var/folders/vy/6xmdzv1x2d5dxk093pz0lf240000gn/T/pip-install-e69l1qgu/mitsuba_0ae8b92a4ab14ebb9578159b7af30010/_skbuild/macosx-14.0-x86_64-3.11/cmake-build
      Please see CMake's output for more information.

      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for mitsuba

At the time of writing, mitsuba's main branch is at 96e0af2d.

njroussel commented 2 months ago

Hi @tomas16

These insctructions are still valid and are still the recommended way of using a local build/fork of the project: https://mitsuba.readthedocs.io/en/latest/src/developer_guide/compiling.html

tomas16 commented 2 months ago

Hi @njroussel, thanks for the feedback. My goal is slightly different from simply compiling mitsuba though: I want to distribute an unreleased (but non-customized) version of mitsuba to users, as a dependency of another project.

Do I understand correctly that I should basically do a custom build and host the wheels somewhere myself? (e.g. my own pip server)

tomas16 commented 2 months ago

Before I embark on that: do you have a rough idea when you guys will release a next official version?

njroussel commented 2 months ago

Hi @tomas16

I want to distribute an unreleased (but non-customized) version of mitsuba to users, as a dependency of another project.

Aha. In that case, the best solution would indeed be to self-host your custom built wheels.

Before I embark on that: do you have a rough idea when you guys will release a next official version?

I'd say 2-3 weeks if all goes smoothly.