cannot pip install custom built version of mitsuba

[msqrt]

I'm on windows and would like to build a custom version of mitsuba with a couple of small changes. The rest of my workflow relies on anaconda, so I'd ideally like to just pip install the custom-built mitsuba, but this doesn't seem to work out of the box. To demonstrate, if I use the following env.yml that pulls the latest mitsuba and tries to install it with pip:

name: test
channels:
  - defaults
dependencies:
  - python = 3.10
  - pip
  - pip:
    - git+https://github.com/mitsuba-renderer/mitsuba3

I get a long error message with the apparent key line ImportError: pybind11::detail::get_type_info: unable to find type info for "drjit::Array<float,0>" resulting from the last build step. Building mitsuba according to the instructions from the documentation works fine, but doesn't seem to produce anything I could directly install with pip. Everything works if I run python from the mitsuba3/build/Release/python folder created by the build, but I'd like to have this be part of my conda environment. My attempts at creating a custom setup.py or trying to build a wheel have only lead to more trouble. So, what should I do?

[msqrt]

As it tends to go, I found a solution now that I've asked about it. The following setup.py in the mitsuba3/build/Release/python folder allows doing pip install . after which the custom-built version is properly copied into the environment:

from setuptools import setup
setup (
    name="mitsuba",
    version = "0.4.3",
    author="",
    author_email="",
    description = "",
    packages=['drjit', 'mitsuba'],
    python_requires=">=3.8",
    package_data={'': ['**']},
    include_package_data=True,
)

What I was previously missing was the package_data part; the python code was copied but all the dlls and such where missing.

Even though it works, I feel like I'm doing something weird and clunky. Is there a better way to do this?

[njroussel]

Hi @msqrt

This has to do with the dependencies. When compiling from source, you will used a locally compiled version of all dependencies whereas the wheel build will only compile mitsuba and pip install the external dependencies.

The issue here is with pybind11 I believe. I've just pushed a commit right now https://github.com/mitsuba-renderer/mitsuba3/commit/fda612b6cf3107fabb84dead1ca9c171a48e1314 that should fix this issue hopefully.

Could you please try again with the latest commit?

[ew00710]

Issue still persists. Can't install to a new environment from master either. Same result with local install pip install . and with pip install git+https://github.com/mitsuba-renderer/mitsuba3.git.

Further parts of the error also mentions cmake & wheels

An error occurred while building with CMake.
ERROR: Failed building wheel for mitsuba
Failed to build mitsuba
ERROR: Could not build wheels for mitsuba, which is required to install pyproject.toml-based projects`

[msqrt]

Sorry for taking some time to reply -- on my end, this indeed fixes the issue. Thank you!

[njroussel]

Great!

@ew00710 Please open an other issue if this still persists on your end. pip installs of a local Mitsuba build have always been quite awkward, we strongly recommend simply updating your PYTHONPATH envvar using the scripts we provide or you can do it "manually" at the top of your Python script by appending your build folder path to sys.path.