Python Kernel dies while rendering when using `blendbsdf` with `"raw":True`.

[jaroslavknotek]

Summary

Python Kernel dies while rendering when using blendbsdf with "raw":True.

System configuration

OS: CentOS Linux release 7.9.2009 (Core) CPU: Intel(R) Xeon(R) Gold 6126 CPU @ 2.60GHz GPU: NVIDIA GeForce RTX 2080 Ti Python: 3.10.9 (main, Mar 8 2023, 10:47:38) [GCC 11.2.0] NVidia driver: 550.54.15 CUDA: 12.4.131 LLVM: -1.-1.-1

Dr.Jit: 0.4.4 Mitsuba: 3.5.0 Is custom build? False Compiled with: GNU 10.2.1 Variants: scalar_rgb scalar_spectral cuda_ad_rgb llvm_ad_rgb

Description

Using example from documentation here with modified weight settings as follows:

{
    'type': 'blendbsdf',
    'weight': {
        'type': 'bitmap',
        'raw':True,
        'value':img
    },
    'bsdf_0': {
        'type': 'conductor',
        'material' : 'Au'
    },
    'bsdf_1': {
        'type': 'roughplastic',
        'diffuse_reflectance': 0.1
    }
}

With value of img taking form of the following:

# array of values from interval [0,1)
img = np.random.rand(100,100)
#or 8bit grayscale
img = np.uint8(np.random.rand(100,100) *255)
# or 8bit RGB
img = np.dstack( [np.uint8(np.random.rand(100,100) *255]*3)

Kernel just dies without any error.

However, saving the very same img into a file using imageio and setting it as a bitmap works fine e.g.:

{
    'type': 'blendbsdf',
    'weight': {
        'type': 'bitmap',
        'filename':'bitmap.png'
    },
# rest of the scene definition
}

Reading the texture into array and serving it with raw set to False fails again. There are no errors in the output, nor station resources (CPU, GPU, RAM) are all available during rendering.

Additionally, other usage of raw bitmap works fine (bumpmap,normalmap) therefore I have used it successfully.

Steps to reproduce

1. Import and setup

import mitsuba
import numpy as np

mi.set_variant('cuda_ad_rgb')

2. Create scene

scene_dict = {
    'type': 'scene', 
    'integrator': {'type': 'path'}, 
    'sensor': '...', 
    'light': { 'type': 'constant'}, 
    'model': {
        'type': 'sphere', 
        'material': {
            'type': 'blendbsdf', 
            'weight': {'type': 'bitmap', 'raw': True}, 
            'bsdf_0': {'type': 'conductor', 'material': 'Au'}, 
            'bsdf_1': {'type': 'roughplastic', 'diffuse_reflectance': 0.1}
        }
    }
}

3. Render Scene

scene = mi.load_dict(scene_dict)
img = mi.render(scene)

Wait for kernel death.

4. Retry with weights in file

imageio.imwrite(
    'test.png',
    np.uint8(np.random.rand(100,100)*255)
)

scene_dict={
    'model':{
        'material':{
             'weight':{
                  'filename':'test.png'
...

This works just fine

5. Perform Sanity Check

Try reading the file that worked into file

scene_dict={
    'model':{
        'material':{
             'weight':{
                  'raw':True,
                  'value':imageio.imread('test.png')
...

And it fails again.

[jaroslavknotek]

Now, I have the same issue even with texture read from file system:

scene:

{
    'type': 'scene',
    'integrator': {'type': 'path'},
    'sensor': {'type': 'perspective',
    'to_world': [[1, 0, 0, 0],
      [0, 0, -1, 10],
      [-0, 1, 0, 0],
      [0, 0, 0, 1]],
     'film': {'type': 'hdrfilm',
         'pixel_format': 'rgba',
         'width': 600,
         'height': 800}},
     'light': {'type': 'constant', 'radiance': {'type': 'rgb', 'value': 1}},
     'model': {
          'type': 'cylinder',
          "material":{
          'type': 'bitmap',
         "value": str(img_path)
    }
}

Is there anything I can do to at least start debugging what's wrong?

[merlinND]

Hello @jaroslavknotek,

A first easy step is to check the console from which you are running the Jupyter server for any logs or exception. In many cases this will already give you a good idea of what's going wrong.

If that's not enough, the typical steps to debug are:

1. Convert your reproducing code from a notebook to a simple Python script
2. Uninstall any pip-based install of Mitsuba and DrJit
3. Clone and compile the latest Mitsuba master in debug mode
4. Run your script and report the full error message
5. Run your script in GDB / LLDB and report the stack trace.

When reporting back, please include your standalone reproducer script, including any data it uses, together with the full error messages and stack trace.

[merlinND]

That being said, looking at this part of the scene definition, I don't understand what you are attempting to do:

     'model': {
          'type': 'cylinder',
          "material":{
          'type': 'bitmap',
         "value": str(img_path)
    }

bitmap is not a BSDF. Is it possible that your forgot to wrap your bitmap-typed dict into a BSDF such as diffuse ?

[jaroslavknotek]

I did forget to wrap bitmap. Thank you.