random crash wih access violation during optimization

[osylum]

Summary

Optimization stops in loop with access violation error. It does not happen systematically.

System configuration

OS: Windows-10 CPU: Intel64 Family 6 Model 165 Stepping 5, GenuineIntel GPU: NVIDIA RTX A4000 Python: 3.9.7 (tags/v3.9.7:1016ef3, Aug 30 2021, 20:19:38) [MSC v.1929 64 bit (AMD64)] NVidia driver: 517.40 CUDA: 10.0.130 LLVM: 15.-1.-1

Dr.Jit: 0.4.1 Mitsuba: 3.2.1 Is custom build? False Compiled with: MSVC 19.34.31942.0 Variants: scalar_rgb scalar_spectral cuda_ad_rgb llvm_ad_rgb

Description

Below is a simple example, similar as https://mitsuba.readthedocs.io/en/stable/src/inverse_rendering/gradient_based_opt.html. The goal for me was to check if I can optimize the index of refraction. I used llvm. It might stop randomly inside the loop with error: Process finished with exit code -1073741819 (0xC0000005). Activating the display of images within the loop or having print out seem to decrease the frequency when it happens, but not remove the problem. Not clear if it is a sync problem. Maybe I do something wrong in the loop, which cause an invalid access?

Thank you

def create_scene(res, filename, eta):
    scene = mi.load_dict({
        'type': 'scene',
        'integrator': {'type': 'prb'},
        'sensor': {
            'type': 'perspective',
            'to_world': T.look_at(
                origin=(0, 0, -2),
                target=(0, 0, 0),
                up=(0, -1, 0)
            ),
            'fov': 60,
            'film': {
                'type': 'hdrfilm',
                'width': res,
                'height': res,
            },
        },
        'textured_plane': {
            'type': 'rectangle',
            'to_world': T.scale(1.2),
            'bsdf': {
                'type': 'twosided',
                'nested': {
                    'type': 'diffuse',
                    'reflectance': {
                        'type': 'bitmap',
                        'filename': filename
                    },
                }
            }
        },
        # note: with smooth dieletric, it can't be optimized because it is a discontinuity - delta-function
        #       in that case, one could try using prb_reparam, or a roughdielectric with small alpha
        'glass_sphere': {
            'type': 'sphere',
            'to_world': T.translate([0, 0, -1]).scale(0.45),
            'bsdf': {
                'type': 'roughdielectric',
                'int_ior': eta,
                'alpha': 0.01
            },
        },
        'light': {
            'type': 'constant',
        }
    })

    return scene

def optimize(mitsuba_scene):
    filename = os.path.join(mitsuba_scenes, 'textures/barlow_001.png')
    image = mi.Bitmap(filename).convert(mi.Bitmap.PixelFormat.RGB, FLOAT_TYPE).resample(res=128)
    print('type image', type(image))
    # convert to drjit
    texture = dr.clip(mi.TensorXf(image),0.,1.)
    #display(texture)

    res = dr.shape(texture)[0]
    scene = create_scene(res, filename, eta=1.06)

    image_ref = mi.render(scene)
    display(image_ref, 'initial')

    key_texture = 'textured_plane.bsdf.brdf_0.reflectance.data'
    key_eta = 'glass_sphere.bsdf.eta'

    params = mi.traverse(scene)
    print(f'{params=}')
    params[key_texture] = texture # test setting TensorXf
    params[key_eta] = 1.90 # test setting float
    params.update()

    image_init = mi.render(scene)
    display(image_init, 'initial')

    # define optimizer and variable to be optimized
    opt = mi.ad.Adam(lr=0.05)
    opt[key_eta] = params[key_eta]
    params.update(opt)

    # define loss function
    def mse(image):
        return dr.mean(dr.sqr(image - image_ref))

    print('start iterations')
    iteration_count = 20
    losses = []
    for it in range(iteration_count):
        image = mi.render(scene, params) 
        loss = mse(image)
        dr.backward(loss)
        opt.step()
        params.update(opt)

        losses.append(loss)
        title = f"it: {it}, eta: {params[key_eta][0]:.3f}, opt: {opt[key_eta][0]:.3f}, loss: {loss[0]:.5f}"
        print(title)
        if False: #it%4:
            display(image, title)

    print('done optimizing')
    title = f"final: eta: {params[key_eta][0]:.3f}, loss: {loss[0]:.5f}"
    display(image, title)

    plt.clf()
    plt.plot(np.arange(iteration_count), losses)
    plt.xlabel('iterations')
    plt.ylabel('loss')
    plt.show()

[osylum]

In case you saw the message above already, I updated the code above (remove transform_enabled and clip_enabled) to be closer to the documentation example. This code produces the same issue.

[njroussel]

Hi @osylum

Have you tried running on the latest releases too?

[osylum]

I tried after upgrading and got the same behavior. I changed the infos about my configuration in the first comment; I have mitsuba 3.2.1 on python 3.9.7

[njroussel]

I am unable to reproduce this issue.

How often does it occur? Are 20 iterations enough?

[osylum]

it fails already at the first iteration for me.