search

mitsuba-renderer / mitsuba3

Mitsuba 3: A Retargetable Forward and Inverse Renderer
https://www.mitsuba-renderer.org/
Other
2.1k stars 246 forks source link

Batch sensor always initializes children's films as hdr films #1382

Closed tjueterb closed 1 week ago

tjueterb commented 1 week ago

Summary

Hi,

I just wanted to bring to your attention that the Batch sensor currently initializes all its child sensors' films as hdrfilm, even when the parent film is a specfilm. I didn't check if this causes any bugs, but it could cause trouble if the child sensors relied on film-specific behavior. In this case, the childrens' film types would not necessarily align with the parent film type.

Best Tobi

System configuration

OS: Ubuntu 23.10 CPU: AMD Ryzen 9 5950X 16-Core Processor GPU: NVIDIA GeForce RTX 3080 Ti Python: 3.11.5 (main, Sep 11 2023, 13:54:46) [GCC 11.2.0] NVidia driver: 545.23.08 CUDA: 12.3.107 LLVM: 11.1.0

Dr.Jit: 1.0.0 Mitsuba: 3.6.0 Is custom build? True Compiled with: Clang 15.0.7 Variants: scalar_rgb cuda_ad_rgb cuda_ad_spectral

Description

I compiled a8a037225e8452eadb5d93ef192b4d6a505ec139 and added a few Log Messages to highlight the behavior.

Steps to reproduce

I added the following Log messages to batch.cpp at lines 120-126:

        Log(Debug, "ParentFilm: %s", m_film->to_string());
        for (size_t i = 0; i < m_sensors.size(); ++i) {
            m_sensors[i]->film()->set_size(ScalarPoint2u(sub_size, size.y()));
            m_sensors[i]->parameters_changed();
            m_needs_sample_3 |= m_sensors[i]->needs_aperture_sample();
            Log(Debug, "m_sensors[%i]: %s", i, m_sensors[i]->film()->to_string());
        }

Then, when loading a specfilm with some children sensors

import mitsuba as mi
mi.set_variant('cuda_spectral', 'llvm_spectral')

scene_dict = {
    'type': 'batch',
    'film': {
        'type': 'specfilm',
        'srf1': {
            'type': 'regular',
            'wavelength_min': 400,
            'wavelength_max': 700,
            'values': '1, 1',
        },
        'srf2': {
            'type': 'regular',
            'wavelength_min': 700,
            'wavelength_max': 800,
            'values': '1, 1',
        }
    },
    'sensor1': {
        'type': 'perspective',
    },
    'sensor2': {
        'type': 'perspective',
    },
}

mi.set_log_level(mi.LogLevel.Debug)
scene = mi.load_dict(scene_dict)

The parent sensor is a specfilm, but the children sensors all have an hdrfilm associated with them:

2024-11-08 15:34:34 DEBUG main  [PluginManager] Loading plugin "plugins/batch.so" ..
2024-11-08 15:34:34 DEBUG main  [PluginManager] Loading plugin "plugins/specfilm.so" ..
2024-11-08 15:34:34 DEBUG main  [PluginManager] Loading plugin "plugins/regular.so" ..
2024-11-08 15:34:34 DEBUG main  [PluginManager] Loading plugin "plugins/perspective.so" ..
2024-11-08 15:34:34 DEBUG wrk1  [PluginManager] Loading plugin "plugins/hdrfilm.so" ..
2024-11-08 15:34:34 DEBUG wrk1  [PluginManager] Loading plugin "plugins/gaussian.so" ..
2024-11-08 15:34:34 DEBUG wrk1  [PluginManager] Loading plugin "plugins/independent.so" ..
2024-11-08 15:34:34 DEBUG main  [BatchSensor] ParentFilm: SpecFilm[
2024-11-08 15:34:34 DEBUG main  [BatchSensor]   size = [768, 576],
2024-11-08 15:34:34 DEBUG main  [BatchSensor]   crop_size = [768, 576],
2024-11-08 15:34:34 DEBUG main  [BatchSensor]   crop_offset = [0, 0],
2024-11-08 15:34:34 DEBUG main  [BatchSensor]   sample_border = 0,
2024-11-08 15:34:34 DEBUG main  [BatchSensor]   compensate = 0,
2024-11-08 15:34:34 DEBUG main  [BatchSensor]   filter = GaussianFilter[stddev=0.50, radius=2.00],
2024-11-08 15:34:34 DEBUG main  [BatchSensor]   file_format = OpenEXR,
2024-11-08 15:34:34 DEBUG main  [BatchSensor]   pixel_format = multichannel,
2024-11-08 15:34:34 DEBUG main  [BatchSensor]   component_format = float16,
2024-11-08 15:34:34 DEBUG main  [BatchSensor]   film_srf = [
2024-11-08 15:34:34 DEBUG main  [BatchSensor]     RegularSpectrum[
2024-11-08 15:34:34 DEBUG main  [BatchSensor]       distr = ContinuousDistribution[
2024-11-08 15:34:34 DEBUG main  [BatchSensor]         size = 5,
2024-11-08 15:34:34 DEBUG main  [BatchSensor]         range = [400, 800],
2024-11-08 15:34:34 DEBUG main  [BatchSensor]         integral = [500],
2024-11-08 15:34:34 DEBUG main  [BatchSensor]         pdf = [1, 1, 1, 2, 1]
2024-11-08 15:34:34 DEBUG main  [BatchSensor]       ]
2024-11-08 15:34:34 DEBUG main  [BatchSensor]     ]
2024-11-08 15:34:34 DEBUG main  [BatchSensor]   ],
2024-11-08 15:34:34 DEBUG main  [BatchSensor]   sensor response functions = (
2024-11-08 15:34:34 DEBUG main  [BatchSensor]     RegularSpectrum[
2024-11-08 15:34:34 DEBUG main  [BatchSensor]       distr = ContinuousDistribution[
2024-11-08 15:34:34 DEBUG main  [BatchSensor]         size = 2,
2024-11-08 15:34:34 DEBUG main  [BatchSensor]         range = [400, 700],
2024-11-08 15:34:34 DEBUG main  [BatchSensor]         integral = [300],
2024-11-08 15:34:34 DEBUG main  [BatchSensor]         pdf = [1, 1]
2024-11-08 15:34:34 DEBUG main  [BatchSensor]       ]
2024-11-08 15:34:34 DEBUG main  [BatchSensor]     ]
2024-11-08 15:34:34 DEBUG main  [BatchSensor]     RegularSpectrum[
2024-11-08 15:34:34 DEBUG main  [BatchSensor]       distr = ContinuousDistribution[
2024-11-08 15:34:34 DEBUG main  [BatchSensor]         size = 2,
2024-11-08 15:34:34 DEBUG main  [BatchSensor]         range = [700, 800],
2024-11-08 15:34:34 DEBUG main  [BatchSensor]         integral = [100],
2024-11-08 15:34:34 DEBUG main  [BatchSensor]         pdf = [1, 1]
2024-11-08 15:34:34 DEBUG main  [BatchSensor]       ]
2024-11-08 15:34:34 DEBUG main  [BatchSensor]     ]
2024-11-08 15:34:34 DEBUG main  [BatchSensor]   )
2024-11-08 15:34:34 DEBUG main  [BatchSensor] ]
2024-11-08 15:34:34 DEBUG main  [BatchSensor] m_sensors[0]: HDRFilm[
2024-11-08 15:34:34 DEBUG main  [BatchSensor]   size = [384, 576],
2024-11-08 15:34:34 DEBUG main  [BatchSensor]   crop_size = [384, 576],
2024-11-08 15:34:34 DEBUG main  [BatchSensor]   crop_offset = [0, 0],
2024-11-08 15:34:34 DEBUG main  [BatchSensor]   sample_border = 0,
2024-11-08 15:34:34 DEBUG main  [BatchSensor]   compensate = 0,
2024-11-08 15:34:34 DEBUG main  [BatchSensor]   filter = GaussianFilter[stddev=0.50, radius=2.00],
2024-11-08 15:34:34 DEBUG main  [BatchSensor]   file_format = OpenEXR,
2024-11-08 15:34:34 DEBUG main  [BatchSensor]   pixel_format = rgb,
2024-11-08 15:34:34 DEBUG main  [BatchSensor]   component_format = float16,
2024-11-08 15:34:34 DEBUG main  [BatchSensor] ]
2024-11-08 15:34:34 DEBUG main  [BatchSensor] m_sensors[1]: HDRFilm[
2024-11-08 15:34:34 DEBUG main  [BatchSensor]   size = [384, 576],
2024-11-08 15:34:34 DEBUG main  [BatchSensor]   crop_size = [384, 576],
2024-11-08 15:34:34 DEBUG main  [BatchSensor]   crop_offset = [0, 0],
2024-11-08 15:34:34 DEBUG main  [BatchSensor]   sample_border = 0,
2024-11-08 15:34:34 DEBUG main  [BatchSensor]   compensate = 0,
2024-11-08 15:34:34 DEBUG main  [BatchSensor]   filter = GaussianFilter[stddev=0.50, radius=2.00],
2024-11-08 15:34:34 DEBUG main  [BatchSensor]   file_format = OpenEXR,
2024-11-08 15:34:34 DEBUG main  [BatchSensor]   pixel_format = rgb,
2024-11-08 15:34:34 DEBUG main  [BatchSensor]   component_format = float16,
2024-11-08 15:34:34 DEBUG main  [BatchSensor] ]
2024-11-08 15:34:34 DEBUG main  [Scene] Scene static GPU acceleration shutdown ..
rtabbara commented 1 week ago

Hi @tjueterb ,

I think this is fine, but I concede it may be slightly confusing. Just to quote the documentation:

The horizontal resolution of the film associated with this sensor must be a multiple of the number of sub-sensors. In addition, all of the sub-sensors’ films, samplers and shutter timings are typically ignored and superseded by the film, sampler and shutter timings specified for the batch sensor itself.

The actual storage for the film is only created once an integrator calls Film::prepare for the specified sensor prior to rendering. So in the case of the batch sensor, it's not the case that we create storage for each of the individual sub-sensors but rather just for the parent with its combined resolution and type.

tjueterb commented 1 week ago

Hi @rtabbara, thanks for the clarification! Best, Tobi