mitsuba-renderer / mitsuba3

Mitsuba 3: A Retargetable Forward and Inverse Renderer
https://www.mitsuba-renderer.org/
Other
2.06k stars 241 forks source link

[bug report] Silent failure when instantiating bitmap from strided array #611

Closed VladislavZavadskyy closed 1 year ago

VladislavZavadskyy commented 1 year ago

Summary

Instantiating Bitmap from an array with non-trivial stride produces the wrong bitmap, raising no errors.

Description

This was found when importing image to bitmap from torch. Torch has channel-first memory layout, so to make it compatible one has to permute dimensions. By default it just changes stride, without changing the data. Initialization from PyObject ignores stride completely and simply copies the memory with memcpy. This results in silent failure to load the correct bitmap and can lead to pesky hard-to-notice bugs in complex pipelines.

I would suggest at least raising an error, to advice the user to make the array contiguous prior to instantiating the bitmap.

Steps to reproduce

import mitsuba as mi
mi.set_variant("cuda_ad_rgb")

from PIL import Image
from torchvision.transforms.functional import to_tensor

img_t = to_tensor(Image.open(IMG_PATH))
mi.Bitmap(img_t.permute(1, 2, 0).numpy())
njroussel commented 1 year ago

Hi @VladislavZavadskyy

Thanks for the bug report. Indeed the strides are completely ignored when converting from an object which implements __array_interface__.

However we do correctly handle strides through the dlpack interface. So, in your example, you can actually write mi.Bitmap(mi.TensorXf(img_t.permute(1, 2, 0))) which would work. This is an efficient workaround for anyone who encounters this issue until we patch/fix it.

njroussel commented 1 year ago

This has been fixed in https://github.com/mitsuba-renderer/mitsuba3/commit/5b784dbd251d04cd77d3fb8f0d7eed85620c5b32.