🐛 Updated vertex positions incorrectly rendered

[TigerHix]

Summary

After modifying vertex positions of an object in scene parameters, rendered image becomes incorrect.

This is similar to #276, but the bug behavior seems different, so I am opening a new issue.

System configuration

• Platform: Windows
• Compiler: MSVC 2019
• Python version: 3.7
• Mitsuba 2 version: Latest (4e7628c)
• Compiled variants:
  • scalar_rgb
  • packet_rgb
  • gpu_rgb
  • gpu_autodiff_rgb

Description

Here's a minimum code snippet to reproduce the bug. Basically, it loads the Cornell box scene, moves the small box by (100, 100, 100), (optionally) calls params.set_dirty(...) + params.update(), and renders the image.

import copy

import mitsuba

mitsuba.set_variant('scalar_rgb')

from mitsuba.core import xml, Bitmap, Struct
from mitsuba.python.util import traverse

scene = xml.load_file('cbox.xml')

params = traverse(scene)
var = 'OBJMesh_6.vertex_positions_buf'

positions_buf = copy.copy(params[var].numpy())

for i in range(len(positions_buf)):
    positions_buf[i] = positions_buf[i] + 100

tp = type(params[var])
params[var] = tp(positions_buf)

# Including this should have no effect, as params.__setitem__ above already calls params.set_dirty
# params.set_dirty(var)
# params.update()

sensor = scene.sensors()[0]
scene.integrator().render(scene, sensor)

film = sensor.film()
img = film.bitmap(raw=True).convert(Bitmap.PixelFormat.RGB, Struct.Type.UInt8, srgb_gamma=True)
img.write('bug.jpg')

Rendered with scalar_rgb and packet_rgb variant - incorrect:

Rendered with gpu_rgb and gpu_autodiff_rgb variants - as if vertices were not translated at all:

I have also tried using the translation method (i.e. Transform4f + transform_point) used by one of the legacy examples, invert_pose.py, and got an even weirder result:

For details please see this comment in #276.

[Speierers]

See my response in the other issue #276

[TigerHix]

Hi @Speierers, thank you for the reply!

Unfortunately, this does not seem to solve the issue. This is the result image after adding ek.cuda_eval() and ek.cuda_sync() after params.update().

Full code below:

import copy

import mitsuba

mitsuba.set_variant('gpu_rgb')

import enoki as ek
from mitsuba.core import xml, Bitmap, Struct
from mitsuba.python.util import traverse

scene = xml.load_file('cbox.xml')

params = traverse(scene)
var = 'OBJMesh_6.vertex_positions_buf'

positions_buf = copy.copy(params[var].numpy())

for i in range(len(positions_buf)):
    positions_buf[i] = positions_buf[i] + 100

tp = type(params[var])
params[var] = tp(positions_buf)

params.set_dirty(var)
params.update()
ek.cuda_eval()
ek.cuda_sync()

sensor = scene.sensors()[0]
scene.integrator().render(scene, sensor)

film = sensor.film()
img = film.bitmap(raw=True).convert(Bitmap.PixelFormat.RGB, Struct.Type.UInt8, srgb_gamma=True)
img.write('bug.jpg')

[TigerHix]

Also, since the bug also occurs even if I'm using scalar_rgb / packet_rgb mode (see main post), I suppose CUDA is not the issue here...?

[Speierers]

In anycase this sort of scene editing in Python is currently only supported in cuda mode, so do not expect this to work in scalar_* or packet_* modes

[Speierers]

Could you try with this?

params = traverse(scene)
var = 'OBJMesh_6.vertex_positions_buf'
params[var] += 100.0
params.update()
ek.cuda_eval()
ek.cuda_sync()

[TigerHix]

Could you try with this?

params = traverse(scene)
var = 'OBJMesh_6.vertex_positions_buf'
params[var] += 100.0
params.update()
ek.cuda_eval()
ek.cuda_sync()

Thanks! But I still get the same result...

[TigerHix]

FYI, if I replace 100.0 with 200.0:

Which looks exactly like #276!

[Speierers]

Which branch are you working with?

It could be that this only works with this branch

[TigerHix]

Which branch are you working with?

Master branch, commit 4e7628c6eed365904ca2ba536b795d1b03410344.

[Speierers]

Please give a try at the other branch mentioned above and let me know if this solves you issue.

[TigerHix]

Please give a try at the other branch mentioned above and let me know if this solves you issue.

Unfortunately it is still not working. I built the branch you mentioned, and on first try, mitsuba2 crashed during the render. I followed the fix you provided in #276 (adding m_vertex_positions_buf.managed(); to mesh.cpp before cuda_eval()), and while mitsuba2 no longer crashes, the result image looks as if nothing was changed:

Full code below:

import copy

import mitsuba

mitsuba.set_variant('gpu_autodiff_rgb')

import enoki as ek
from mitsuba.core import xml, Bitmap, Struct
from mitsuba.python.util import traverse

scene = xml.load_file('cbox.xml')

params = traverse(scene)
var = 'OBJMesh_6.vertex_positions_buf'
params[var] += 100.0
params.update()
ek.cuda_eval()
ek.cuda_sync()

sensor = scene.sensors()[0]
scene.integrator().render(scene, sensor)

film = sensor.film()
img = film.bitmap(raw=True).convert(Bitmap.PixelFormat.RGB, Struct.Type.UInt8, srgb_gamma=True)
img.write('bug.jpg')

Any further thoughts would be appreciated!

[Speierers]

Could you print the values of params to make sure those are changed after the += 100?

[TigerHix]

Could you print the values of params to make sure those are changed after the += 100?

Yes, these are indeed changed. See my code change below:

...
print(params[var])
params[var] += 100.0
params.update()
print(params[var])
...

2021-11-22 04:41:18 INFO  main  [Scene] Building scene in OptiX ..
[130, 165, 65, 82, 165, .. 62 skipped .., 0, 225, 130, 0, 65]
[230, 265, 165, 182, 265, .. 62 skipped .., 100, 325, 230, 100, 165]
2021-11-22 04:41:19 INFO  main  [SamplingIntegrator] Rendering finished. (took 1.144s)

[Speierers]

Unfortunately I won't be able to debug this myself as we are busy working on the next generation of Mitsuba. I would recommend you add print statements in this function to make sure it gets called when calling params.update() (e.g. making sure that the scene BHV is rebuilt after modifying the geometry).

[JungShinPlusPlus]

@Speierers Hi, Just curious if there is any update regarding updating the position(or vertex buffer) of the object. thank you :)

[Speierers]

On the Mitsuba 2 side, nothing has changed as we are focusing on the upcoming release of our new system.