Loading Mesh from Memory

[uguuuuuu]

I am trying to optimize a SDF by converting it to a mesh and then render it. But I did not find anyway to load a mesh from a v, f, uv, and uv indices arrays. Is there such a way or I have to modify the source code?

[andyyankai]

it really depends on how you want to optimize the SDF. The most simple way might be that for the "Mesh" class, there is a "load" function. After you load the explicit mesh. You simply call scene.configure() However, if you are using something like a differential marching cube, you might need to update your vertex position in the forward pass like: sc.param_map['Mesh[0]'].vertex_positions = vertex_pos; so you can pass the gradient correctly from SDF to explicit mesh. You might want to check this as well: https://github.com/uci-rendering/psdr-cuda/issues/11

[uguuuuuu]

Thank you for replying. I am trying to optimize the SDF by using differentiable marching tetrahedron to convert it to a mesh and then rendering it using psdr. If I loaded the mesh by first writing it to an obj and then using Mesh.load(), that would mean I have to do that every iteration during raining since the topology, thus the face indices, and the number of vertices would change after every step of gradient descent and I could not just do like sc.param_map['Mesh[0]'].vertex_positions = vertex_pos.

[uguuuuuu]

Another issue is that loading a mesh and then render it and do that again with a different mesh results in weird artifacts.

integrator = psdr_cuda.DirectIntegrator()
scene = psdr_cuda.Scene()
scene.load_file(scene_file, False)
scene.opts.spp = 32
scene.opts.width = 284
scene.opts.height = 216

dmtet = DMTetGeometry(32, 2.1)
m = dmtet.getMesh()
obj.write_obj('.', m)
scene.param_map['Mesh[0]'].load('mesh.obj')
scene.configure()
img = integrator.renderC(scene, 2)
save_img(img, 'img.png', (284, 216))
dmtet = DMTetGeometry(32, 2.1)
m = dmtet.getMesh()
obj.write_obj('.', m)
scene.param_map['Mesh[0]'].load('mesh.obj')
scene.configure()
img = integrator.renderC(scene, 2)
save_img(img, 'img1.png', (284, 216))

[andyyankai]

For SDF opt, you might want to create a new scene for each iteration (yeah you have to do it at least in this repo...). Then load the explicit mesh from a differentiable marching tetrahedron into the scene and render it with your customized torch.autograd.Function to get the gradient and backpropagate to sdf node.

[andyyankai]

Also, do not use scene.param_map['Mesh[0]'].load('mesh.obj') use scene.load_file or scene.load_string directly AFTER you save the explicit mesh into a separate file.