NVlabs / nvdiffmodeling

Differentiable rasterization applied to 3D model simplification tasks
Other
455 stars 30 forks source link

The exported mesh cannot be restored after being loaded using nvdiffmodeling #29

Closed Lucklycat closed 1 year ago

Lucklycat commented 1 year ago

Thank you very much for your work! Taking ewer as an example, I re-use nvdifmodeling to load and render the exported mesh. The code is as follows: ` eye = [0.5, 0.0, 3.0] up = np.array([0.0, 1.0, 0.0]) at = np.array([0, 0, 0]) a_mv = util.lookAt(eye, at, up) a_mvp = np.matmul(proj_mtx, a_mv).astype(np.float32)[None, ...] a_lightpos = np.linalg.inv(a_mv)[None, :3, 3] a_campos = np.linalg.inv(a_mv)[None, :3, 3]

background = torch.tensor(util.checkerboard(2048, 8), dtype=torch.float32, device='cuda')

params = {'mvp': a_mvp, 'lightpos': a_lightpos, 'campos': a_campos,
          'resolution': [2048, 2048],
          'time': 0}

_opt_ref = mesh.center_by_reference(render_ref_mesh.eval(params), ref_mesh_aabb, 2.0)
img_ref = render.render_mesh(glctx, _opt_ref, a_mvp, a_campos, a_lightpos, 20.0, 2048,
                             num_layers=1, spp=1, background=background,
                             min_roughness=0.08)

img_ref = util.scale_img_nhwc(img_ref, [2048, 2048])
result_image = util.tonemap_srgb(img_ref[0])
np_result_image = result_image.detach().cpu().numpy()
util.save_image(out_dir + '/' + ('test_%d.png' % img_cnt), np_result_image)
img_cnt = img_cnt + 1

` It ensures that all parameters are consistent, but it is found that the rendered image result after import is far from the final result during training: image

Using the same code, I imported the original ewer's obj, and the rendered image is as follows, which proves that there is no problem with the super-parameter setting: image

What is the reason for this problem?

jmunkberg commented 1 year ago

In the first image above, it looks like the left example is rendered without the normal map