Wrong fragments appeared after second pass

[sun510001]

Hello, Firstly, thanks for your excellent work. I'm trying nvdiffrec and nvdiffrecmc on my own data.

It works pretty good at first pass:

but lots of wrong fragments are appeared after second pass:

This problem occurs with both nvdiffrec and nvdiffrecmc. Lock pos can solve this problem, but I want to use regularizer to make mesh smoother. I have tried laplace_scale 3000 6000 and 10000; laplace: absolute and relative, but wrong fragments still appeared after second pass.

Any input here would be greatly appreciated, thank you!

[jmunkberg]

Thanks @sun510001 !

Increasing the Laplacian regularizer would have been my first tests as well. You can test the "lock_pos" : true option in the config, which disables geometry optimization in the second pass . You can also test decreasing the learning rate in the second pass, like "learning_rate": [0.03, 0.003], or similar.

Also, nvdiffrec(mc) is quite sensitive to inaccuracies in the foreground segmentation masks and poses. Your reconstructed silhouettes look a bit uneven, so if there is a way to provide better masks with your dataset, that could likely help quality a bit here.

[sun510001]

@jmunkberg Thanks for your reply.

I checked the previously generated meshes and datasets. The datasets with good matting generated fewer fragments in the textures of the mesh.