NVlabs / nvdiffrecmc

Official code for the NeurIPS 2022 paper "Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoising".
Other
363 stars 28 forks source link

Missing texture features in early stages of optimization #8

Closed yyeboah closed 1 year ago

yyeboah commented 1 year ago

I'm optimizing by initializing with existing geometry with UV map and the base geometry being used is a human facial model. I am observing that while the materials appear right at the end of optimization, some facial features are lost at this point. Specifically eye and lip details. Below the paramters used for optimization: { "base_mesh": "data/face/mesh.obj", "ref_mesh": "data/face/mesh.obj", "random_textures": true, "iter": 5000, "save_interval": 100, "texture_res": [ 1024, 1024 ], "train_res": [1024, 1024], "batch": 4, "learning_rate": [0.03, 0.01], "ks_min" : [0, 0.1, 0.0], "ks_max" : [0, 1.0, 1.0], "envlight": "data/irrmaps/aerodynamics_workshop_2k.hdr", "validate" : false, "lock_pos" : true, "display": [{"latlong" : true}], "background" : "white", "denoiser": "none", "no_perturbed_nrm" : true, "n_samples" : 8, "out_dir": "face" }

It would appear that these missing features are being treated as noise ? Are there some assumptions in the material estimation stage that would lead to such behaviour ?