search

NVlabs / nvdiffrec

Official code for the CVPR 2022 (oral) paper "Extracting Triangular 3D Models, Materials, and Lighting From Images".
Other
2.1k stars 222 forks source link

Recreating MLP parametrized SDF experiments #9

Closed davidas1 closed 2 years ago

davidas1 commented 2 years ago

Hello,

First of all, I would like to thank you for sharing this great work. I'm trying to run some experiments on a custom scene I shot and it looks like I'm getting the high-frequency noise you mentioned in the paper (re. DTU dataset), so I wanted to test the MLP parameterization method you suggested, to see if it improves the results. As it's not implemented in the code, I tried adding a NeRF-like MLP to predict SDF values (both with frequency encodings and hash-grid), but I'm having trouble initializing the weights and training with it. Can you give some more information on how to initialize the training in this case, or if I should use a different loss or hyper parameters for t_f, n_base, etc..

frankshen07 commented 2 years ago

Thank you for your interest in our work! For MLP parametrization, we pretrained the SDF parametrized by MLP to represent a sphere, s.t. sdf(p) = (util.length(p) / mesh_scale) - 0.4. Also, we lower the initial learning rate for MLP parameters to 5e-4 (lr for texture & lighting stays the same).

JiyouSeo commented 1 year ago

Hello, can you tell me how many iter you have run it? And did you set the MLP output shape as (size, sdf) which mean (size,1) or (size, sdf+deform) which mean (size, 4) ?