the PBR texture may not right

[renrenzsbbb]

Thanks for your great work. I want to input the obj file with base color texture, and hope that model can help me to get pbr texture map. When training, the render image is similar to target img. However, when I show the image in the blender or meshlab, the texture map is wrong. Can you give me some advices?

[jmunkberg]

Please refer to https://github.com/NVlabs/nvdiffrecmc#use-the-extracted-3d-models-in-blender to see how to map the PBR material parameters properly.

[renrenzsbbb]

Thanks for your quick reply. I double check it, my operation is right. When I train other object like bob in the git or winnie-pooh, the result is good. but in the object, I could not get good result. Can you give me more advices?

[jmunkberg]

If it looks good in training, then I suspect there is an issue when moving the model to a new renderer (with different lighting). We use a PBR model (following the Disney BSDF) with a base color, metalness, roughness (stored in the G and B channels of the Ks texture) and a tangent space normal map. We also capture the lighting in an .hdr probe. See the paper for details. MeshLab won't map these parameters correctly out of the box, neither will Blender. The script https://github.com/NVlabs/nvdiffrecmc#use-the-extracted-3d-models-in-blender shows an example on how to setup the material and import the captured .hdr probe.

You can also constrain the material properties in the config with the flags

    "kd_min" : [0.03, 0.03, 0.03],
    "kd_max" : [0.8, 0.8, 0.8],
    "ks_min" : [0, 0.08, 0],
    "ks_max" : [0, 1.0, 1.0],

One example here: https://github.com/NVlabs/nvdiffrec/blob/main/configs/nerd_ehead.json

In case you know the lighting and you are only interested in optimizing the materials, you can disable light optimization. Here is an example which disables light optimization: https://github.com/NVlabs/nvdiffrec/blob/main/configs/spot_fixlight.json