NVlabs / nvdiffrec

Official code for the CVPR 2022 (oral) paper "Extracting Triangular 3D Models, Materials, and Lighting From Images".
Other
2.09k stars 222 forks source link

Not able to learn specular and diffuse properties #141

Open iraj465 opened 11 months ago

iraj465 commented 11 months ago

Hey i trying to model objects for plastic-like objects

Input video:

ezgif com-video-to-gif

Config:

{
  "ref_mesh": "./plastic-grocery",
  "random_textures": true,
  "iter": 2000,
  "save_interval": 500,
  "texture_res": [
    2048,
    2048
  ],
  "train_res": [
    1080,
    1920
  ],
  "batch": 4,
  "learning_rate": [
    0.03,
    0.003
  ],
  "kd_min": [
    0.03,
    0.03,
    0.03
  ],
  "kd_max": [
    0.8,
    0.8,
    0.8
  ],
  "ks_min": [
    0,
    0.08,
    0
  ],
  "ks_max": [
    0,
    1,
    1
  ],
  "dmtet_grid": 128,
  "mesh_scale": 2.4,
  "lock_light": true,
  "laplace": "absolute",
  "laplace_scale": 5000,
  "background": "white",
  "display": [
    {
      "latlong": true
    },
    {
      "bsdf": "kd"
    },
    {
      "bsdf": "ks"
    },
    {
      "bsdf": "normal"
    }
  ],
  "out_dir": "grocery"
}

This is the output:

Screenshot 2023-07-24 at 3 09 59 PM

@jmunkberg i found that the textures are washed out, what can be done to model the textures accurate to the video itself?

iraj465 commented 11 months ago

During first pass the textures are blurry, but still okay DAY04VD25_DAY04VD25_img_dmtet_pass1_000004

But even during the second pass, the textures haven't improved and are very blurry

DAY04VD25_DAY04VD25_img_mesh_pass_000004