search

NVlabs / nvdiffrecmc

Official code for the NeurIPS 2022 paper "Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoising".
Other
363 stars 28 forks source link

Why does the performance change depending on the flag base_mesh? #35

Closed cmh1027 closed 6 months ago

cmh1027 commented 6 months ago

These are metrics of Chair dataset of NeRF-Synthetic obtained by performing stage 1 and 2 from scratch. PSNR 28.989 / SSIM 0.944 / LPIPS 0.060 And these are metrics obtained by performing only stage 1, and restart the program with base_mesh flag. PSNR 26.268 / SSIM 0.912 / LPIPS 0.091

Not only Chair, but also every other scenes show the same degenerated performance. I can't see the reason why this happens. Aren't they supposed to show the same performance?

jmunkberg commented 6 months ago

Perhaps it is due to the learning rate warmup to avoid Adam spikes? When running two-pass optimization, we use a LR warmup over 100 steps in the beginning of the second pass: https://github.com/NVlabs/nvdiffrecmc/blob/main/train.py#L666, which is not enabled when only running only the second pass (using a base_mesh). https://github.com/NVlabs/nvdiffrecmc/blob/main/train.py#L682