search

Lakonik / SSDNeRF

[ICCV 2023] Single-Stage Diffusion NeRF
https://lakonik.github.io/ssdnerf/
MIT License
432 stars 23 forks source link

Test PSNR not improving on custom dataset #32

Closed xiyichen closed 9 months ago

xiyichen commented 9 months ago

Nice work! I'm trying to train a custom face dataset (FaceScape) using the one-view reconstruction setting. Since the cameras in this dataset is already in OpenCV format, there's no conversion needed. I have tuned the radius parameter to reflect the change in the object size. However, the test PSNR never improves, although training PSNR does.

Screenshot 2023-11-21 at 19 59 10

scene_122_01_000018_psnr7 6_ssim0 24_lpips0 726 scene_122_01_000010_psnr9 0_ssim0 30_lpips0 711 scene_122_01_000004_psnr8 3_ssim0 29_lpips0 723 It also seems the rendered faces are not correctly localized. However I double checked the camera parameters, and made sure they are in the same convention as SRN car dataset and the poses loaded are c2w matrices. The only difference is that the intrinsics for each camera are different, but I've modified the dataloader correspondingly.

I'm a bit lost on what could be wrong. Do you make other assumptions on the camera parameters? Are there any other parameters that I should tune?

Lakonik commented 9 months ago

Thank you for your interest in our work!

How is the training PSNR actually? The screenshot only shows very early results so it's hard to tell if anything goes wrong. Evaluating the model at this early stage is not really helpful.

If the image resolution has been changed, then you probably need to tweak some parameters like guidance scale. But if it's just the camera intrinsics then it should be ok. Also starting from the SRN car config for face reconstruction is a good choice.

xiyichen commented 9 months ago

Thank you for your fast response! It turned out it was indeed just too early in the training stage. After training overnight, now the evaluation results look reasonable.