search

city-super / Scaffold-GS

[CVPR 2024 Highlight] Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering
https://city-super.github.io/scaffold-gs
Other
635 stars 47 forks source link

Question about evaluation on NeRF Synthetic #60

Open francescodisario opened 2 weeks ago

francescodisario commented 2 weeks ago

Hi and first of all thanks for your amazing work! I have a question regarding your evaluation method for NeRF Synthetic. In the paper you reported 33.68 db PSNR on NeRF Synthethic and you compared it with the vanilla GS which is around 33.20. To achieve such good results, did you set the background color to white? I believe that from your code you keept the background black, while in vanilla-gs (and il all previous works), the bg color i white.

In fact, if I try to execute your code with bg white, I can't reproduce your result (I am around 0.6db less), while vanilla-gs with bg color black is around 34db PSNR.

I don't think it's a big problem, but I just wanted to be sure that everything is working correctly and point out that the bg color may really affect accuracies results. So it would have been better to compare them with the same bg color.

inspirelt commented 2 weeks ago

Hi, we evaluate nerfsynthetic with the default black background. And I believe the gap in results may caused by some minor updates in recent months, like the density control strategies. You can adopt your current reproduced results in your research as long as clarifying experiment settings.