bennyguo / instant-nsr-pl

Neural Surface reconstruction based on Instant-NGP. Efficient and customizable boilerplate for your research projects. Train NeuS in 10min!
MIT License
856 stars 84 forks source link

Problem on calculating comp_rgb #54

Open finninmunich opened 1 year ago

finninmunich commented 1 year ago

Hello! I see in your codes, it is: comp_rgb = comp_rgb + self.background_color * (1.0 - opacity) I thought it should be comp_rgb = comp_rgb*opacity + self.background_color * (1.0 - opacity) Did I miss something? could somebody explain that? Thank you in advance!

bennyguo commented 1 year ago

Hi! The volume rendering procedure produces colors with premultiplied alpha, so we don't have to apply opacity to comp_rgb. Think of a red pixel with opacity=0.5, the volume rendering outputs RGB=(0.5,0,0) and opacity=0.5.

finninmunich commented 1 year ago

Thanks for your reply!

Here I also have a silly question. In Nerfacc we have something like this weights = render_weight_from_alpha(alpha, ray_indices=ray_indices, n_rays=n_rays) opacity = accumulate_along_rays(weights, ray_indices, values=None, n_rays=n_rays) I think in some cases, the sum of the weights may become larger than 1, so the opacity will also become something larger than 1. Is that correct? :)

bennyguo commented 1 year ago

image Actually, the sum of the weights cannot be larger than 1. Intuitively, the weight at some point is decayed by the visibility of this point, i.e., w_{i} < 1 - \sum_{k=0}^{i-1}w_{k}, so that \sum_{k=0}^{i}w_{k} should always be less than 1.

finninmunich commented 1 year ago

Thanks for your reply and help! Appreciate!!!