about the ray for calculating entropy loss

[chensjtu]

In paper, "where R s denotes a set of rays from training images, Ru denotes a set of rays from randomly sampled unseen images, and ⊙ indicates element-wise multiplication.", you have mentioned that the entropy loss is calculated within the union of seen and unseen rays. However, in the lego code, the entropy loss is only down on unseen rays. acc = acc[self.N_samples:] sigma = sigma[self.N_samples:] why only use later 1024 rays for this loss?

[universome]

Ah, I was going to ask the same question :) Right now, this looks like a typo to me — shouldn't it be acc[:self.N_samples] instead?

@mjmjeong could you please help us with that?

[mjmjeong]

We used both seen and unseen rays to calculate the ray entropy minimization loss in the paper. Although there is no significant difference in the performance, we modified the code to use only unseen rays following reasons.

In the few-shot setting, since the distribution of R_u itself is almost close to the whole ray distribution, using only R_u is enough. Also, R_s is sampled from the few selected views while R_u is sampled from almost full views. Then, probabilistically, one ray from seen views is sampled more frequently than another ray from unseen views. In this regard, we think the potential unbalance optimization issue can happen, so we modified it for better stabilization.

@universome The code you mentioned is using only seen rays for the ray entropy minimization loss. In this case, the noise in the background area is not removed completely as shown in Figure 4 in the main paper.