What can apply_weights do?

[sora158]

I can't understand self.gaussian.apply_weights(cur_cam, weights, weights_cnt, mask) in the process of update masks.

def apply_weights(self, camera, weights, weights_cnt, image_weights):

rasterizer = camera2rasterizer( camera, torch.tensor([0.0, 0.0, 0.0], dtype=torch.float32, device="cuda") ) rasterizer.apply_weights( self.get_xyz, None, self.get_opacity, None, weights, self.get_scaling, self.get_rotation, None, weights_cnt, image_weights, ) Please tell me about what does weights_cnt mean and how this function work ?

[sora158]

I think key code for apply_weights is from apply_weights.cu:

     // Eq. (3) from 3D Gaussian splatting paper.
      for (int ch = 0; ch < CHANNELS; ch++) {
        atomicAdd(weights + (collected_id[j] * CHANNELS + ch), C[ch]);
        // atomicAdd(weights + (collected_id[j] * CHANNELS + ch), C[ch] * T);
        atomicAdd(cnt + collected_id[j], 1);
        // if (C[ch] > 0.0f) {
        //   atomicAdd(weights + (collected_id[j] * CHANNELS + ch), C[ch] * T);
        //   atomicAdd(cnt + collected_id[j], 1);
        // }
      }
      T = test_T;

      // Keep track of last range entry to update this
      // pixel.
      last_contributor = contributor;
    }
  }

However, I can't align it with the paper quoted Eq.6; I think this line atomicAdd(weights + (collected_id[j] * CHANNELS + ch), C[ch] * T);matches Eq.6, am I right? $w_i^j=\sum o_i(\boldsymbol{p}) T_i^j(\boldsymbol{p}) \mathcal{M}^j(\boldsymbol{p})$, Eq.6

[buaacyw]

I think key code for apply_weights is from apply_weights.cu:

     // Eq. (3) from 3D Gaussian splatting paper.
      for (int ch = 0; ch < CHANNELS; ch++) {
        atomicAdd(weights + (collected_id[j] * CHANNELS + ch), C[ch]);
        // atomicAdd(weights + (collected_id[j] * CHANNELS + ch), C[ch] * T);
        atomicAdd(cnt + collected_id[j], 1);
        // if (C[ch] > 0.0f) {
        //   atomicAdd(weights + (collected_id[j] * CHANNELS + ch), C[ch] * T);
        //   atomicAdd(cnt + collected_id[j], 1);
        // }
      }
      T = test_T;

      // Keep track of last range entry to update this
      // pixel.
      last_contributor = contributor;
    }
  }

However, I can't align it with the paper quoted Eq.6; I think this line atomicAdd(weights + (collected_id[j] * CHANNELS + ch), C[ch] * T);matches Eq.6, am I right? wij=∑oi(p)∗Tij(p)∗Mj(p), Eq.6

Yes. You are right. It's for mask weights accumulation.