A question about `Dataset Update`

[ScienceLi1125]

Hi! When I read the paper, I have some questions about unlike NeRF, 3DGS does not use rays for training and we need to render full images, meaning we cannot have a mixed signal of edited images and un-edited images. Can you provide a detailed explanaion about it?

[ayaanzhaque]

Sure! So in traditional NeRF training, individual rays are sampled from cameras in the training set, and then the NeRF is volumetrically rendered along the ray to get a color value for the pixel. Thus, for any given training iteration, a random batch of _pixels__ can be sampled from all of the images in the training set, and the NeRF can be optimized on a per-pixel basis. Thus, this allows us to do a parallel "dataset update", where we update an individual image every so often, and then every NeRF optimization step, there will be a mixed set of pixels sampled from edited images and unedited images.

However, for GS, entire images must be rasterized at once, and the loss is computed on a per-image basis. Thus, it would not be possible to have a mixed set of edited signal and unedited signal in any given optimization step. Thus, we opt for an approach where a subset of images in the dataset are updated, and then the GS is optimized by randomly sampling entire images from the dataset, and over n iterations, some number of these iterations will optimize the GS using an edited image. Hopefully this answers your question

[ScienceLi1125]

Thank you for your reply! I have understood what you mean, your answer is very helpful to me!