search

buaacyw / GaussianEditor

[CVPR 2024] GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splatting
https://buaacyw.github.io/gaussian-editor/
Other
1k stars 47 forks source link

Does semantic tracking occur during local editing? #18

Closed MnKnight1 closed 4 months ago

MnKnight1 commented 7 months ago

I'm still a bit confused about the paper: does semantic tag annotation happen during the editing process? I see that semantic tracking is implemented in update_mask(), but I don't see Gaussian Semantic Tracing during the local edit optimization process in function edit() or in class InstructPix2PixGuidance()\EditGuidance()

buaacyw commented 7 months ago

No. We only add semantic tag at the beginning of training. Given 2D segmenation mask from SAM, this tag annotation takes few microseconds (the time of a rendering). During training, the masked area changed due to the updating of Gaussians with semantic tag. And if these Gaussians generate new Gaussian points through densification, the new Gaussians are also with the semantic tag.