SJoJoK / 3DGStream

[CVPR 2024 Highlight] Official repository for the paper "3DGStream: On-the-fly Training of 3D Gaussians for Efficient Streaming of Photo-Realistic Free-Viewpoint Videos".
https://sjojok.github.io/3dgstream
MIT License
302 stars 18 forks source link

what do you mean by "only the transformed ones carried into the next timestep." #3

Closed NeutrinoLiu closed 5 months ago

NeutrinoLiu commented 5 months ago

in the caption of figure 3 of your CVPR paper, you mentioned that "only the transformed ones carried into the next timestep". can you further elaborate this statement.

for instance, gaussians in the time stamp t are: S_t = {s1 .... sn} you have derived transformations of your gaussians through MLP and derived new gaussians for the next moment: B_t+1 = {b1 ... bm} where bi = si + △si you also have your new additional gaussians: A_t+1 = {a1 ... ak} then for the next time stamp rendering, guassians are: S_t+1 = {...B_t+1, ...A_t+1}

is this "only the transformed ones carried into the next timestep" means:

In another word, did you perform pruning and split for stage 1? or to say, the number of "basic" gaussians has never changed since timestamp 0 ? Also want to make sure that, NTC is per-timestamp correct? means the total memory utilization will be sizeofNTC * timestamps

SJoJoK commented 5 months ago

In another word, did you perform pruning and split for stage 1?”

No, we didn't.

the number of "basic" gaussians has never changed since timestamp 0 ?

Yes.

Also want to make sure that, NTC is per-timestamp correct? means the total memory utilization will be sizeofNTC * timestamps

Yes.

Actually, all these questions might be confirmed/answered by scanning the code of the renderer of our 3DGStream Viewer, especially how we implemented CUDARenderer.draw().

Hope this helps.

NeutrinoLiu commented 5 months ago

In another word, did you perform pruning and split for stage 1?”

No, we didn't.

the number of "basic" gaussians has never changed since timestamp 0 ?

Yes.

nice work. looking forward to your open source code.