Incremental Reconstruction For 3D GS

[LaFeuilleMorte]

Hi, I've read your paper. Very smart ideas! So in your discussion I learned that this method can only deal with two images input. I recently read a paper about improvements on dust3r like methods. It's named:

3D Reconstruction with Spatial Memory https://hengyiwang.github.io/projects/spanner

So I was wondering if multi-view image inputs could be done like that:

1. input multi-view images
2. regression of points in global coordinate
3. regression of gaussian params with point clouds and multi-view images.

[btsmart]

I would be very interested in seeing Splatt3R extended to incremental reconstruction methods like Spann3R. I have not yet had the chance to dig into Spann3R, but I suspect if this were done in the most straightforward, naive way, each time a particular point in the scene is observed in a camera frame, a new Gaussian would be placed there. This could work, but I think there would be a few issues, as many Gaussians would be competing to represent the same region, larger gaussians would occlude smaller ones, any prediction errors would accumulate in the scene, etc. Spann3R's spatial memory might be able to help with a few of these problems, but you would likely want to be able to use information from future frames when predicting Gaussians, rather than just previous frames.

However, I would be very interested in seeing the results from this process. There is clearly a lot of interest in feed-forward Gaussian reconstruction from more than two images, and I suspect many people (including myself) will be experimenting to find something that works well.