btsmart
commented
3 weeks ago Hello! Unfortunately our method is currently only designed for inference from one or two images.
DUSt3R and MASt3R support more than two images by making a prediction for each image pair and then performing explicit pose prediction and global alignment to combine the generated pointclouds. A similar method could work with our approach by superimposing the generated Gaussian splats on top of each other, however that would result in many Gaussians overlapping each other (with potentially larger Gaussians hiding the details given by smaller Gaussians), all of the erroneously predicted Gaussians would accumulate in the scene (which DUSt3R/MASt3R partially address by filtering out low confidence points), and the global alignment is a per-scene optimization process that we wished to avoid with our model.
This is an interesting problem though and we plan to explore how to more effectively perform inference from more than two images in the future.
GuoPingPan
During the hugging face experience, I found that if there are more than two images, an error will be reported. How can I expand it to multiple images?