search

donydchen / mvsplat

🌊 [ECCV'24] MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images
https://donydchen.github.io/mvsplat
Other
498 stars 22 forks source link

how support LLFF Mip-NeRF 360 dataset? #1

Closed goometasoft closed 3 months ago

goometasoft commented 3 months ago

thank your mvsplat , i like it very much !

how support LLFF Mip-NeRF 360 dataset?

donydchen commented 3 months ago

Hi @goometasoft, thanks for your interest in our work.

To support other datasets, one key thing is the camera parameter. MVSplat uses OpenCV-style camera-to-world matrices and normalised camera intrinsics. By organising the datasets in such formats, it should be able to use our pre-trained weights.

Another thing is about view selection. Since MVSplat leverages the cost volume to fuse multi-view information, we assume that the input context views should have enough overlap, e.g., at least 40% to 60%.

We have written data pre-processing scripts and data loaders for DTU and Mip-NeRF 360 in our experiments, data from both datasets works well with our RE10K pretrained weights. But we have not well-cleaned those parts of the codes, and we will release them once we finish organising them in the future. Besides, pixelSplat, which is the code base of MVSplat, has also shared some scripts on how to use data in COLMAP format at https://github.com/dcharatan/pixelsplat/issues/17, you might want to look that up for some more reference.