How to correctly use AI video to reconstruct scenes?

[DreamerYinYu]

Hello, I am trying to use cultural video platforms such as Runway to reconstruct scenes through 3DGS, but some of the results are very poor. Is it a problem with the video or other issues? thank you. Iteration count 30000

![Uploading 02.png…]()

[DreamerYinYu]

[jaco001]

This is hard for algorytm:

1. From AI video each frame is slightly different and you loose consistency of detail.
2. For good reconstruction you need shots from many angles of same scene. With one 'sweep' of scene, your reconstruction will be bound to same camera move at last.

[AsherJingkongChen]

You need a single-view reconstruction for solution, not 3D-GS and COLMAP.

[DreamerYinYu]

您需要一个单一视图的重建来求解，而不是 3D-GS 和 COLMAP。

Hello, is it related to the camera model SIMPLE.PINHOLE or PINHOLE? thank you

[DreamerYinYu]

您需要一个单一视图的重建来求解，而不是 3D-GS 和 COLMAP。

您好，是不是这个视频涉及到镜头畸变，而我再使用COLMAP时选择的是SIMPLE PINHOLE模型，两者不匹配的原因？

[DreamerYinYu]

您需要一个单一视图的重建来求解，而不是 3D-GS 和 COLMAP。

我是不是应该选择没有畸变的AI视频？或者在使用COLMAP时，选择考虑畸变 的相机模型？希望能得到您的回复，感谢！

[AsherJingkongChen]

您需要一个单一视图的重建来求解，而不是 3D-GS 和 COLMAP。

我是不是应该选择没有畸变的AI视频？或者在使用COLMAP时，选择考虑畸变 的相机模型？希望能得到您的回复，感谢！

@DreamerYinYu I assumed that you have not enough proper frames to predict the point cloud.

However, I think you can check the undistorted images (new files after running convert.py). They should all be converted to pinhole ones by COLMAP.

Besides, you can use tools to visualize the initial state of points, like COLMAP's GUI viewer and SIBR viewer.

If you are sure that you have not enough proper frames for 3D-GS, you can try models able to handle sparse view tasks like InstantSplat.