Ask for prerequisite

[RickMaruyama]

Hello! I am very interested in your project to test on my personal dataset. Could you let me know following prerequisite?

• does your system work with linux20.04? -> I could not find any information for linux about Real Time Viewer, feeling uncertain if it works with linux. If windows is better in this case, I'll choose it.
• What is required datasets to input to system? -> in my understanding, multi-view monocular videos (RGB) + numpy array. but what is the numpy array? Do I need something else?
• What is output format of 4D GS?

Thank you for your advice!!

[lizhan17]

Real Time Viewer: it can works in linux. As if you can build 3dgs viewer for linux. then just modified the code with our provided code. (keep the orignal .git folder in 3dgs)

stay tuned. I plan to push a multi-view videos + turbo version. The quality may be worse but could be trained on desktop.

output format of spacetime gs is ply with additional temporal informaiton. so it can't be directed used with ply editor like blender.

[lizhan17]

correction a multi-view video loader, not for mono camera captured video.

[lizhan17]

please check the new command and functions for mulitview cameras without priors.

see https://github.com/oppo-us-research/SpacetimeGaussians/issues/55#issuecomment-2185372592

[RickMaruyama]

Thank you! I tested now and got following error with changing movie angles. Do you have any tips to succeed on finding image pair?? I think I use some movies tracking some feature points to identify one object though..

==============================================================================
Finding good initial image pair
==============================================================================

  => No good initial image pair found.

Elapsed time: 0.004 [minutes]
ERROR: failed to create sparse model
ERROR:root:Mapper failed with code 256. Exiting.
Traceback (most recent call last):
  File "script/pre_no_prior.py", line 291, in <module>
    convert_selected_cam_matrix_to_colmapdb(colmap_project_root, frame_num, pose_ref_frame_num, image_ext)
  File "script/pre_no_prior.py", line 123, in convert_selected_cam_matrix_to_colmapdb
    cam_extrinsics = read_extrinsics_binary(cameras_extrinsic_file)
  File "./thirdparty/gaussian_splatting/colmap_loader.py", line 175, in read_extrinsics_binary
    with open(path_to_model_file, "rb") as fid:
FileNotFoundError: [Errno 2] No such file or directory: '/home/ubuntu/SpacetimeGaussians/data/videos/point/colmap_0/distorted/sparse/0/images.bin'

[lizhan17]

sorry, currently code base can't support moving cameras captured videos.

the colmap should either support single moving camera with static content (you can think it is a multi-camera with images) or static multi-camera videos.

Moving camera support should be a future work.

[RickMaruyama]

I am sorry, probably I made you minsunderstood. I have a dataset of 3-5 movies taken at fixed place but from different angles. so the cameras are not moving, sorry. Since it failed to find good initial images of pair, I chose 3 out of 5 one another, but all failed. So probably it could be a cause from matching algorithm used in colmap is not capable to find features in my limited simultaneous movies, maybe because the features are not enough. I'd like you to give me advice or opinion on it. Maybe adopting anothre feature matching algorithm?

Again, thank you for your sincere support.

[lizhan17]

I guess the colmap require many cameras with overlapped content from different angle. (the more camera the better)