Help to use ground-truth poses with COLMAP

[leblond14u]

check this to use your own camera settings https://colmap.github.io/faq.html. I think we need to adjust the point3d.txt file so that we can upload our own point cloud data, but it seems that here https://github.com/graphdeco-inria/gaussian-splatting/issues/118 it uploads them directly to the transforms.json file.

Originally posted by @meriemjabri in https://github.com/graphdeco-inria/gaussian-splatting/issues/205#issuecomment-1722250758

[leblond14u]

Hi,

As my previous attempts to use the ground-truth poses directly with 3DGS are unfruitful #702 , I'm trying to process my new dataset scene (ICL NUIM) with COLMAP and the ground-truth poses (The convention used for the dataset is camera to world transformation). I created the cameras.txt, points3D.txt, images.txt in a sparse/0 folder and populated the images.txt and cameras.txt files with my extrinsics and intrinsics respectively.

I then applied the following colmap commands :

colmap feature_extractor     
--database_path Datasets/Augmented_ICL-NUIM_Dataset/livingroom_1_colmap_GT/distorted/database.db     
--image_path Datasets/Augmented_ICL-NUIM_Dataset/livingroom_1_colmap_GT/input 
--SiftExtraction.use_gpu 'yes' 
--ImageReader.single_camera 1 
--ImageReader.camera_model PINHOLE 
--ImageReader.camera_params "525.0, 525.0, 319.5, 239.5"

colmap exhaustive_matcher 
--database_path Datasets/Augmented_ICL-NUIM_Dataset/livingroom_1_colmap_GT/distorted/database.db

colmap mapper         
--database_path Datasets/Augmented_ICL-NUIM_Dataset/livingroom_1_colmap_GT/distorted/database.db         
--image_path Datasets/Augmented_ICL-NUIM_Dataset/livingroom_1_colmap_GT/input         
--output_path  Datasets/Augmented_ICL-NUIM_Dataset/livingroom_1_colmap_GT/distorted/sparse/0         
--Mapper.ba_global_function_tolerance "0.000001" 
--input_path Datasets/Augmented_ICL-NUIM_Dataset/livingroom_1_colmap_GT/sparse/0

colmap image_undistorter     
--image_path Datasets/Augmented_ICL-NUIM_Dataset/livingroom_1_colmap_GT/input     
--input_path Datasets/Augmented_ICL-NUIM_Dataset/livingroom_1_colmap_GT/distorted/sparse/0     
--output_path Datasets/Augmented_ICL-NUIM_Dataset/livingroom_1_colmap_GT/     
--output_type COLMAP

The processing finishes but I'm not sure I have the right outputs as I don't have anything changed in my sparse/0 folder. image.txt stays unchanged as well as points3D.txt. The official documentation being a bit cryptic I would appreciate any help on this.

Thanks in advance, Best,

Hugo

[kangy11]

Hello, I have encountered a similar issue. May I ask if you have resolved this problem now?

[leblond14u]

Yes solved it yesterday,

There's no need to recompute anything with COLMAP. The documentation is missleading. If you are already able to populate your image.txt and camera.txt file and you have your ground truth txt or ply pointcloud, just run the 3DGS with those files in your sparse/0 folder. The colmap dataloader will simply take the camera.txt intrinsics and camera poses from image.txt.

[kangy11]

Thank you for your reply. It seems my situation is a bit different from yours. I want to use GT camera poses to obtain the initial point clouds, rather than using ground truth point clouds.

[leblond14u]

Hi @kangy11. If you also possess depth images a simple re-projection of the depth from the ground truth camera poses will give you a point-cloud. If you don't and still want to get a point-cloud maybe consider depth estimation networks.

[Pacifica-H]

Yes solved it yesterday,

There's no need to recompute anything with COLMAP. The documentation is missleading. If you are already able to populate your image.txt and camera.txt file and you have your ground truth txt or ply pointcloud, just run the 3DGS with those files in your sparse/0 folder. The colmap dataloader will simply take the camera.txt intrinsics and camera poses from image.txt.

Hi, I want to use my own groundtruth poses and pointcloud but my pose data has the format as below:

Quaternion is defined in JPL form, and the coordinate system used is the same as colmap, the ground-truth pose gives the position and orientation of the event camera with respect to the world(c2w). I wonder what I should do to adjust these data to 3d-gs?

[leblond14u]

You can check the data format for the image.txt file here .

[Pacifica-H]

You can check the data format for the image.txt file here .

Should I change my pose file into image.txt format or change the R T loading code of 3dgs https://github.com/graphdeco-inria/gaussian-splatting/blob/b17ded92b56ba02b6b7eaba2e66a2b0510f27764/scene/dataset_readers.py#L82

[leblond14u]

The easiest thing to do in my opinion is to convert your data format into the COLMAP one to avoid messing with the dataloading sequence. But you could very well code your own, I've done it too. It boils down to personal preferences at this point.

[pattgene]

Yes solved it yesterday,

There's no need to recompute anything with COLMAP. The documentation is missleading. If you are already able to populate your image.txt and camera.txt file and you have your ground truth txt or ply pointcloud, just run the 3DGS with those files in your sparse/0 folder. The colmap dataloader will simply take the camera.txt intrinsics and camera poses from image.txt.

Thank you for your great guiding here. I am working similar thing, I want to use my own point cloud and pose to train 3dgs. I am a little bit confuse, it seem you modified sparse/0 folder with your own

1. image.txt(cam pose)
2. camera.txt (cam intrinsic)
3. pointcloud.ply Is this correct? Then just run 3dgs and the fuction ''readColmapSceneInfo'' will then be able to process your modified data?

You get these 3 from: [http://redwood-data.org/indoor/dataset.html](ICL NUM) right? May I see how you structure image.txt, camera.txt, pointcloud.txt please. I am confuse by the COLMAP documentation on how to format those.