Colmap unable to find poses for custom datasets

[Anji-Builds]

I have been trying to set up custom dataset for training on nerfstudio using KITTI360 and pandaset. For the pose estimation in the above screenshot, I have used pandaset sequence 001 -->camera --> front camera images. I was able to only get pose estimation for 5 images.

I also tried for KITTI360 dataset (data_2d_nvs_drop50 --> train_00 --> 2013_05_28_drive_0008_sync -->image_00), which gave even poorer results.

Is there any way to improve the results? Or are there any additional requirements needed to get the poses for all the images? Am I missing something?

Furthermore, what is the minimum recommended no. of image poses required for training the nerf models?

[nectorv]

I have similar issue. Colmap use to work on one of my custom datasets but now, for the same images, it can find poses for only 2 of the 150 images.

[alancneves]

By default, NerfStudio uses COLMAP with vocab_tree matcher to find feature matches. Maybe, for this dataset, it's not good enough to find good matches.

• To solve that, change the --matching-method param to exhaustive, when using the ns-process.

ns-process-data images --matching-method=exhaustive --data <images_folder> --output-dir <output_dir>

where _imagesfolder is the root of the images folder and _outputdir is obvious.

• Another approach is to test different configs/params on COLMAP previously and import the result in NerfStudio format. With the sparse data in hand, pass it to the NerfStudio during the dataset processing step as bellow:

ns-process-data images --data <images_folder> --output-dir <output_dir> --skip-colmap --colmap-model-path <colmap_path>

where _imagesfolder is the root of the images folder, _outputdir is obvious and _colmappath is the model to be used, e.g., /data/sparse/0.

NerfStudio will build the image folder with the colmap folder that you provided. After that, you keep with the training step as usual, passing the output_dir as a parameter.

ns-train nerfacto .... --data <output_dir>

[alancneves]

I have similar issue. Colmap use to work on one of my custom datasets but now, for the same images, it can find poses for only 2 of the 150 images.

I've saw someone saying that should be related to a bug on COLMAP 3.10 version. Worths to try another, such as 3.8

https://github.com/colmap/colmap/issues/2427

[abrahamezzeddine]

COLMAP creates several models and tries to join them into a single model as I understand, if that does not work, the models will remain split.

I have however noticed that out of the x models, one of them is a good one, but nerfstudio is unable to pick it.

Can you check the COLMAP folder and see if you several models and import them into the COLMAP GUI interface and inspect them?

Then you can move the “good” model into sparse/0 and still run nerfstudio nerf.

I explained the issue here #3435

[nectorv]

The solution for me was to use https://github.com/NVlabs/instant-ngp/blob/master/scripts/colmap2nerf.py with colmap.

[Parmisian]

It is possible to just use colmap to create the sparse/0/*.bin directory either with GUI or from the terminal and then during the ns-data-process step, use the --skip-colmap option. Of course, this is not an optimal solution. I believe that the matcher type might be causing generation of more than one model.