custom_dataset docs: Workflow for bundler -> colmap for RealityCapture, Metashape etc

[jo-chemla]

Is your feature request related to a problem? Please describe. For testing Gaussian-Splatting, I wanted to create a workflow to ingest data from RealityCapture. The bundler standard, although dating from 2007, is one of the more common formats for storing camera registration.

I used naver/kapture toolset for bundler -> kapture -> colmap conversion. This workflow can be useful to others and could be added in the custom_dataset doc - can do a PR if anyone think it's worth it!

Describe the solution you'd like The complete workflow is descibed in this thread, and relies on this kapture_export_colmap.py script

:: Export from RC an imagelist.lst + bundle.out
:: Convert RC imagelist.lst to imagelist-local.lst

:: Install Kapture
pip install kapture

:: Bundler -> Kapture
py C:\Python310\Scripts\kapture_import_bundler.py -v debug -i dataset-bundler\bundle.out
  -l dataset-bundler\imagelist-local.lst -im dataset-bundler\images
  --image_transfer link_absolute -o dataset-kapture --add-reconstruction

# Kapture crop remove black strips + radial->pinhole conversion using kapture-cropper.py
# https://gist.github.com/jo-chemla/258e6e40d3d6c2220b29518ff3c17c40
py kapture-cropper.py -v info -i dataset-kapture\ --border_px 10

:: Kapture -> Colmap
py C:\Python310\Scripts\kapture_export_colmap.py -v debug -f -i dataset-kapture
  -db dataset-colmap\colmap.db --reconstruction dataset-colmap\reconstruction-txt

:: Optional Colmap txt to bin conversion
mkdir dataset-colmap\sparse\0
COLMAP.bat model_converter --input_path dataset-colmap\reconstruction-txt
  --output_path dataset-colmap\sparse\0 --output_type BIN

:: Convert cameras from RADIAL to PINHOLE NOT NEEDED ANYMORE
:: xcopy dataset-bundler\images dataset-colmap\images /i
:: curl https://raw.githubusercontent.com/graphdeco-inria/gaussian-splatting/main/convert.py -o gaussian-splatting-pinhole-cameras-convert.py
:: py gaussian-splatting-pinhole-cameras-convert.py -s dataset-colmap --colmap_executable ..\COLMAP-3.8-windows-cuda
:: [--resize] #If not resizing, ImageMagick is not needed

Describe alternatives you've considered A nerf-studio workflow already exists for ingesting RealityCapture or Metashape data, but this one, based on bundler data, is more generic and could be applied to any software exporting Bundler.

[dedoogong]

@jo-chemla hello! thanks for your fantastic awesome job! Yes I 1000% agree with your opinion. as Metashpe, RC also should be considered main stream tool and it should be possible to get camera registration data from RC seemlessly.

I run your codes successfully for running gaussian splatting from inria code(not nerf studio). But when I run convert.py in GS git repo, I got same errors repeatedly like ' Processed file [1/1047] Name: 00001.jpg ERROR: Single camera specified, but images have different dimensions. Processed file [2/1047] Name: 00000.jpg Dimensions: 3743 x 2160 Camera: #1 - OPENCV Focal Length: 4491.60px Features: 9594 Processed file [3/1047] Name: 00002.jpg ERROR: Single camera specified, but images have different dimensions. Processed file [4/1047] Name: 00003.jpg ERROR: Single camera specified, but images have different dimensions. Processed file [5/1047] Name: 00004.jpg ERROR: Single camera specified, but images have different dimensions. Processed file [6/1047] Name: 00005.jpg ERROR: Single camera specified, but images have different dimensions. Processed file [7/1047] Name: 00006.jpg ERROR: Single camera specified, but images have different dimensions. Processed file [8/1047] Name: 00007.jpg ERROR: Single camera specified, but images have different dimensions. Processed file [9/1047] Name: 00008.jpg ERROR: Single camera specified, but images have different dimensions. ... ' I checked the exported bundle.out file again, and it shows that there is 1047 cameras and yes I have 1047 undistorted images(so all images have different size) exported together with the bundle.out. why it shows "Single" camera?? do you have any idea or guess?

Ah, plus, I didnt' use "--add-reconstruction" option as I didn't run reconstruction step in RC as I just wanted to get alignment result. Should I run recon step and add the option? As the number of points is 0 in the bundle.out, I manually changed 0 to 501294 by calculating the line numbers for points(end line number - camera line number / 6).

Please help me!

Thank you!

[dedoogong]

even after reconstruction step and export bundle out again, it shows "0" number of point like "# Bundle file v0.3 1047 0 .... " but there are 3 milion lines for points. can you give me some hint? what is correct pipeline or option settings for exporting bundle out? I saw your export setting in "https://forums.unrealengine.com/t/support-colmap-export/1277698/5" but it shows only undistored image exporting options and it's not related to point option.

[YaroslavShchekaturov]

I have the same problem!

[Parmisian]

Hey, for those who might still have struggle with this: I solved the problem by keeping the exporting images resolution "fixed" during the bundler export (though now I have other errors)

[jo-chemla]

Hi all, Sorry I did not receive the notifications on the thread. Indeed the workflow has some hiccups at the moment, like processing the reconstruction in RC is preferred to have a point count different than zero. A recent RC release also fixed the black borders errors, so a portion of this workflow can now be avoided. Please don't hesitate to report back if you have more precise errors!

[Sudhansh6]

Hi Thanks for the scripts! How do we use the generated colmap.db file for, say, Gaussian Splatting?

[jo-chemla]

Hi @nerfstudio-project team. Interested in adding the workflow described in the original thread to the custom_dataset doc?

Now that parsers are implemented for metashape, realitycapture etc it might seem less useful, but allowing import for any bundle.out alignment might still be useful to the doc!

Should I make a simple PR copy-pasting my solution?