rerun-io / rerun

Visualize streams of multimodal data. Free, fast, easy to use, and simple to integrate. Built in Rust.
https://rerun.io/
Apache License 2.0
7.09k stars 354 forks source link

Supporting SIMPLE_RADIAL camera, default from COLMAP #2542

Open relh opened 1 year ago

relh commented 1 year ago

I remember talking to you guys about this at CVPR but I forget what the resolution was. I have run COLMAP with the default settings which generates a camera with SIMPLE_RADIAL parameters. However, when running the structure from motion demo with my own COLMAP output, I ran into the assertion error about camera.model == 'PINHOLE' (because the default from COLMAP is 'SIMPLE_RADIAL').

Is your feature request related to a problem? Please describe. I want to be able to run rerun with my default output from COLMAP.

Describe the solution you'd like Support more cameras!

Describe alternatives you've considered I can still run it without the assert, it just looks funny.

Additional context If you could support a weak-perspective camera too that would be great!

Thanks~.

Wumpf commented 1 year ago

Thanks again for coming by our booth at CVPR and bringing this up. Rerun is definitely in need for more different kind of camera and camera distortion models. We'll need to reconsolidate all these different asks into different workstreams and see where the delimiting lines and imlementation strategies for these are 🤔

relh commented 1 year ago

I was able to get my COLMAP output slightly visualized by rerun by ignoring the assert, shown here in my kitchen incase it's of curiosity.

espresso_rerun

I thought I might be able to get around it by manually editing the cameras.txt file to ditch the k and then reconverting to binary but COLMAP's model_converter does not approve of such things.

relh commented 1 year ago

For anyone with a related issue, I found a way to get around my issue by just closing my ears and yelling "radial distortion doesn't exist".

Basically, I take the COLMAP cameras and throw away the K and I can get an improved visualization:

colmap model_converter \
        --input_path $WORKSPACE_PATH/sparse \
        --output_path $WORKSPACE_PATH/sparse \
        --output_type TXT

colmap model_converter \
        --input_path $WORKSPACE_PATH/sparse/0 \
        --output_path $WORKSPACE_PATH/sparse/0 \
        --output_type TXT

python fix_cam.py
rm $WORKSPACE_PATH/sparse/cameras.bin
rm $WORKSPACE_PATH/sparse/0/cameras.bin

colmap model_converter \
        --input_path $WORKSPACE_PATH/sparse \
        --output_path $WORKSPACE_PATH/sparse \
        --output_type BIN

colmap model_converter \
        --input_path $WORKSPACE_PATH/sparse/0 \
        --output_path $WORKSPACE_PATH/sparse/0 \
        --output_type BIN
pinhole_equivalent

Where fix_cam.py looks like this:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

def convert_camera_model(input_file):
    with open(input_file, 'r') as fin:
        lines = fin.readlines()

    new_lines = []
    for i, line in enumerate(lines):
        if i < 3:  # Skip the first 3 lines (header)
            new_lines.append(line)
        else:
            parts = line.strip().split()
            parts[1] = 'PINHOLE'  # Change camera model
            parts.insert(5, parts[4])  # Duplicate fx as fy
            parts = parts[:-1] # Ditch k
            new_lines.append(' '.join(parts) + '\n')

    with open(input_file, 'w') as fout:
        fout.writelines(new_lines)

convert_camera_model('./pinhole_workspace/sparse/cameras.txt')
convert_camera_model('./pinhole_workspace/sparse/0/cameras.txt')