search

isl-org / Open3D

Open3D: A Modern Library for 3D Data Processing
http://www.open3d.org
Other
11.54k stars 2.32k forks source link

is trajectory file used for the reconstruction system? #5335

Closed TWang1017 closed 2 years ago

TWang1017 commented 2 years ago

Checklist

My Question

Hi, I collected rosbag file with realsense d455 and then used the open3d reconstruction system pipeline to produce a model.

In the meantime, i found the trajectory.log file in the scene folder which i reckon is the reference for the reconstruction pipeline in open3d.

I then tried to make some comparison with image-based method, colmap and openmvs etc with the known poses from open3d (pose from trajectory.log, intrinsics from intrinsic.json). I inverted the trajectory as i need extrinsic (i.e world to camera).

The extrinsics and intrinsics have experimented in the colmap with point_triangulator to produce a sparse model for inspection before dense reconstruction and it succeeded with two issues. The general shape is right. The white is produced by open3D and the sparse model in red is produced by colmap

  1. the scale of the sparse model from colmap is much small than open3d, what i suspect is that the model with open3d pipeline is somehow optimized and the trajectory and intrinsics file are not the final one?
  2. the colmap model has a pyramidal shape, should I undistort the images from rosbag before feeding into the colmap? if so, could you plz suggest me a python code for it?

image

I have been stuck on the scale and pyramidal issue here for one week, any leads and advice are much appreciated.

theNded commented 2 years ago

COLMAP's pose does not have a metrical scale from pure RGB images. RealSense has a metric scale so the trajectory is in meters. To align them, you may want to use point to point registration and turn on with_scaling (http://www.open3d.org/docs/release/python_api/open3d.pipelines.registration.TransformationEstimationPointToPoint.html). This will give you the approximate scale between two trajectories and thus metrics.

COLMAP itself doesn't guarantee the reconstruction accuracy so distortions are more likely inevitable from the pipeline, I wouldn't think calibrating the images will make a big difference.

TWang1017 commented 2 years ago

COLMAP's pose does not have a metrical scale from pure RGB images. RealSense has a metric scale so the trajectory is in meters. To align them, you may want to use point to point registration and turn on with_scaling (http://www.open3d.org/docs/release/python_api/open3d.pipelines.registration.TransformationEstimationPointToPoint.html). This will give you the approximate scale between two trajectories and thus metrics.

COLMAP itself doesn't guarantee the reconstruction accuracy so distortions are more likely inevitable from the pipeline, I wouldn't think calibrating the images will make a big difference.

Thanks you very much for the hints, will try it out!