isl-org / Open3D

Open3D: A Modern Library for 3D Data Processing
http://www.open3d.org
Other
11.35k stars 2.29k forks source link

Summarize the bug (e.g., "Segmentation Fault for Colored Point Cloud Registration") #6148

Closed flydragon2018 closed 1 year ago

flydragon2018 commented 1 year ago

Checklist

Describe the issue

draw_geometries() not working!

I can't find the data conversion between cpu and cuda devices.

Steps to reproduce the bug

import time 

import numpy as np 
import open3d as o3d
import open3d.core as o3c
import copy

def draw_registration_result(source, target, transformation):
    source_temp = copy.deepcopy(source)
    target_temp = copy.deepcopy(target)
    source_temp.paint_uniform_color([1, 0.706, 0])
    target_temp.paint_uniform_color([0, 0.651, 0.929])
    source_temp.transform(transformation)

    print(type(source_temp))
    print(type(target_temp))
    s=o3d.t.geometry.PointCloud(source_temp)
    t=o3d.t.geometry.PointCloud(target_temp)

    o3d.visualization.draw_geometries([s,t])

# Input point-clouds
demo_icp_pcds = o3d.data.DemoICPPointClouds('~/open3d_data/extract/DemoICPPointClouds')
source = o3d.t.io.read_point_cloud(demo_icp_pcds.paths[0])
target = o3d.t.io.read_point_cloud(demo_icp_pcds.paths[1])

voxel_sizes = o3d.utility.DoubleVector([0.1, 0.05, 0.025])

# List of Convergence-Criteria for Multi-Scale ICP:
criteria_list = [
    o3d.t.pipelines.registration.ICPConvergenceCriteria(relative_fitness=0.0001,
                                relative_rmse=0.0001,
                                max_iteration=20),
    o3d.t.pipelines.registration.ICPConvergenceCriteria(0.00001, 0.00001, 15),
    o3d.t.pipelines.registration.ICPConvergenceCriteria(0.000001, 0.000001, 10)
]

# `max_correspondence_distances` for Multi-Scale ICP (o3d.utility.DoubleVector):
max_correspondence_distances = o3d.utility.DoubleVector([0.3, 0.14, 0.07])

# Initial alignment or source to target transform.
init_source_to_target = o3d.core.Tensor.eye(4, o3d.core.Dtype.Float32)

# Select the `Estimation Method`, and `Robust Kernel` (for outlier-rejection).
estimation =o3d.t.pipelines.registration.TransformationEstimationPointToPlane()

# Save iteration wise `fitness`, `inlier_rmse`, etc. to analyse and tune result.
callback_after_iteration = lambda loss_log_map : print("Iteration Index: {}, Scale Index: {}, Scale Iteration Index: {}, Fitness: {}, Inlier RMSE: {},".format(
    loss_log_map["iteration_index"].item(),
    loss_log_map["scale_index"].item(),
    loss_log_map["scale_iteration_index"].item(),
    loss_log_map["fitness"].item(),
    loss_log_map["inlier_rmse"].item()))

# Setting Verbosity to Debug, helps in fine-tuning the performance.
# o3d.utility.set_verbosity_level(o3d.utility.VerbosityLevel.Debug)

s = time.time()

registration_ms_icp = o3d.t.pipelines.registration.multi_scale_icp(source, target, voxel_sizes,
                                           criteria_list,
                                           max_correspondence_distances,
                                           init_source_to_target, estimation,
                                           callback_after_iteration)

ms_icp_time = time.time() - s
print("Time taken by Multi-Scale ICP: ", ms_icp_time)
print("Inlier Fitness: ", registration_ms_icp.fitness)
print("Inlier RMSE: ", registration_ms_icp.inlier_rmse)

# To create a point cloud on CUDA, specify the device.
s_pcd = o3d.t.geometry.PointCloud(o3c.Device("cuda:0"))
t_pcd=o3d.t.geometry.PointCloud(o3c.Device("cuda:0"))
s_pcd=o3d.t.geometry.PointCloud(source)
t_pcd=o3d.t.geometry.PointCloud(target) 

draw_registration_result(s_pcd,t_pcd, registration_ms_icp.transformation)

Error message

TypeError: draw_geometries(): incompatible function arguments. The following argument types are supported:

  1. (geometry_list: List[open3d.cuda.pybind.geometry.Geometry], window_name: str = 'Open3D', width: int = 1920, height: int = 1080, left: int = 50, top: int = 50, point_show_normal: bool = False, mesh_show_wireframe: bool = False, mesh_show_back_face: bool = False) -> None
  2. (geometry_list: List[open3d.cuda.pybind.geometry.Geometry], window_name: str = 'Open3D', width: int = 1920, height: int = 1080, left: int = 50, top: int = 50, point_show_normal: bool = False, mesh_show_wireframe: bool = False, mesh_show_back_face: bool = False, lookat: numpy.ndarray[numpy.float64[3, 1]], up: numpy.ndarray[numpy.float64[3, 1]], front: numpy.ndarray[numpy.float64[3, 1]], zoom: float) -> None

Invoked with: [PointCloud on CPU:0 [198835 points (Float32)]. Attributes: curvature (dtype = Float32, shape = {198835, 1}), colors (dtype = Float64, shape = {198835, 3}), normals (dtype = Float32, shape = {198835, 3})., PointCloud on CPU:0 [137833 points (Float32)]. Attributes: curvature (dtype = Float32, shape = {137833, 1}), colors (dtype = Float64, shape = {137833, 3}), normals (dtype = Float32, shape = {137833, 3}).]

Expected behavior

visualize properly.

Open3D, Python and System information

- Operating system: Ubuntu 20.04  
- Python version: Python 3.8  
- Open3D version: 0.17.0
- System architecture: x86  
- Is this a remote workstation?:   no
- How did you install Open3D?: pip  
- Compiler version (if built from source):

Additional information

none

theNded commented 1 year ago

Please use o3d.visualization.draw([s,t]) or o3d.visualization.draw_geometries([s.to_legacy(),t.to_legacy()])