search

isl-org / Open3D

Open3D: A Modern Library for 3D Data Processing
http://www.open3d.org
Other
11.44k stars 2.3k forks source link

Cannot make deform_as_rigid_as_possible method work on other meshes besides Armadillo.ply #4049

Open goloste opened 3 years ago

goloste commented 3 years ago

Issue Description Hello, I'm trying to deform a mesh by using the as rigid as possible (ARAP) method as shown in this example.

The method works fine on the Armadillo mesh, producing the expected deformation, but fails to produce ARAP deformation on other meshes, either because it gives the following error:

or because the single deformation produced is the translation of the handle vertex to the desired position.

I wrote the code below to illustrate the issue by following the original example:

import open3d as o3d
import numpy as np
import copy

# Load meshes
myMesh = o3d.io.read_triangle_mesh("rotated_template.ply")
armadillo_mesh = o3d.io.read_triangle_mesh("Armadillo.ply")

print("myMesh mesh info: ", myMesh)
print("armadillo mesh info: ", armadillo_mesh)

# Paint meshes yellow
yellow = [0.8, 0.8, 0.1]
myMesh.paint_uniform_color(yellow)
armadillo_mesh.paint_uniform_color(yellow)

# Scale and translate myMesh mesh
myMesh.scale(45, center=myMesh.get_center())
myMesh.translate((100.0, 0.0, 0.0))

# Rotate armadillo mesh 180° about y axis
R = armadillo_mesh.get_rotation_matrix_from_xyz((0,np.pi,0))
armadillo_mesh.rotate(R)

# Create a sphere to be used as a marker
rad = 1
marker = o3d.geometry.TriangleMesh.create_sphere(radius=rad, resolution=20)

myMesh_handle_idx = 23
armadillo_handle_idx = 1593

myMesh_start_marker_pos = np.asarray(armadillo_mesh.vertices[armadillo_handle_idx])
myMesh_start_marker = copy.deepcopy(marker)
myMesh_start_marker.paint_uniform_color([0.0, 1.0, 0.0])
myMesh_start_marker.translate(myMesh_start_marker_pos)

armadillo_start_marker_pos = np.asarray(myMesh.vertices[myMesh_handle_idx])
armadillo_start_marker = copy.deepcopy(marker)
armadillo_start_marker.paint_uniform_color([0.0, 1.0, 0.0])
armadillo_start_marker.translate(armadillo_start_marker_pos)

offset = (4.0, 4.0, 4.0)
myMesh_stop_marker_pos = myMesh_start_marker_pos + offset
myMesh_stop_marker = copy.deepcopy(marker)
myMesh_stop_marker.paint_uniform_color([1.0, 0.0, 0.0])
myMesh_stop_marker.translate(myMesh_stop_marker_pos)

armadillo_stop_marker_pos = armadillo_start_marker_pos + offset
armadillo_stop_marker = copy.deepcopy(marker)
armadillo_stop_marker.paint_uniform_color([1.0, 0.0, 0.0])
armadillo_stop_marker.translate(armadillo_stop_marker_pos)

# Display meshes
o3d.visualization.draw_geometries([myMesh,
                                   armadillo_mesh,
                                   myMesh_start_marker,
                                   armadillo_start_marker,
                                   myMesh_stop_marker,
                                   armadillo_stop_marker], mesh_show_wireframe=True)

# Perform ARAP deformation on armadillo
armadillo_vertices = np.asarray(armadillo_mesh.vertices) # Get armadillo vertices

static_ids = [idx for idx in np.where(armadillo_vertices[:, 2] < 10)[0]]  # Get indexes for vertices with z < 10
static_pos = []
for id in static_ids:
    static_pos.append(armadillo_vertices[id]) # Append vertex current position

constraint_ids = o3d.utility.IntVector(static_ids + [armadillo_handle_idx])

# Set the desired position for handle vertex after ARAP transformation
armadillo_handle_pos = armadillo_vertices[armadillo_handle_idx] + offset
constraint_pos = o3d.utility.Vector3dVector(static_pos + [armadillo_handle_pos])

print("Performing ARAP on Armadillo")
with o3d.utility.VerbosityContextManager(o3d.utility.VerbosityLevel.Debug) as cm:
    deformed_armadillo = armadillo_mesh.deform_as_rigid_as_possible(constraint_ids,
                                               constraint_pos,
                                               max_iter=30)

# Perform ARAP deformation on myMesh
myMesh_vertices = np.asarray(myMesh.vertices) # Get myMesh vertices

static_ids = [idx for idx in np.where(myMesh_vertices[:, 2] < 10)[0]]  # Get indexes for vertices with z < 10
static_pos = []
for id in static_ids:
    static_pos.append(myMesh_vertices[id]) # Append vertex current position

constraint_ids = o3d.utility.IntVector(static_ids + [myMesh_handle_idx])

# Set the desired position for handle vertex after ARAP transformation
myMesh_handle_pos = myMesh_vertices[myMesh_handle_idx] + offset
constraint_pos = o3d.utility.Vector3dVector(static_pos + [myMesh_handle_pos])

print("Performing ARAP on MyMesh")
with o3d.utility.VerbosityContextManager(o3d.utility.VerbosityLevel.Debug) as cm:
    deformed_myMesh = myMesh.deform_as_rigid_as_possible(constraint_ids,
                                                            constraint_pos,
                                                            max_iter=30)

o3d.visualization.draw_geometries([myMesh,
                                   armadillo_mesh,
                                   myMesh_start_marker,
                                   armadillo_start_marker,
                                   myMesh_stop_marker,
                                   armadillo_stop_marker,
                                   deformed_armadillo,
                                   deformed_myMesh], mesh_show_wireframe=True, mesh_show_back_face=True)

Console output

myMesh mesh info:  TriangleMesh with 75000 points and 25000 triangles.
armadillo mesh info:  TriangleMesh with 172974 points and 345944 triangles.
Performing ARAP on Armadillo
[Open3D DEBUG] [DeformAsRigidAsPossible] setting up S'
[Open3D DEBUG] [DeformAsRigidAsPossible] done setting up S'
[Open3D DEBUG] [DeformAsRigidAsPossible] setting up system matrix L
[Open3D DEBUG] [DeformAsRigidAsPossible] done setting up system matrix L
[Open3D DEBUG] [DeformAsRigidAsPossible] setting up sparse solver
[Open3D DEBUG] [DeformAsRigidAsPossible] done setting up sparse solver
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=0, energy=5.124176e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=1, energy=2.717055e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=2, energy=2.259923e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=3, energy=2.056035e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=4, energy=1.938971e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=5, energy=1.862051e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=6, energy=1.807044e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=7, energy=1.765457e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=8, energy=1.732802e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=9, energy=1.706462e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=10, energy=1.684791e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=11, energy=1.666682e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=12, energy=1.651359e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=13, energy=1.638256e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=14, energy=1.626951e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=15, energy=1.617117e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=16, energy=1.608503e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=17, energy=1.600907e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=18, energy=1.594169e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=19, energy=1.588161e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=20, energy=1.582775e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=21, energy=1.577924e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=22, energy=1.573538e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=23, energy=1.569555e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=24, energy=1.565924e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=25, energy=1.562604e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=26, energy=1.559557e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=27, energy=1.556753e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=28, energy=1.554165e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=29, energy=1.551770e+01
Performing ARAP on MyMesh
[Open3D DEBUG] [DeformAsRigidAsPossible] setting up S'
[Open3D DEBUG] [DeformAsRigidAsPossible] done setting up S'
[Open3D DEBUG] [DeformAsRigidAsPossible] setting up system matrix L
[Open3D DEBUG] [DeformAsRigidAsPossible] done setting up system matrix L
[Open3D DEBUG] [DeformAsRigidAsPossible] setting up sparse solver
[Open3D DEBUG] [DeformAsRigidAsPossible] done setting up sparse solver
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=0, energy=7.978340e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=1, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=2, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=3, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=4, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=5, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=6, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=7, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=8, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=9, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=10, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=11, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=12, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=13, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=14, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=15, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=16, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=17, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=18, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=19, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=20, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=21, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=22, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=23, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=24, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=25, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=26, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=27, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=28, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=29, energy=7.108065e+01

Process finished with exit code 0

Images These are the loaded meshes myMesh mesh info: TriangleMesh with 75000 points and 25000 triangles. armadillo mesh info: TriangleMesh with 172974 points and 345944 triangles. original_meshes

The green spheres mark the vertex selected as handle point for the transformation, the red spheres mark the desired position after the transformation armadillo_markers myMesh_markers

The transformation results are shown below, The yellow meshes are the original meshes, the transformation result is shown in grey ARAP_result armadillo_result myMesh_result

Environment:

Additional context I tried already changing handle vertex, using different scales, loading different meshes with different densities but i keep getting either a "Failed to build solver" error or an incorrect deformation

senhypevr commented 1 year ago

hi goloste, did you solove this problem? I have the same issue as you. Thanks! @goloste