How can I create a dataset using only point clouds(.ply)?

[1404561326521]

Dear Author,

Thank you for providing the open-source project. I have successfully replicated the model on the dataset you provided, and it performs very well. However, I am currently facing an issue with implementing pose estimation on my own dataset. I only have point cloud models (.ply) without any other information, and all my point cloud models belong to the same category. I have included some examples here. Could you please guide me on how to create the corresponding dataset? I would greatly appreciate it!

Thank you very much!

[cviviers]

Hi @1404561326521,

I had a quick look at the data. What exactly are you trying to do? Do you want the pose of the complete gum+teeth with respect to an image/X-ray?

This work is for obtaining the pose of a rigid object with respect to a camera/X-ray system.

[1404561326521]

Hi @1404561326521,

I had a quick look at the data. What exactly are you trying to do? Do you want the pose of the complete gum+teeth with respect to an image/X-ray?

This work is for obtaining the pose of a rigid object with respect to a camera/X-ray system.

I'm very sorry I didn't accurately describe my issue. My goal is to transform the entire model (teeth and gums) from its original pose to the desired pose, as shown in the figure. I have many models in their original poses, and each has a different pose. However, the final desired pose is consistent across all models. My idea is to use pose estimation methods to achieve this. Since I am new to pose estimation, I don't fully understand it yet, and I'm not sure if pose estimation methods can accomplish my goal.

[cviviers]

Do you have the pose in both cases in world coordinates and just want to know how to adjust the second pose to match the first?

[1404561326521]

Yes, I have two models with different poses. The second pose was obtained by manually adjusting the original model (shown in the first image) to align it with the desired coordinate system (shown in the second image). My ultimate goal is to predict a rigid transformation matrix that can transform any arbitrarily posed model into the pose shown in the second image. As you can see, the point cloud data I previously provided has already been manually adjusted to the desired pose, and their initial poses can be arbitrary.

[cviviers]

If you have the pose is both cases you can simply compute the difference. Unless I am missing something

Computing Pose Difference using Homogeneous Transformation Matrices

The difference between two poses in homogeneous coordinates involves both translation and rotation, represented in a 4x4 transformation matrix.

Homogeneous Transformation Matrix

A pose ( P ) is represented as:

[ P = \begin{bmatrix} R & T \ 0 & 1 \end{bmatrix} ]

Where:

• ( R ) is the ( 3 \times 3 ) rotation matrix.
• ( T ) is the ( 3 \times 1 ) translation vector.
• The bottom row is ( [0 \ 0 \ 0 \ 1] ), which ensures the matrix operates in homogeneous coordinates.

Given Two Homogeneous Matrices

Let’s assume two poses:

• ( P_1 ) for the first object.
• ( P_2 ) for the second object.

[ P_1 = \begin{bmatrix} R_1 & T_1 \ 0 & 1 \end{bmatrix} , \quad P_2 = \begin{bmatrix} R_2 & T_2 \ 0 & 1 \end{bmatrix} ]

Pose Difference Calculation

To compute the relative pose ( \Delta P ) between ( P_1 ) and ( P_2 ), the relative pose is obtained by multiplying the inverse of ( P_1 ) with ( P_2 ):

[ \Delta P = P_1^{-1} \cdot P_2 ]

Inverse of ( P_1 )

The inverse of a homogeneous transformation matrix is given by:

[ P_1^{-1} = \begin{bmatrix} R_1^T & -R_1^T T_1 \ 0 & 1 \end{bmatrix} ]

Pose Difference

Now, compute the relative pose:

[ \Delta P = \begin{bmatrix} R_1^T & -R_1^T T_1 \ 0 & 1 \end{bmatrix} \cdot \begin{bmatrix} R_2 & T_2 \ 0 & 1 \end{bmatrix} ]

Performing the multiplication:

[ \Delta P = \begin{bmatrix} R_1^T R_2 & R_1^T T_2 - R_1^T T_1 \ 0 & 1 \end{bmatrix} ]

Final Result

• Rotation Difference: ( R_{\Delta} = R_1^T R_2 )
• Translation Difference: ( T_{\Delta} = R_1^T (T_2 - T_1) )

Thus, the homogeneous transformation matrix that represents the difference between the two poses is:

[ \Delta P = \begin{bmatrix} R{\Delta} & T{\Delta} \ 0 & 1 \end{bmatrix} ]

Where:

• ( R_{\Delta} = R_1^T R_2 ) (rotation difference).
• ( T_{\Delta} = R_1^T (T_2 - T_1) ) (translation difference, accounting for the rotation of the first pose).

[cviviers]

It seems not all the markdown is supported. If you do not have the pose in the second instance, the problem becomes a bit harder.

[1404561326521]

Sorry, I don't understand what you mean. Now I only have the point cloud model (ply), and I still don't know how to obtain other data related to pose estimation. I looked up relevant materials and found that I can use blender to render the model to synthesize data.

[1404561326521]

I would like to know if the pose estimation task can achieve my purpose, I even tried the point cloud registration method, but the effect is not very good!

[cviviers]

Hi @1404561326521,

I still do not fully understand the problem, but I do not think it is going to be answered by the work discussed in this repository. The work here is to determine the pose of an object from an image, so very different from your use case.

I think it will be best if we close the issue since it is unrelated, but feel free to email me with a clear description of your problem and I can see if I can help? c.g.a.viviers@tue.nl