search

SJTUzhou / 3D-Teeth-Reconstruction-from-Five-Intra-oral-Images

Smartee project: 3D Teeth Reconstruction from Orthodontic Photos
MIT License
44 stars 5 forks source link

Parametric Teeth Model for Customized Teeth Templates #4

Closed grtzsohalf closed 4 months ago

grtzsohalf commented 1 year ago

Hi, it seems that the code for constructing the parametric teeth model from the teeth templates and its data preprocessing steps are not provided in this repo.

Could you provide them as well? Or could you teach me how to transform my customized teeth templates into the parametric teeth model as the files in ssm/cpdGpParams and ssm/eigValVec?

Thank you very much! Look forward to your feedback.

SJTUzhou commented 1 year ago

The code for data preprocessing was done by another student in our group. I will ask him for the source code but I am not sure when he will give me that code.

Our teeth template is provided in "./ssm" directory: 1) "meanAlignedPG_{x}.npy" is a numpy file that stores the point cloud for tooth {x} where {x} is the FDI tooth index. 2) "Munormals.npy" stores the normals of the teeth point clouds with the shape (28,1500,3). 3) "eigVal{x}.npy" and "eigVec_{x}.npy" are the eigenvalues and eigenvectors of tooth {x} following the method of point distribution model. 4) "PoseCovMats.npy" stores the covariance matrices for the relative pose of 28 teeth compared with its their mean pose. 5) "ScaleCovMat.npy" is the covariance matrix of the scales of the 28 teeth.

I will try to update the code for data preprocessing as soon as possible.

grtzsohalf commented 1 year ago

Sounds great!

Thank you @SJTUzhou for the reply. I will wait for the code when it is available.

YangZiz commented 9 months ago

May I ask if there are any results? I am also looking forward to it. @SJTUzhou

SJTUzhou commented 8 months ago

Hi. I have added the preprocessing codes from my colleages. I am sorry for late update and the Chinese comments in these scripts.

cpdGp_align_ssm.py is the process of build tooth shape models from a set of tooth point cloud samples that are roughly aligned through tooth-row registration. It outputs the downsampled (N=1500) tooth point cloud with dense point correspondences and the parameters of the similarity registration. The data eigValVec can be obtained by calling the function "saveEigValVec" in ssm_utils.py.

gp_non_rigid_registration.py is the Gaussian process non-regid registration. With a pre-defined source point cloud, it can perform non-regid deformation to fit the source point cloud to the target ones. It needs some time to do the low-rank approximation after designating the source point cloud.

ssm_utils.py contains a set of functions related to statistical shape models. For example, how to obtain the eigenvalues and eigenvectors from the aligned tooth point clouds with point correspondences and how to evaluate the SSM.

SJTUzhou commented 8 months ago

The functions to compute the covariance matrices of poses and scales of teeth are missing. However, it is easy to calculate them with the saved parameters in the process of building SSMs in cpdGp_align_ssm.py.

grtzsohalf commented 5 months ago

Thanks for your update! I have another question about the training data used for building the tooth shape model: I think your training data are watertight surface meshes right? How do you preprocess raw teeth into these watertight surface meshes? Thanks!

cmsbuptvisoin commented 4 months ago

error I get this error when run cpdGp_align_ssm.py.

SJTUzhou commented 4 months ago

@cmsbuptvisoin
There are some conflicts when using ray and cycpd together. Please see https://github.com/gattia/cycpd/issues/19 You can also change with other registration methods.

SJTUzhou commented 4 months ago

@grtzsohalf Hi. Actually it is not watertight exactly because it is not important to me. I just need the vertex of the surface. Oral scanner can produce a digital mesh of the teeth and gum. First you should segment and identify them using some 3D mesh semantic segmentation deep learning models, you can label each vertex with FDI or gum tag. Then separate each tooth and use some software or algorithms to fix the single tooth mesh by padding a fake "bottom" to the tooth crown. This part in this project was done by our partner company, and the model and methods they used were not shared with me due to patent issues.

cmsbuptvisoin commented 4 months ago

@cmsbuptvisoin There are some conflicts when using ray and cycpd together. Please see gattia/cycpd#19 You can also change with other registration methods.

Tks for your reply and help. I have fixed this problem.

grtzsohalf commented 4 months ago

@grtzsohalf Hi. Actually it is not watertight exactly because it is not important to me. I just need the vertex of the surface. Oral scanner can produce a digital mesh of the teeth and gum. First you should segment and identify them using some 3D mesh semantic segmentation deep learning models, you can label each vertex with FDI or gum tag. Then separate each tooth and use some software or algorithms to fix the single tooth mesh by padding a fake "bottom" to the tooth crown. This part in this project was done by our partner company, and the model and methods they used were not shared with me due to patent issues.

Thanks for your explanation! Congrats again on your nice work.