search

SJTUzhou / 3D-Teeth-Reconstruction-from-Five-Intra-oral-Images

Smartee project: 3D Teeth Reconstruction from Orthodontic Photos
MIT License
44 stars 5 forks source link

readme

Smartee project: Model-based 3D Teeth Reconstruction from Five Intra-oral Orthodontic Photos

preprocessing

Please see cpdGp_align_ssm.py, ssm_utils.py, gp_non_rigid_registration.py

Usage

1） To train a teeth boundary segmentation model:

1-1） Put your train and validation data into the folders in ./seg/train and ./seg/valid respectively

1-2） Train a model with default settings, Run "python -m seg.run_train"

Note: The labeled teeth boundary is dilated during training and evaluation to serve as the real label. It is because the manually labelel teeth boundary is too thin and not highly accurate for edge detection. The dilation process is controlled by a factor named EXPANSION_RATE in ./seg/seg_const.py

Attention: the provided teeth boundary segmentation model is not accurate and a precise teeth boundary is important for the 3D following teeth reconstruction.

2） To run teeth reconstrution demo: "python main.py"

3） To visualize the projection of teeth reconstruction into input photo: "python visualization.py"

Overview

Based on the work of Wu. et al. [1], we develop a template-based framework leveraging the prior shape knowledge of human teeth to reconstruct digital 3D models of upper and lower teeth from the typical five orthodontic intra-oral photographs. The framework consists of three phases: parameterization of the arrangement and shape of teeth through statistical shape modelling, U-net based teeth boundary extraction in intra-oral images, and 3D teeth reconstruction based on the prior parametric teeth model.

teeth reconstruction framework

Reconstruction result

Maxillary view Mandibular view Left view Right view Anterior view
orthodontic photo: maxillary view orthodontic photo: mandibular view orthodontic photo: left view orthodontic photo: right view orthodontic photo: anterior view
reconstructed teeth: maxillary view reconstructed teeth: mandibular view reconstructed teeth: left view reconstructed teeth: right view reconstructed teeth: anterior view
projection: maxillary view projection teeth: mandibular view projection teeth: left view projection teeth: right view projection teeth: anterior view

Requirements

Reference

[1] Yizhou Chen, Shuojie Gao, Puxun Tu, and Xiaojun Chen, Automatic 3D Teeth Reconstruction from Five Intra-oral Photos Using Parametric Teeth Model, IEEE Transactions on Visualization and Computer Graphics, 2023, DOI: 10.1109/TVCG.2023.3277914

[2] Wu, C., Bradley, D., Garrido, P., Zollhöfer, M., Theobalt, C., Gross, M. H., & Beeler, T. (2016). Model-based teeth reconstruction. ACM Trans. Graph., 35(6), 220-1.