czh-98 / REALY

REALY: Rethinking the Evaluation of 3D Face Reconstruction (ECCV 2022)
https://www.realy3dface.com/
MIT License
247 stars 19 forks source link

train datasets ? #3

Closed wangjue-wzq closed 2 years ago

wangjue-wzq commented 2 years ago

What are the training sets for these models as listed on the web page ( https://www.realy3dface.com/ )?

wangjue-wzq commented 2 years ago

I used the pre-trained model in deep3dface (https://github.com/sicxu/Deep3DFaceRecon_pytorch) to test (frontal-view), and the error is as follows, which is very different from the results given on the web page. What is the reason?

图片

average nmse@nose = 23.167 average nmse@mouth = 20.792 average nmse@forehead = 37.798 average nmse@cheek = 11.772 median nmse@nose = 23.201 median nmse@mouth = 20.776 median nmse@forehead = 37.569 median nmse@cheek = 14.989 standard deviation nmse@nose = 1.858 standard deviation nmse@mouth = 2.579 standard deviation nmse@forehead = 2.841 standard deviation nmse@cheek = 5.433 average nmse@all = 23.382

图片
czh-98 commented 2 years ago

What are the training sets for these models as listed on the web page ( https://www.realy3dface.com/ )?

We use the official pretrained model provided by these methods. You can refer to these papers for details about experimental settings.

czh-98 commented 2 years ago

I used the pre-trained model in deep3dface (https://github.com/sicxu/Deep3DFaceRecon_pytorch) to test (frontal-view), and the error is as follows, which is very different from the results given on the web page. What is the reason?

图片

average nmse@nose = 23.167 average nmse@mouth = 20.792 average nmse@forehead = 37.798 average nmse@cheek = 11.772 median nmse@nose = 23.201 median nmse@mouth = 20.776 median nmse@forehead = 37.569 median nmse@cheek = 14.989 standard deviation nmse@nose = 1.858 standard deviation nmse@mouth = 2.579 standard deviation nmse@forehead = 2.841 standard deviation nmse@cheek = 5.433 average nmse@all = 23.382

图片

Please check:

  1. whether you used a suitable template (i.e., Deep3D.obj) for evaluation.
  2. whether you have disabled the "process=False" for the "trimesh" package since the default setting to save ".obj" will change the topology.
  3. whether the predictions are aligned correctly.

For Deep3D, the save process (i.e., 2) might be the reason for such an issue.

wangjue-wzq commented 2 years ago

I realize that the unit of nmse is not mm, but the result given is mm. May I ask if there is an accurate conversion relation here?

czh-98 commented 2 years ago

I realize that the unit of nmse is not mm, but the result given is mm. May I ask if there is an accurate conversion relation here?

The scales are derived from the original scans in the HeadSpace dataset and our globally aligned scans (see data/metricial_scale.txt). The evaluation of this repo considers the scale automatically.

wangjue-wzq commented 2 years ago

Thanks!