czh-98 / REALY

REALY: Rethinking the Evaluation of 3D Face Reconstruction (ECCV 2022)
https://www.realy3dface.com/
MIT License
239 stars 19 forks source link

train datasets ? #3

Closed wangjue-wzq closed 1 year ago

wangjue-wzq commented 1 year ago

What are the training sets for these models as listed on the web page ( https://www.realy3dface.com/ )?

wangjue-wzq commented 1 year ago

I used the pre-trained model in deep3dface (https://github.com/sicxu/Deep3DFaceRecon_pytorch) to test (frontal-view), and the error is as follows, which is very different from the results given on the web page. What is the reason?

图片

average nmse@nose = 23.167 average nmse@mouth = 20.792 average nmse@forehead = 37.798 average nmse@cheek = 11.772 median nmse@nose = 23.201 median nmse@mouth = 20.776 median nmse@forehead = 37.569 median nmse@cheek = 14.989 standard deviation nmse@nose = 1.858 standard deviation nmse@mouth = 2.579 standard deviation nmse@forehead = 2.841 standard deviation nmse@cheek = 5.433 average nmse@all = 23.382

图片
czh-98 commented 1 year ago

What are the training sets for these models as listed on the web page ( https://www.realy3dface.com/ )?

We use the official pretrained model provided by these methods. You can refer to these papers for details about experimental settings.

czh-98 commented 1 year ago

I used the pre-trained model in deep3dface (https://github.com/sicxu/Deep3DFaceRecon_pytorch) to test (frontal-view), and the error is as follows, which is very different from the results given on the web page. What is the reason?

图片

average nmse@nose = 23.167 average nmse@mouth = 20.792 average nmse@forehead = 37.798 average nmse@cheek = 11.772 median nmse@nose = 23.201 median nmse@mouth = 20.776 median nmse@forehead = 37.569 median nmse@cheek = 14.989 standard deviation nmse@nose = 1.858 standard deviation nmse@mouth = 2.579 standard deviation nmse@forehead = 2.841 standard deviation nmse@cheek = 5.433 average nmse@all = 23.382

图片

Please check:

  1. whether you used a suitable template (i.e., Deep3D.obj) for evaluation.
  2. whether you have disabled the "process=False" for the "trimesh" package since the default setting to save ".obj" will change the topology.
  3. whether the predictions are aligned correctly.

For Deep3D, the save process (i.e., 2) might be the reason for such an issue.

wangjue-wzq commented 1 year ago

I realize that the unit of nmse is not mm, but the result given is mm. May I ask if there is an accurate conversion relation here?

czh-98 commented 1 year ago

I realize that the unit of nmse is not mm, but the result given is mm. May I ask if there is an accurate conversion relation here?

The scales are derived from the original scans in the HeadSpace dataset and our globally aligned scans (see data/metricial_scale.txt). The evaluation of this repo considers the scale automatically.

wangjue-wzq commented 1 year ago

Thanks!