search

reshow / PRNet-PyTorch

The training and evaluation code for PRNet (《Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network》)
73 stars 16 forks source link

NME calculation #13

Open HOMGH opened 4 years ago

HOMGH commented 4 years ago

Hi @reshow . Thank you for sharing the pytoch version of PRNet. I had a question regarding calculating NME. using PRNet we would have 43k vertices in our predicted 3D mesh. but for AFLW2000 the number of vertices is 53k (or 39k for reduced version). So how can we calculate NME when we have different number of vertices? Any idea about that? Thanks :)

reshow commented 4 years ago

The 43k (or 64k if the foreface mask is not used) vertices is transformed from 53k vertices (as in processor.py, runPosemap). I think an inverse transformation of that may help to obtain the 53k vertices.

Or you can transform the 53k vetices of 3DDFA into 64k vertices of PRNet and then calculate NME.

Shiyu555 commented 3 years ago

hi，I still don't know how to calculate bbox when calculating 3D NME，d=sqrt( (max_x-min_x)*(max_y-min_y)) is right? but I find that in your github code d=np.sqrt((maxx - minx) 2 + (maxy - miny) 2 + (maxz - minz) ** 2) Hoping for your reply,thanks!

reshow commented 3 years ago

hi，I still don't know how to calculate bbox when calculating 3D NME，d=sqrt( (max_x-min_x)*(max_y-min_y)) is right? but I find that in your github code d=np.sqrt((maxx - minx) 2 + (maxy - miny) 2 + (maxz - minz) ** 2) Hoping for your reply,thanks!

It should be d=np.sqrt((maxx - minx) 2 + (maxy - miny) 2). I give 2 ways to calculate NME since it is not always the same in different works. You can refer to the repo of 3DDFA https://github.com/cleardusk/3DDFA/blob/master/benchmark/benchmarkCrop.m , line 30-43 and https://github.com/XgTu/2DASL/blob/master/evaluation/nme_for_alignment_68pts_2d.m , line 46.

reshow commented 3 years ago

hi，I still don't know how to calculate bbox when calculating 3D NME，d=sqrt( (max_x-min_x)*(max_y-min_y)) is right? but I find that in your github code d=np.sqrt((maxx - minx) 2 + (maxy - miny) 2 + (maxz - minz) ** 2) Hoping for your reply,thanks!

I checked my codes again, and found that "d=np.sqrt((maxx - minx) 2 + (maxy - miny) 2 + (maxz - minz) ** 2) " is not used anywhere. I'm sorry to have confused you. But I can't update this repo in the near future, just forget it.

Shiyu555 commented 3 years ago

hi，I still don't know how to calculate bbox when calculating 3D NME，d=sqrt( (max_x-min_x)*(max_y-min_y)) is right? but I find that in your github code d=np.sqrt((maxx - minx) 2 + (maxy - miny) 2 + (maxz - minz) ** 2) Hoping for your reply,thanks!

I checked my codes again, and found that "d=np.sqrt((maxx - minx) 2 + (maxy - miny) 2 + (maxz - minz) ** 2) " is not used anywhere. I'm sorry to have confused you. But I can't update this repo in the near future, just forget it.

Thank you for your prompt reply! I'm reading PRNET code recently，and I'm confused how the author's NME results are calculated. In loss.py,line 39,40"pred[i, 2] = pred[i, 2] - torch.mean(pred[i, 2])""gt[i, 2] = gt[i, 2] - torch.mean(gt[i, 2])".Why subtract the mean?I didn't find the corresponding code in 2DASL

reshow commented 3 years ago

hi，I still don't know how to calculate bbox when calculating 3D NME，d=sqrt( (max_x-min_x)*(max_y-min_y)) is right? but I find that in your github code d=np.sqrt((maxx - minx) 2 + (maxy - miny) 2 + (maxz - minz) ** 2) Hoping for your reply,thanks!

I checked my codes again, and found that "d=np.sqrt((maxx - minx) 2 + (maxy - miny) 2 + (maxz - minz) ** 2) " is not used anywhere. I'm sorry to have confused you. But I can't update this repo in the near future, just forget it.

Thank you for your prompt reply! I'm reading PRNET code recently，and I'm confused how the author's NME results are calculated. In loss.py,line 39,40"pred[i, 2] = pred[i, 2] - torch.mean(pred[i, 2])""gt[i, 2] = gt[i, 2] - torch.mean(gt[i, 2])".Why subtract the mean?I didn't find the corresponding code in 2DASL

2D to 3D reconstruction is a ill-posed problem. In PRNet, the 3D to 2D projection is assumed to be a weakly perspective projection. So the absolute value of the coordinates on the z-axis is meaningless, but the relative value is meaningful. So I subtract the mean.

Shiyu555 commented 3 years ago

hi，I still don't know how to calculate bbox when calculating 3D NME，d=sqrt( (max_x-min_x)*(max_y-min_y)) is right? but I find that in your github code d=np.sqrt((maxx - minx) 2 + (maxy - miny) 2 + (maxz - minz) ** 2) Hoping for your reply,thanks!

I checked my codes again, and found that "d=np.sqrt((maxx - minx) 2 + (maxy - miny) 2 + (maxz - minz) ** 2) " is not used anywhere. I'm sorry to have confused you. But I can't update this repo in the near future, just forget it.

Thank you for your prompt reply! I'm reading PRNET code recently，and I'm confused how the author's NME results are calculated. In loss.py,line 39,40"pred[i, 2] = pred[i, 2] - torch.mean(pred[i, 2])""gt[i, 2] = gt[i, 2] - torch.mean(gt[i, 2])".Why subtract the mean?I didn't find the corresponding code in 2DASL

2D to 3D reconstruction is a ill-posed problem. In PRNet, the 3D to 2D projection is assumed to be a weakly perspective projection. So the absolute value of the coordinates on the z-axis is meaningless, but the relative value is meaningful. So I subtract the mean.

oh，I see. Thanks for your help and patience!

reshow commented 3 years ago

hi，I still don't know how to calculate bbox when calculating 3D NME，d=sqrt( (max_x-min_x)*(max_y-min_y)) is right? but I find that in your github code d=np.sqrt((maxx - minx) 2 + (maxy - miny) 2 + (maxz - minz) ** 2) Hoping for your reply,thanks!

I checked my codes again, and found that "d=np.sqrt((maxx - minx) 2 + (maxy - miny) 2 + (maxz - minz) ** 2) " is not used anywhere. I'm sorry to have confused you. But I can't update this repo in the near future, just forget it.

Thank you for your prompt reply! I'm reading PRNET code recently，and I'm confused how the author's NME results are calculated. In loss.py,line 39,40"pred[i, 2] = pred[i, 2] - torch.mean(pred[i, 2])""gt[i, 2] = gt[i, 2] - torch.mean(gt[i, 2])".Why subtract the mean?I didn't find the corresponding code in 2DASL

2D to 3D reconstruction is a ill-posed problem. In PRNet, the 3D to 2D projection is assumed to be a weakly perspective projection. So the absolute value of the coordinates on the z-axis is meaningless, but the relative value is meaningful. So I subtract the mean.

oh，I see. Thanks for your help and patience!

: )

HOMGH commented 3 years ago

The 43k (or 64k if the foreface mask is not used) vertices is transformed from 53k vertices (as in processor.py, runPosemap). I think an inverse transformation of that may help to obtain the 53k vertices.

Or you can transform the 53k vetices of 3DDFA into 64k vertices of PRNet and then calculate NME.

Hi @reshow , I'm still struggling with NME calculation of PRNet on AFLW2000 dataset. The PRNet output has 43K vertices and AFLW2000 groundtruth has 53K. As you mentioned 43K vertices are transformed from 53k vertices. So, in order to have the same number of vertices, can we transform 53K vertices of AFLW2000 to 43K and calculate the NME? Is it correct?