search

yihuacheng / GazeTR

The codes and models in 'Gaze Estimation using Transformer, ICPR2022'.
109 stars 20 forks source link

Reproducibility problem on MPIIFaceGaze #1

Closed vikrant7 closed 2 years ago

vikrant7 commented 3 years ago

Hi @yihuacheng, I trained your pre-trained model on MPIIFaceGaze. I haven't made any changes in the script for training as well as pre-processing of dataset. I performed the leave-one-person-out evaluation on this dataset as mentioned in your paper. I am using PyTorch 1.7.0. I got the following best angular errors for respective person:

Person Best error
0 2.37
1 4.36
2 4.41
3 4.49
4 3.05
5 3.79
6 3.07
7 4.34
8 4.44
9 4.15
10 5.89
11 5.42
12 4.09
13 3.71
14 6.23
Mean 4.254

The mean of this best angular errors comes out to be 4.254, which is far away from the reported 4.00 error. Please let me know if I am missing something over here. Also, help me to reproduce the reported results.

yihuacheng commented 3 years ago

I show my result in the following. The result shows 4.25 at begining while decreases to 4.0 at the end. You could check your code again.

The performance of each epoch. Epoch Error
10 4.26
20 4.02
30 4.23
40 4.11
50 4.16
60 4.06
70 4.00
80 4.06
The performance of each person in 20 epoch. Person Error
0 2.57
1 4.51
2 4.35
3 2.77
4 2.59
5 3.63
6 3.67
7 4.19
8 4.13
9 3.54
10 5.49
11 5.77
12 4.24
13 3.38
14 5.51
vikrant7 commented 3 years ago

@yihuacheng Thanks for sharing this. I was able to reproduce results using your ETH-XGaze pretrained model. But I am not able to reproduce results when I am training from scratch on ETH-XGaze. Please share the ETH-XGaze training parameters.

Enable72 commented 2 months ago

@yihuacheng can you provide the GazeTR-pure pretrained model which has been mentioned in your paper?