pcr-upm / opal23_headpose

Headpose estimation using OPAL (2023)
MIT License
44 stars 4 forks source link

Testing the 300W-LP model using Biwi data set #2

Closed bobetocalo closed 1 year ago

bobetocalo commented 1 year ago

13219 frames protocol (MTCNN)

-m test -p alignment -a /home/database/alignment/faces/headpose/biwi-kinect/biwi_ann_mtcnn.txt --alignment opal23_headpose --database 300wlp --gpu 0 --rotation-mode euler --target-dist 1.0

MAE: 4.128371429841625
 > [yaw, pitch roll]: [4.54266856 5.04610023 2.7963455 ]
GE: 7.485612039705865
Confusion matrix:
[[   0    0    0    0    0    0    0    0    0    0    0    0    0]
 [   6   90    6    1    0    0    0    0    1    0    0    0    0]
 [   0  229  475   92    0    0    0    0    0    0    0    0    0]
 [   0    0  264 1025   25    0    0    0    0    1    0    0    0]
 [   0    0    0  324 1222   72    0    1    2    0    0    0    0]
 [   0    0    0    0  145 1767  440    2    1    0    0    0    0]
 [   0    0    0    0    1   97 2873  463    0    0    0    0    0]
 [   0    0    0    0    1    0    4  705  489    0    0    0    0]
 [   0    0    0    0    0    3    0    0  546  634   12    0    0]
 [   0    0    0    0    0    2    1    0    0  392  436   23    0]
 [   0    0    0    0    0    0    0    0    0    0  123  179    0]
 [   0    0    0    0    0    0    0    0    0    0    0   43    1]
 [   0    0    0    0    0    0    0    0    0    0    0    0    0]]
Accuracy (accurate class): 70.058%
Accuracy (adjacent class): 99.607%

Aligning the coordinates system of 300W-LP and Biwi

MAE: 3.158045331545884
[yaw, pitch roll]: [3.98229502 3.09118803 2.40065294]
GE: 5.421985821420112
Confusion matrix:
[[   0    0    0    0    0    0    0    0    0    0    0    0    0]
 [  18   84    0    1    0    0    0    0    1    0    0    0    0]
 [   1  285  504    6    0    0    0    0    0    0    0    0    0]
 [   0    0  427  885    2    0    0    0    0    1    0    0    0]
 [   0    0    0  491 1115   12    0    2    1    0    0    0    0]
 [   0    0    0    0  313 1946   93    2    1    0    0    0    0]
 [   0    0    0    0    1  268 2902  263    0    0    0    0    0]
 [   0    0    0    0    1    0   12  858  328    0    0    0    0]
 [   0    0    0    0    0    3    0    2  727  461    2    0    0]
 [   0    0    0    0    0    2    1    0    0  532  315    4    0]
 [   0    0    0    0    0    0    0    0    0    0  174  128    0]
 [   0    0    0    0    0    0    0    0    0    0    0   43    1]
 [   0    0    0    0    0    0    0    0    0    0    0    0    0]]
Accuracy (accurate class): 73.909%
Accuracy (adjacent class): 99.818%

-m test -p alignment -a /home/database/alignment/faces/headpose/biwi-kinect/biwi_ann_mtcnn.txt --alignment opal23_headpose --database 300wlp --gpu 0 --rotation-mode 6d --target-dist 1.0

MAE: 3.982902935212144
 > [yaw, pitch roll]: [4.58377271 4.64960589 2.7153302 ]
GE: 7.298686612056729
Confusion matrix:
[[   0    0    0    0    0    0    0    0    0    0    0    0    0]
 [   1   98    4    0    0    0    0    0    0    0    1    0    0]
 [   0  240  482   74    0    0    0    0    0    0    0    0    0]
 [   0    3  301  975   35    0    0    0    0    0    0    1    0]
 [   0    1    0  305 1225   87    0    0    1    1    1    0    0]
 [   0    0    0    0  133 1738  482    0    2    0    0    0    0]
 [   0    2    0    0    3  107 2833  489    0    0    0    0    0]
 [   0    0    0    1    0    0    3  689  506    0    0    0    0]
 [   0    0    1    2    0    0    0    0  559  624    9    0    0]
 [   0    1    1    1    0    0    0    0    0  422  424    5    0]
 [   0    0    0    0    0    0    0    0    0    0  134  168    0]
 [   0    0    0    0    0    0    0    0    0    0    0   35    9]
 [   0    0    0    0    0    0    0    0    0    0    0    0    0]]
Accuracy (accurate class): 69.521%
Accuracy (adjacent class): 99.720%

Aligning the coordinates system of 300W-LP and Biwi

MAE: 3.2106035019075443
[yaw, pitch roll]: [3.97802068 3.23859747 2.41519236]
GE: 5.481217108539611
Confusion matrix:
[[   0    0    0    0    0    0    0    0    0    0    0    0    0]
 [  12   90    1    0    0    0    0    0    0    0    1    0    0]
 [   0  297  499    0    0    0    0    0    0    0    0    0    0]
 [   0    2  476  834    2    0    0    0    0    0    1    0    0]
 [   0    1    0  461 1135   21    0    0    1    1    1    0    0]
 [   0    0    0    1  296 1935  121    0    2    0    0    0    0]
 [   0    2    0    0    3  231 2920  278    0    0    0    0    0]
 [   0    0    0    1    0    0    9  844  345    0    0    0    0]
 [   0    0    1    2    0    0    0    2  741  447    2    0    0]
 [   0    2    0    1    0    0    0    0    0  590  259    2    0]
 [   0    0    0    0    0    0    0    0    0    0  173  129    0]
 [   0    0    0    0    0    0    0    0    0    0    0   36    8]
 [   0    0    0    0    0    0    0    0    0    0    0    0    0]]
Accuracy (accurate class): 74.113%
Accuracy (adjacent class): 99.796%

15677 frames protocol (Biwi depth)

-m test -p alignment -a /home/database/alignment/faces/headpose/biwi-kinect/biwi_ann_test.txt --alignment opal23_headpose --database 300wlp --gpu 0 --rotation-mode euler --target-dist 1.0

MAE: 6.447405759246767
 > [yaw, pitch roll]: [ 4.47072111 10.66922019  4.20227598]
GE: 12.60062220956212
Confusion matrix:
[[   0    0    0    0    0    0    0    0    0    0    0    0    0]
 [   0   67   94    0    0    0    0    0    0    0    0    0    0]
 [   0   37  728  337    1    0    0    0    0    0    0    0    0]
 [   0    0   77 1099  364    8    0    0    0    0    0    0    0]
 [   0    0    0   50 1426  434   56    0    0    0    0    0    0]
 [   0    0    0    0   89 1979  545   14    3    0    0    0    0]
 [   0    0    0    1    9  335 3284  512   76   15    0    0    0]
 [   0    0    0    0    0    2  110  956  311   18    0    0    0]
 [   0    0    0    0    0    0   11  119  949  302    1    0    0]
 [   0    0    0    0    0    0    0    0   28  716  146    0    0]
 [   0    0    0    0    0    0    0    0    0   19  282   19    0]
 [   0    0    0    0    0    0    0    0    0   23   18    7    0]
 [   0    0    0    0    0    0    0    0    0    0    0    0    0]]
Accuracy (accurate class): 73.311%
Accuracy (adjacent class): 98.482%

Aligning the coordinates system of 300W-LP and Biwi

MAE: 3.943650444892901
[yaw, pitch roll]: [3.3972432  4.74894498 3.68476315]
GE: 7.006388915416922
Confusion matrix:
[[   0    0    0    0    0    0    0    0    0    0    0    0    0]
 [   0  113   48    0    0    0    0    0    0    0    0    0    0]
 [   0   86  905  111    1    0    0    0    0    0    0    0    0]
 [   0    0  154 1257  133    4    0    0    0    0    0    0    0]
 [   0    0    0  201 1503  252   10    0    0    0    0    0    0]
 [   0    0    0    0  210 2279  134    6    1    0    0    0    0]
 [   0    0    0    1   10  521 3311  317   67    5    0    0    0]
 [   0    0    0    0    1    5  189 1088   99   15    0    0    0]
 [   0    0    0    0    0    0   18  188 1059  116    1    0    0]
 [   0    0    0    0    0    0    0    1  122  719   48    0    0]
 [   0    0    0    0    0    0    0    0    0   27  282   11    0]
 [   0    0    0    0    0    0    0    0    0   23   25    0    0]
 [   0    0    0    0    0    0    0    0    0    0    0    0    0]]
Accuracy (accurate class): 79.837%
Accuracy (adjacent class): 98.922%

-m test -p alignment -a /home/database/alignment/faces/headpose/biwi-kinect/biwi_ann_test.txt --alignment opal23_headpose --database 300wlp --gpu 0 --rotation-mode 6d --target-dist 1.0

MAE: 6.304375144193904
 > [yaw, pitch roll]: [ 4.5951342  10.30364028  4.01435095]
GE: 12.338446971295719
The PostScript backend does not support transparency; partially transparent artists will be rendered opaque.
Confusion matrix:
[[   0    0    0    0    0    0    0    0    0    0    0    0    0]
 [   0   89   72    0    0    0    0    0    0    0    0    0    0]
 [   0  111  661  331    0    0    0    0    0    0    0    0    0]
 [   0    1  108 1093  345    1    0    0    0    0    0    0    0]
 [   0    0    0   82 1511  373    0    0    0    0    0    0    0]
 [   0    0    1    2   99 2068  448    8    1    2    1    0    0]
 [   0    5   11   12   29  362 3205  481   84   23    7   12    1]
 [   0    1    1    1    1    4  108  942  316   17    6    0    0]
 [   0    0    0    0    0    0    3   97  945  332    5    0    0]
 [   0    0    0    0    0    0    0    0   32  715  143    0    0]
 [   0    0    0    0    0    0    0    0    0   11  253   49    7]
 [   0    0    0    0    0    0    0    0    0    7   27   14    0]
 [   0    0    0    0    0    0    0    0    0    0    0    0    0]]
Accuracy (accurate class): 73.330%
Accuracy (adjacent class): 98.380%

Aligning the coordinates system of 300W-LP and Biwi

MAE: 4.045650051577039
[yaw, pitch roll]: [3.67301102 4.96327577 3.50066336]
GE: 7.1751158152310595
Confusion matrix:
[[   0    0    0    0    0    0    0    0    0    0    0    0    0]
 [   0  128   33    0    0    0    0    0    0    0    0    0    0]
 [   0  146  812  145    0    0    0    0    0    0    0    0    0]
 [   0    0  156 1258  134    0    0    0    0    0    0    0    0]
 [   0    0    0  180 1632  154    0    0    0    0    0    0    0]
 [   0    0    3    0  206 2316   95    7    0    2    1    0    0]
 [   1    5   11   14   25  503 3260  299   74   19    9   10    2]
 [   0    1    1    2    1    6  160 1058  151   12    5    0    0]
 [   0    0    0    0    0    0   15  138 1054  172    3    0    0]
 [   0    0    0    0    0    0    0    0   89  732   69    0    0]
 [   0    0    0    0    0    0    0    0    0   17  251   39   13]
 [   0    0    0    0    0    0    0    0    0    7   28   13    0]
 [   0    0    0    0    0    0    0    0    0    0    0    0    0]]
Accuracy (accurate class): 79.824%
Accuracy (adjacent class): 98.412%