search

tornadomeet / mxnet-face

Using mxnet for face-related algorithm.
Apache License 2.0
544 stars 206 forks source link

the result.txt of running get_predict_file is different from your predict.txt #23

Closed xzqjack closed 7 years ago

xzqjack commented 8 years ago

Hi, i have run verification/test.sh again with your provided data 'aligin-lfw dataset'. The new predicted result is different your verification/predict.txt and its acc is very bad.

Abel_Pacheco/Abel_Pacheco_0001.png  Abel_Pacheco/Abel_Pacheco_0004.png  0.451991    1
Akhmed_Zakayev/Akhmed_Zakayev_0001.png  Akhmed_Zakayev/Akhmed_Zakayev_0003.png  0.835496    1
Akhmed_Zakayev/Akhmed_Zakayev_0002.png  Akhmed_Zakayev/Akhmed_Zakayev_0003.png  0.714418    1
Amber_Tamblyn/Amber_Tamblyn_0001.png    Amber_Tamblyn/Amber_Tamblyn_0002.png    0.694672    1
Anders_Fogh_Rasmussen/Anders_Fogh_Rasmussen_0001.png    Anders_Fogh_Rasmussen/Anders_Fogh_Rasmussen_0003.png    0.618321    1
Anders_Fogh_Rasmussen/Anders_Fogh_Rasmussen_0001.png    Anders_Fogh_Rasmussen/Anders_Fogh_Rasmussen_0004.png    0.610199    1
Angela_Bassett/Angela_Bassett_0001.png  Angela_Bassett/Angela_Bassett_0005.png  0.826952    1
Angela_Bassett/Angela_Bassett_0002.png  Angela_Bassett/Angela_Bassett_0005.png  0.550906    1
Angela_Bassett/Angela_Bassett_0003.png  Angela_Bassett/Angela_Bassett_0004.png  0.45209 1
Ann_Veneman/Ann_Veneman_0003.png    Ann_Veneman/Ann_Veneman_0005.png    0.877394    1
Ann_Veneman/Ann_Veneman_0006.png    Ann_Veneman/Ann_Veneman_0010.png    0.758687    1
Ann_Veneman/Ann_Veneman_0010.png    Ann_Veneman/Ann_Veneman_0011.png    0.58771 1
Anthony_Fauci/Anthony_Fauci_0001.png    Anthony_Fauci/Anthony_Fauci_0002.png    0.788583    1
Antony_Leung/Antony_Leung_0001.png  Antony_Leung/Antony_Leung_0002.png  0.669878    1
Antony_Leung/Antony_Leung_0002.png  Antony_Leung/Antony_Leung_0003.png  0.609032    1
Anwar_Ibrahim/Anwar_Ibrahim_0001.png    Anwar_Ibrahim/Anwar_Ibrahim_0002.png    0.533982    1
Abel_Pacheco/Abel_Pacheco_0001.png  Abel_Pacheco/Abel_Pacheco_0004.png  0.999479    1
Akhmed_Zakayev/Akhmed_Zakayev_0001.png  Akhmed_Zakayev/Akhmed_Zakayev_0003.png  0.999885    1
Akhmed_Zakayev/Akhmed_Zakayev_0002.png  Akhmed_Zakayev/Akhmed_Zakayev_0003.png  0.999717    1
Amber_Tamblyn/Amber_Tamblyn_0001.png    Amber_Tamblyn/Amber_Tamblyn_0002.png    0.999251    1
Anders_Fogh_Rasmussen/Anders_Fogh_Rasmussen_0001.png    Anders_Fogh_Rasmussen/Anders_Fogh_Rasmussen_0003.png    0.999579    1
Anders_Fogh_Rasmussen/Anders_Fogh_Rasmussen_0001.png    Anders_Fogh_Rasmussen/Anders_Fogh_Rasmussen_0004.png    0.999673    1
Angela_Bassett/Angela_Bassett_0001.png  Angela_Bassett/Angela_Bassett_0005.png  0.999726    1
Angela_Bassett/Angela_Bassett_0002.png  Angela_Bassett/Angela_Bassett_0005.png  0.999633    1
Angela_Bassett/Angela_Bassett_0003.png  Angela_Bassett/Angela_Bassett_0004.png  0.999088    1
Ann_Veneman/Ann_Veneman_0003.png    Ann_Veneman/Ann_Veneman_0005.png    0.999508    1
Ann_Veneman/Ann_Veneman_0006.png    Ann_Veneman/Ann_Veneman_0010.png    0.999353    1
Ann_Veneman/Ann_Veneman_0010.png    Ann_Veneman/Ann_Veneman_0011.png    0.999434    1
Anthony_Fauci/Anthony_Fauci_0001.png    Anthony_Fauci/Anthony_Fauci_0002.png    0.99928 1
Antony_Leung/Antony_Leung_0001.png  Antony_Leung/Antony_Leung_0002.png  0.999453    1
Antony_Leung/Antony_Leung_0002.png  Antony_Leung/Antony_Leung_0003.png  0.99954 1
Anwar_Ibrahim/Anwar_Ibrahim_0001.png    Anwar_Ibrahim/Anwar_Ibrahim_0002.png    0.999303    1

The only change is that

  1. i use the newest mxnet, not your provided face_mxnet;
  2. i use io.imread instead of 'cv2.imread'

Is there any import notice i ignore? Any hits will be appreciated. Thank you.

tornadomeet commented 8 years ago

do you using the pre-trainned model or trainned by yourself? another, the newes mxnet changed some default behavior, such as the pooling shape, so it may bring some big difference. i'm also not very sure whether io.imread is the same as cv2.imread or not.

xzqjack commented 8 years ago

@tornadomeet
Thanks for your reply.

I have just made a test and found that the image read by cv2.imread is different from the image read by skimage.io.imread about pixel values. The difference is about 20, whcih is so large that would have huge influence （maybe bad influence） on training and testing results. That means the way of reading image is a big problems in Python. Is there union choice? For example, all researchers use cv2.imread or PIL or some other ways.

By the way, i changed to use cv2.imread and get the same results as your predict.txt

tornadomeet commented 8 years ago

@xzqjack en, i think different image library(read) should get the same result if used in the right way. so we need to check why it comes the difference, :)

xzqjack commented 8 years ago

@tornadomeet The reason is that the order of color channel is different. In cv2.imread and skimage.io.imread, one of them is rbg and another is bgr. The difference is worth paying attention to.

About converting color image to gray image, cv2 is different from skimage.io too. There is still huge difference on some pixel values, which still affect the training results and test results(maybe some kind of adversary noise).