search

Xianhua-He / cvpr2024-face-anti-spoofing-challenge

Accepted by CVPR Workshop 2024
25 stars 2 forks source link

Model Testing on Protocol Dataset Produces Opposite Results #2

Closed tuin33 closed 2 months ago

tuin33 commented 2 months ago

Hi! We tested your model on the Protocol part of the dataset that provided by the challenge. However, results outputed by the model is opposite with the label files. Therefore, we would like to confirm that weather the model is applied in the currect dataset. We also noticed that while we take the min value index in outputs instead of the max value, results seem like to be normal. The confuse matrix we get is shown below. Thanks for your time! The confuse matrix of taking the max value: QQ截图20240426112231 The confuse matrix of taking the min value: QQ截图20240426112103

Xianhua-He commented 2 months ago

@tuin33 We are using the dataset provided by the competition, Label: live=0, fake=1. The label may be opposite to your dataset, but the image from the dataset should be consistent. For more detailed instructions, please refer to: https://codalab.lisn.upsaclay.fr/competitions/17490#learn_the_details-evaluation.

Xianhua-He commented 1 month ago

@tuin33 Hello, I wonder if your problem has been solved? I just learned from the competition organizer that the labels provided by the dataset for the competition are reversed. I am glad that you are interested in my work. If it is helpful to your research, you are welcome to cite my work. If you have any other details you would like to know, please feel free to raise an issue or add me on WeChat：13381144493

tuin33 commented 1 month ago

@tuin33 Hello, I wonder if your problem has been solved? I just learned from the competition organizer that the labels provided by the dataset for the competition are reversed. I am glad that you are interested in my work. If it is helpful to your research, you are welcome to cite my work. If you have any other details you would like to know, please feel free to raise an issue or add me on WeChat：13381144493

Yes!!! It's very glad to receive your concern about this issue. We have changed the labels in the dataset as you said, and it became normal.