ZhangYuanhan-AI / CelebA-Spoof

[ECCV2020] A Large-Scale Face Anti-Spoofing Dataset
531 stars 92 forks source link

APCER/BPCER definitions in paper #41

Closed AIshutin closed 3 years ago

AIshutin commented 3 years ago

Hi!

I've read appendix of the paper and experiments part of OULU-NPU, which is referenced in paper, however, in my understanding definitions of APCER in CelebA-Spoof and OULU-NPU are different (and that seems strange). Can you please clarify what exactly do you mean by APCER and BPCER?

In OULU-NPU:

Screenshot from 2020-12-22 13-54-41

i. e. this maximum error across different attacks.

In CelebA-Spoof:

Screenshot from 2020-12-22 13-56-53

Screenshot from 2020-12-22 13-57-51

which seems to be simply FAR and FRR.

Thanks.

ZhangYuanhan-AI commented 3 years ago

Hi: Yes, it is different from the definition of Oulu-NPU, because we follow the calculation fashion of APCER and BPCER from this repo, this paper is accepted by CVPR.

AIshutin commented 3 years ago

Thanks for help.

AIshutin commented 3 years ago

You wrote in the paper: Screenshot from 2020-12-23 10-09-57

(ACER is calculated at EER)

On the other hand, in CDCN: result ACER is minimum of ACER(thr_i).

Just to finally confirm, should I follow the code in CDCN`s implementation, not the CelebA-Spoof paper?

AIshutin commented 3 years ago

Sorry, I still don't understand( . Can you please explain, what's the difference between FAR and APCER, when calculated at EER threshold?

AIshutin commented 3 years ago

Then, BPCER = FRR_i (which FRR_i == FAR_i) -> BPCER = APCER, no?

(From the paper I suppose not, but I can't find the mistake.)

ZhangYuanhan-AI commented 3 years ago

Thanks for your remind, we realize we do totally follow the calculation fashion from this repo.

AIshutin commented 3 years ago

Again, big thanks for help.