search

biometricITC / cPP-biometrics

Contains the development of a Collaborative Protection Profile for biometrics
MIT License
10 stars 2 forks source link

Supporting Document v1.1 Section 7.3 AVA_VAN.1 #376

Closed xahun closed 2 years ago

xahun commented 2 years ago

Supporting Document v1.1 Section 7.3 AVA_VAN.1 - ATE_IND tests using the toolbox are essentially AVA_VAN in nature since they are addressing a potential flaw in design rather than an objectively determinable and testable function, so it is unclear why the toolbox is seemingly required twice. Selecting artefacts that show higher IAPAR seems irrelevant given that ATE_IND.1 testing with the toolbox seems to require comprehensive testing. Unless AVA_VAN.1 testing is required even if the optional FIA_MBE_EXT.3 and FIA_MBV_EXT.3 requirements are excluded, I suggest removing the AVA_VAN.1 material and instead for AVA_VAN.1 refer to the ATE_INT.1 tests that use the toolbox since it includes both functional and penetration type test cases. It really seems like its just more of the same type testing and this should all be combined into one run through of the toolbox.

n-kai commented 2 years ago

SD follows a guidance (Fingerprint Spoof Detection Evaluation Guidance (FSDEG)) developed by BSI (at early stage of cPP development, we had a German editor and followed his advice) to develop evaluation activities for ATE_IND and AVA_VAN.

cPP and SD will be reviewed by CCRA and I want to avoid conflict with BSI.

FSDEG states as follows.

It is the clear focus of any testing activity conducted in the course of the ATE classes to determine whether the spoof detection functionality is able to detect spoofed biometric characteristics with a sufficient reliability. To do so the developer as well as the evaluator refer to a standard toolbox [Toolbox] that is maintained by the certification body. ..... While the testing activities in context of ATE allow to develop an overall statement about the performance of the TOE with respect to its spoof detection mechanism they cannot provide any assurance about the spoof detection functionality with respect to variations of fakes. It falls into the focus of the vulnerability assessment to evaluate whether the use of additional fakes that have not been part of the toolbox or variations of fakes from the toolbox can lead to increased error rates.

During the vulnerability analysis the evaluator will use all their knowledge that they gained during the evaluation of the other assurance classes for penetration testing. It is the aim of this testing activity to determine whether variations of fakes can lead to a deterioration of the security relevant error rate.

Only if the tests carried out for ATE show that the TOE is able to recognize the fakes from the toolbox with the required reliability and the penetration tests showed that also dedicated variations of fakes will not compromise the spoof detection functionality the TOE shall pass the evaluation.

woodbe commented 2 years ago

Closed based on discussion that this could need to be reviewed during the initial evaluation if the lines between ATE_IND and AVA_VAN are not completely clear in terms of what should be presented in case other possible attack vectors cannot be determined.