FAR minimum value

[woodbe]

At the moment we are looking at setting a 1:10K minimum allowed FAR value for any validated biometric. A question though is whether we should specify a higher number or not. Based on how things would look right now, at an 80% confidence interval, we would be looking at minimum subject numbers along the lines of those specified here in the FIDO Alliance biometric spec:

https://fidoalliance.org/specs/biometric/requirements/#Ruleof3FAR

For the 1:10K, a test would need roughly 245 participants (or 123 if it is something like an eye or finger where it is allowed to collect 2 samples from one person). As you can see in the chart, going to 1:25K would mean 390 subjects (or 195 using 2 samples per person).

So the question is whether we should look at raising to a higher number or leave it at 1:10K. Note that we can always make a TD or full revision that will change this later, so we are not locked into this value forever. So for anyone who would be evaluating, what is the expectation for doing this and whether we should raise the number or not?

[gfiumara]

One thing to consider might be the cost to the vendors once the FAR is raised. Using the FIDO numbers for the two FAR above, the number of subjects increases by 145. Assuming subjects are recruited for the test with a $100 stipend, the test will now cost a minimum of $14,500 more just for subjects, let alone nearly doubling the amount of time for informed consent, data collection, etc.

Similarly, do we know throughput of testing labs, as to their ability to handle this influx of subjects for multiple vendors?

I am in favor of more stringent accuracy requirements and more subjects, but I also want to be cognizant of pricing smaller manufacturers out of evaluation and/or making the evaluation infeasible for evaluators. Are these things we need to consider?

[n-kai]

I propose 1:10K with rule of 3 for FAR/FMR (FAR/FMR SHALL meet the requirement of less than 1:10K for the upper bound of a 95% confidence interval) because the following reasons.

Minimum number of participants Number should be 245 so that SMEs (Small and medium-sized enterprises) whose number of employees is a few hundreds can conduct the performance testing using their employees and their families without additional cost.

Method for estimation We should follow the rule of 3 (95% confidence) because it is described in ISO/IEC 19795-1 (and we don't need to explain by ourselves why we take this value because it's described in internationally recognized standard). Some organizations (e.g. *1) ) require the value should be estimated with at least 95% confidence so if we take 80 % confidence, such estimation can not meet this requirement and CC certificates can not be used as an evidence that the TOE meet the requirement.

*1) Additional requirements for biometrics by DEA https://www.deadiversion.usdoj.gov/21cfr/cfr/1311/subpart_c100.htm#116

Minimum allowed FAR value 1:10K can meet both the FIDO and NIST SP 800-63-B. Only those vendors who need to meet 1:50K that is defined in Android 10 Compatibility Definition should specify such lower value. So, for face recognition product, vendors need at least 245 participants to achieve 1:10K FAR (and 123 participants for fingerprint, eye and vein if it is allowed to collect 2 samples from one person)

[woodbe]

@n-kai I'm not sure we need to worry about the DEA requirements as that would seem to be for more than the current mobile devices support (the idea that you are specifically you logging into the prescription app isn't something that is known on the mobile device since we only know that a template has been made, but if you had 2 people add templates to share a device, you have no way of knowing which one logged in).

The main concern about the confidence interval is in terms of what the actual sensor can handle. I will agree that everyone is largely targeting the 1:50K today, whether they test live to that or not, so maybe it isn't an issue about the confidence interval being 95% (when the initial discussions happened in FIDO it was also nearly 4 years ago, which I find hard to believe at this point), so maybe that would not be the same choice if started from scratch today (while they are mainly fixed on that for now given that is how they started).

This does seem to be agreement though on the 1:10K as the minimum threshold though.

[The-Fiona]

There are definitely practical considerations, and especially (currently) if COVID controls about gathering folks together in one place are applicable.

The ability for all (including SMEs) to provide the results with a reasonable amount of resource is a very important consideration.

As technology improves, and better FAR/FRR rates are achieved I am wondering about the practicality of relying only on the rule of 3 to demonstrate this. Are there other methods, other than the rule of 3, that could be included in the PP to support the claims?
For example, could we consider a hybrid technique? Rule of 3 up to 10K and supporting technical evidence for other claims??? I feel sure that FIDO and others are having this same practical issue. I'd like to discuss this possibility with the experts.

[woodbe]

from @iyliaR

Regarding the fixing a minimum value to 1:10K. I am voting for this. Below are some of my thoughts.

1. Latest Android requirement are fixing the FAR value at 1:50K and I believe iOS have done earlier . However, the current standards (mentioned by N-Kai) still recommend at 1:10K. I would prefer it to be standardize with the standards as it is easier to reference in testing.

2. I assume this will be for use case 1 accessing the mobile phone (correct me if I am misunderstood), therefore it will be more on the user convenience. Maybe for use case 2 , where the user would need higher security to access apps (banks, government service, etc.), 1:50K would be suitable.

3. As for 2 people add templates to share a device, this would be a common situation in during this pandemic where a lot of kids are sharing devices with their parents and siblings. I think multifactor authentication would help in this.

[woodbe]

So it would seem overall that we are in agreement about 1:10K being the minimum, and it would be up to the vendor to go higher if they want, but that is optional.