w3c / captcha-accessibility

Inaccessibility of CAPTCHA
https://w3c.github.io/captcha-accessibility/
Other
3 stars 4 forks source link

Working Draft Feedback: More nuanced verdict of the effectiveness of CAPTCHA #21

Open hpetersFC opened 2 years ago

hpetersFC commented 2 years ago

WRT 1.3 CAPTCHA Effectiveness and 4. Conclusion

Both above sections state that CAPTCHA as a security solution are “becoming increasingly ineffective”, and are “ill suited” to the purpose of telling humans and robots apart.

I’d like to offer a different perspective, and say that - as always - it depends.

The efficacy of CAPTCHA varies dramatically depending on the chosen solution.

Traditional CAPTCHA solutions are well known to provide little defence against automated solving, that’s correct. On the other hand, solutions such as Arkose Labs * are highly effective against automated abuse, to the point of offering a warranty to their customers.

Whether a CAPTCHA can be cracked is not the best measure for the efficacy of the solution. They can all be cracked in theory.

Even the best CAPTCHAs today can theoretically be cracked, given enough time, money, and determination - including the highly effective Arkose Labs challenges. But it would take a combination of methods and efforts, requiring skills as well as a large amount of time and/or computing power i.e. money. The key is to make an attack too expensive to be commercially viable.

Effectiveness should be considered for the whole solution, not an individual CAPTCHA let alone an individual request.

An individual CAPTCHA variant might be automatable in a test lab situation, but in the context of the entire defence solution deployed in a real-life production environment, it ideally proves impossible to mount a viable automated attack. For example, and without going into too much detail, the CAPTCHA may dynamically employ different additional defence features in a layered approach; the CAPTCHA variant might change over time or immediately in response to current attack profiles; the CAPTCHA may let a request pass but flag it for later blocking downstream; etc.
These capabilities are very difficult to judge when using a simple demo of a given CAPTCHA to scrutinise its security. That's why large corporations usually test several solutions on their production traffic for several weeks to evaluate results.

Background

)* For transparency: I am PM at Arkose Labs, who offers solutions for fraud and abuse mitigation - an interactive CAPTCHA is part of our toolbox. Over the past couple of years, many large corporations such as Microsoft, Amazon and many more, have switched away from reCaptcha and other traditional CAPTCHAs to our solution because it has proven highly effective.

I’m not saying this to market our solution here - but rather support the validity of the claims I am making above. The notion that "CAPTCHAs don't work" is common, and something that we continuously work to refute. I'm happy to elaborate on anything as required.

Summary

I would appreciate a more nuanced wording on the effectiveness of CAPTCHA. Many traditional solutions are indeed ineffective, but this is not true for all solutions. CAPTCHAs can be a highly effective part of the abuse mitigation strategy if chosen carefully and considered in the larger context of longterm fraud and abuse mitigation.