w3c / ai-web-impact

An analysis of the systemic impact, on the Web, of AI systems, and in particular ones based on Machine Learning models, and the role that Web standardization may play in managing that impact
https://www.w3.org/reports/ai-web-impact/
21 stars 3 forks source link

Effects on security measures #22

Open iherman opened 5 months ago

iherman commented 5 months ago

The section on security concentrates on the danger of impersonations. However, there may be additional dangers regarding the security measure taken by Websites for secure access. An obvious endangered approach is the use of captchas: an AI system may be in position to answer those captchas question, thereby making it, essentially, useless. Systems relying on biometric data might also be endangered (simple face or voice recognitions, for example).

Although the details of those techniques have not been developed at W3C, they also affect access to many Websites, so it is very much in the purview of this study.

cc @simoneonofri

simoneonofri commented 5 months ago

Hi @iherman,

That's a very good point. From my analysis, AI raises the bar from both a defender's and an attacker's point of view. It is a powerful tool that complicates the fateful cat-and-mouse game.

As you say, to defend sites from anti-automation, there are CAPTCHAs. CAPTCHAs can be resolved by AIs relatively easily, and then the defender perhaps puts other countermeasures in place to protect himself. Also, there is an ongoing discussion in several communities regarding CAPTCHAs' usefulness (and the use, for example, of JavaScript challenges that do not worsen the user's UX).

Conversely, defenders can use WAFs that use AIs to protect themselves from attacks and not use regexp or visitation behaviors to figure out whether it is a bot (which may also have privacy effects).

Is a particularly interesting and broad topic.

iherman commented 5 months ago

CAPTCHAs can be resolved by AIs relatively easily

which is all the more ironic because, as far as I know, CAPTCHAS, as far as I know, were originally used (maybe still are) to train AI systems :-)

simoneonofri commented 5 months ago

CAPTCHAs were used to avoid programs, and now they are still used to train programs (AI) using humans for free :)

Reminds me a little bit of:

It was said that you would destroy the Sith, not join them! Bring balance to the Force, not leave it in darkness!