search

subugoe / metar

Documentation and suggested best practices for data analysis at WAG
https://subugoe.github.io/metaR
MIT License
1 stars 0 forks source link

debug last-two use of yubikey 5c nfc on mobile #106

Closed maxheld83 closed 3 years ago

maxheld83 commented 3 years ago

This is (probably) strictly speaking not an issue with the Yubikey, but with Google's Advanced Protection Program (APP), for which YubiKeys are advertised. It is a potentially pretty grave bug and may lead to people (partially) locking themselves out of their accounts. As an average user, it's nearly impossible to get Googles in support or their bugtrackers, so I am hoping that you have a better angle here.

I've bought about USD 800 worth of YubiKeys in the last couple of days, so I hope this warrants some attention.

If nothing else, I think users should be warned about this.

Here are the steps to reproduce the bug:

  1. Sign up for Google Advanced Protection Program
  2. Use YubiKey "foo" and Yubikey "bar" as the required first two FIDO keys.
  3. Use them for an initial sign-in.
  4. Add more keys (this is now outside of the APP signup). Say, Yubikey "zap".
  5. (Sign out and re-require 2FA on all devices)

2FA sign in to google on a DESKTOP (see below): Works with any of the signed up keys, as expected. 2FA sign in to google from MOBILE (see below): Works only with the two keys originally used to set up APP. The other keys, listed indistinguishably on accounts.google.com WILL NOT WORK.

This behavior is nowhere documented. In fact, had I not accidentally noted down, which were the keys I had signed up initially, I would have never noticed the pattern. I was able to reproduce this several times and with other devices.

The last devices/OS in question where:

  • Desktop: Safari 14.0 macOS 10.15.6 ("Catalina"), MacBook Pro 16 (2019)
  • Mobile: Safari iOS 14.0, iPhone 8
  • The two first keys were:
  • YubiKey 5C NFC
  • Feitian Multipass K13 Key (Second-Gen) (Also tried it with only yubikeys).

Hope you can get someone at google to pay attention to this. It can get very, very nasty and, in any event, really undermines the confidence in FIDO keys.

maxheld83 commented 3 years ago

yubikey support could not reproduce the behavior, only apparent difference was that they tested only 3 keys total and were using a windows machine for signup.

maxheld83 commented 3 years ago

I was also surprised that setting up APP would revoke all old keys -- thanks for reporting that! But that was not the biggest problem.

I just ran through steps 4ff from your testing procedure again on both Safari 14.0 and Chrome 85.0, both on macOS 10.15.6, and I still see a problem, though the behavior seems to have changed yet again. Now on my iPhone (steps 5ff exactly as you did them, same iPhone, same OS), I can only ever successfully sign in with whichever two keys I last added to the account. There's a small possibility that this was the problem all along, and that I got my records confused, but I'm pretty sure initially only the first ever two keys (from your Step 3) were ever accepted on iPhone 8 iOS 14.0 Gmail.

This whole thing is super weird and expensive to reproduce.

There now appear to be three possibilities: Your and my google account behave differently, because we're in different releases of google server-side software (possible?). The problem arises only on a mac desktop (since you were on Windows) (that'd be super weird). The problem was created when I switched to APP through some intermittent bug or bad release and can now no longer be reproduced. I currently toggle on/off my APP for security reasons, but will get a chance to do so safely again in the next two days, and will then report back. Am I missing something?

I know this must sound like a "user error" type of situation, but I think I've documented and reproduced this quite thoroughly on several devices (though only the same kind of hardware and versions).

maxheld83 commented 3 years ago

I've now had a chance to further reproduce the bug: it persists.

It appears that, under the below conditions, only about whichever 2 (sometimes 1-3, not entirely deterministic!?) Yubikey 5C NFC were added last to a Google Account as security keys can be used to sign in on iOS.

I've reproduced this, using: 4 separate google accounts, 2 with APP 2 without (affects all!) enabled/disabled APP again (problem persists) mobile device is always iPhone 8 iOS 14.0.1, sign in inside Gmail. enrolled (and tested) keys using Safari 14.0 macOS 10.15.6 ("Catalina"), MacBook Pro 16 (2019) and Chrome 85 (does not matter) The problem does not affect: Other services with Fido2/Webauthn, such as: Dropbox, Wordpress.com, Discourse -- they all work fine on mobile with (more than 3 Yubikey 5C NFC keys) Yubikey 5Ci keys appear to add to the count (they replace Yubikey 5C NFCs, if added latter), but they do not themselves ever get replaced. They always work on mobile (though obviously, through a different interface). Same with the Feitian Multi-Interface. The problem cannot be reproduced with only Feitian and Yubikey 5Cis. So I would hypothesize that this bug sits at the intersection of the Google Fido2/Webauthn implementation and the Yubikey 5C NFC, in particular. Take either element away (other services or other keys) and the problem disappears.

My current (one of many) configuration with the problems is attached. You'll see that I was able to sign in on mobile only with the two 5C NFC keys added last.

I've been able to reproduce this many times now, with several accounts. APP doesn't appear to matter.

Could you perhaps try to reproduce this on your end using: 7 keys or so, so we have a similar number of total keys (I think you tried with only 3) A mac to sign up the keys (though it would be really baffling if that is the cause). I find this bug very unsettling, since the behavior is completely undocumented/unexpected and may contribute to a partial lock-out. I've now been investing wayy too much time into this, and I'd appreciate if we could get to the definitive bottom of this.

maxheld83 commented 3 years ago

received a reply from yubikey; they're still not able to reproduce, now using mac for signup and 5 keys.

maxheld83 commented 3 years ago

It's quite frustrating that this behavior can't be reproduced.

I just confirmed it again -- the problem still exists for me as per the earlier email.

Answers to your questions are below.

There's one last difference between our setups I can think of: You seemed to have used max of 2 of the new YubiKey 5C NFC in your testing. I have used 5 of the new YubiKey 5C NFC on one account (plus 4 others).

As I mentioned in my previous email, I think this problem might be related to the 5C NFC keys. I myself have only been able to reproduce the issue with >3 YubiKey 5C NFC keys.

It appears that, under the below conditions, only about whichever 2 (sometimes 1-3, not entirely deterministic!?) Yubikey 5C NFC were added last to a Google Account as security keys can be used to sign in on iOS.

Could you try adding more of the 5C NFC keys, and then see whether the 5C NFC keys added previously still work? (Ideally with 5C NFC keys with subsequent serial numbers -- this is what I used).

maxheld83 commented 3 years ago

thanks for digging further into this. I really appreciate it.

I guess this simply cannot be reproduced then.

I'll leave the issue open in our own issue tracker.

I will get a chance to check this with a bunch of colleagues in a couple of weeks and will respond if that still reproduces the problem.

maxheld83 commented 3 years ago

Here's another attempt at reprexing this:

Keys in order of signing up and order of testing sign in:

  1. YubiKey 5C NFC "black": FAILURE
  2. YubiKey 5C NFC "light blue": FAILURE
  3. YubiKey 5C NFC "purple": FAILURE
  4. YubiKey 5Ci "golden": SUCCESS
  5. Feitian Bluetooth/USB Security Key: NA
  6. Plug-Up Security Key: NA
  7. YubiKey 5C NFC "green": SUCCESS

So, problem reproduces as above.

IMG_1905 IMG_1906 IMG_1907 IMG_1908 IMG_1909 IMG_1910 IMG_1911

maxheld83 commented 3 years ago

and another one:

Keys in order of signing up and order of testing sign in:

  1. YubiKey 5C NFC "light blue": FAILURE
  2. YubiKey 5C NFC "black": FAILURE
  3. YubiKey 5Ci "golden": SUCCESS
  4. Plug-Up Security Key: NA
  5. Feitian Bluetooth/USB Security Key: NA
  6. YubiKey 5C NFC "purple": SUCCESS
  7. YubiKey 5C NFC "green": SUCCESS

again, same problem

maxheld83 commented 3 years ago

uh I've now tested this across several accounts with a new batch of keys, and it appears that this only affects the old batch. Both report the same Firmware, so it's not clear that they are actually different.

I reproduced the bug:

  1. on a newly APP-ed account
  2. on an old APP account

Always only the old keys were affected.

So this does not appear to be a set-up-time problem, but a hardware version problem.

Alternatively, though this is harder to test, the problem might be caused by the old batch of keys having a bunch of other settings/services enabled, though I wouldn't know how/why that might be relevant.

I've reached out to YubiKey support again.

maxheld83 commented 3 years ago

It would be nice to get this resolved for the old keys, though it's probably not super important for setting this up anew.

maxheld83 commented 3 years ago

the bug that won't stop giving.

Even with a new batch of keys, the bug still occurs. It appears that it just won't work with several google accounts on one key.

Steps to reproduce: Set up >3 keys (say, blue, yellow, green, black) for account foo@gmail.com (via USB-C on Safari 14.02 macOS 11.1, in my case, but I don't think it matters) Set up >3 keys (say, blue, yellow, green, black) for account bar@gmail.com (via USB-C on Safari 14.02 macOS 11.1, in my case, but I don't think it matters) Now login in to google.com in all combinations via NFC (Safari on iOS 14.3 iPhone 8) Expected behavior: Login via NFC is possible with all >3 keys. Login via USB works with all keys (it does!). Observed behavior: Login via NFC is possible with only a subset of the keys (~2). Theory: When the google or the browser asks the yubikey for the key exchange via NFC, it somehow forgets / it specifies the host (ie. google.com), but it forgets the account (ie. foo or bar). So, the yubikey returns some secret for google.com, but not necessarily the correct one for the account in question. Sometimes it matches, sometimes it doesn't. Conditions & Caveats: This only applies to the YubiKey 5C NFCs, and only to the NFC interface. USB-C works all the time. This only appears to affect Google.com (maybe even only APP?). I was not able to reproduce this behavior with dropbox or the yubikey playground.

maxheld83 commented 3 years ago

yubico seems to think that this bug is the same as https://bugs.webkit.org/show_bug.cgi?id=220415, but I'm a bit skeptical.

maxheld83 commented 3 years ago

this ultimately couldn't be resolved 😤 so there will remain some limitations (i.e. FIDO won't work for more than 2 keys on mobile for Google APP.

ttwwhh commented 1 year ago

I know this is an old thread, but I have probably the same issue. Wondering if anyone has any further insights.

Added the keys to my Google account on my MacBook Pro (High Sierra) no problem. All works fine USB.

On my iPhone SE 2020, only the last key added works (ie. authenticates) in NFC mode. Mind you, NFC works fine and I get pop ups and Yubico's NFC app sees them. If I delete a non-working using NFC key and add it back, now that one works and the other three don't work with NFC. All the while, they all work USB on the Mac.

I've been working with Yubico and they say they cannot reproduce the issue.

eduardstal commented 1 year ago

@ttwwhh I am experiencing precisely the same issue. Autehtication using Safari, while logging into any google service (mobile browser pop-up), will only work with the 3rd key (my case) or basically the last key that was added to the google advanced protection key management.

Pretty sure this issue is on google's side of things. Yubico couldn't possibly fix this afaik.

Weird