Receiver can't decrypt messages, missing identity and fingerprint key

[TheTimeWalker]

Description

One of our coworker booted up her Riot Desktop and is greeted by a big list of "Could not decrypt messages" from every sender that she could receive. Curiously, all messages are the same that they do not contain any identity or fingerprint key.

Example with one of my messages.

Steps to reproduce are unknown, the only thing that can be said for reproduction is that she didn't open the Desktop client for a week. She never logged out so the device is still in the account's device list.

Android Riot also fails to decrypt messages. They however do have a fingerprint and identity key. It even shows from what device they appear but it still silently fails in the background.

Logs being sent: yes

Version information

macOS Mojave, Riot Desktop 1.4.1 Android 7.0, Riot.im 0.9.6

Bounty

I have added a 20$ bounty for anybody that can fix this issue: https://www.bountysource.com/issues/81389159-receiver-can-t-decrypt-messages-missing-identity-and-fingerprint-key

[TheTimeWalker]

Okay so I have checked out what happens in the logs and I found this part very interesting:

What it seems like is that Riot marks the device list as obsolete and probably removes the device list while the key retrieval and decryption is still working. This probably breaks the decryption as it can't put the keys to the specific devices it needs to and fails. This could also explain why it feels like this happens randomly and less reproducible than it seems.

Especially noteworthy: all the messages of these three people mentioned in the console couldn't be decrypted which includes myself too from my mobile phone.

[TheTimeWalker]

I have added a 20$ bounty for anybody that can fix this issue: https://www.bountysource.com/issues/81389159-receiver-can-t-decrypt-messages-missing-identity-and-fingerprint-key

[imtbl]

I can confirm this (or a very similar) behavior sometimes happening to both messages I sent myself (from a different device) and messages sent by a friend.

Even though all those devices (both mine and my friend's) are verified with each other, I sometimes get the following notification when accessing the room and trying to read the messages on a different device than the one the message was sent from:

** Unable to decrypt: The sender's device has not sent us the keys for this message. **

Looking at the end-to-end encryption information of such messages, it says unknown device under Sender device information and identity and fingerprint keys are also sometimes missing.

The only thing I am quite certain of at this point is that it never occurs on a device if Riot is open while messages are coming in. It only happens if Riot is closed at the time the messages are written and it pulls them once it's launched and the room is accessed.

This bug seems to be independent of the platform, since I have experienced this on macOS, iOS, iPadOS, Windows and Arch Linux so far. It also happened when using accounts on both matrix.org and a self-hosted homeserver (using the latest version of Synapse).

[TheTimeWalker]

This issue is still hitting us all very often, myself included. This happens on both 1:1 chats and group chats. It seems to be a Riot Desktop only bug and almost always happens when trying to fetch and decrypt messages when for example the laptop was offline for a while - generally can be 30min. With this it's not feasible to have encryption by default for people that exclusively use Riot Desktop and none of the other clients 😞

[ara4n]

i’m actively hunting these bugs currently in the run-up to enabling e2ee by default. please submit bug reports from the sending & receiving device immediately after an undecryptable message and i will dig into it.

[TheTimeWalker]

Others and I did send a few logs for this in hopes that this could solve it. The difficult part is that the time distance between sender and receiver is sometimes too big and the failed decryption happens at seemingly random.

As said prior, this pretty much happens at any time when the receiver wasn't online for a while, either by having the laptop closed or not connected to the internet. And this also can't be done by quickly disconnecting for a minute, sending and then connecting again. You need to be offline for at least 30min.

From my POV after looking into it a while, it feels like there's some discrepancy between when Riot invalidates the device list and when it's in the process of decrypting. I did try to dabble into the SDK code but couldn't find a solution to it myself yet either. And from my personal standpoint, sadly after suggesting this client in our work space, I'm getting lash back as decryption doesn't work properly for people that exclusively use Riot Desktop only where E2E was the main selling point. (Note it's only group chats of maybe 4-5 people) :frowning_face:

[TheTimeWalker]

Since this has been a while and cross signing was implemented, let me just write into here that this issue still persists with the latest Riot Web/Desktop 1.6.2

As usual, this only happens if the Riot Web/Desktop instance was offline for a while and reconnects. This doesn't happen if it's connected to the server. Weirdly enough, it occasionally fails to decrypt my messages even though my device list has been the same since cross signing dropped to release

Just noticed now, you can see those red shields on the left side. It seems that anytime it can't decrypt those messages then it is marked as "Encrypted by a deleted session"

[TheTimeWalker]

Riot Web 1.6.8 Same symptoms as usual. The new thing that I noticed is this:

Sometimes it seems like the messages get decrypted but their authenticity can not be guaranteed which is very strange. Looking at the source of the message and comparing it with the device list of the sent message, the device id are matching. That means it fails to compare the source device id with the person's device list. This also only happened after waking up from standby (Riot reconnects) so my assumption is that it's connected to this issue.

FWIW, this is what I see a lot in my logs:

[uhoreg]

The "The authenticity of this encrypted message can't be guaranteed on this device" message means that it got the key from key backup or from key sharing, and is due to the way the message signatures are done.

[TheTimeWalker]

Well, it doesn't seem like that. I've had it happen again and the shield is consistently staying even for newly received messages right immediately. Even after restarting element-desktop

[rgpublic]

I also got this after logging out and in again on the desktop. All my historic messages now have a gray warning shield with "the authenticity of this encrypted message can't be guaranteed on this device" warning. My own messages even have a red shield with "Encrypted by a deleted session". New messages seem to work okay again.

I don't understand. How can this happen? Why can't the authenticity be guaranteed? On my phone (where I didn't log out and in), I don't have the warning shields. Doesn't that mean that Element on my phone can in fact still guarantee the authenticity? And my desktop trusts my phone, right? Is there any way to solve this or should I just wait until all the red/gray warning shields have scrolled out into oblivion by new incoming messages?

[kazetsukaimiko]

Hey guys, for what it is worth I ran into this issue when a friend switched from using a browser (we'll call this client A) to using the electron app (we'll call this client B). His electron app/ client B would show "grey sheids" for all messages even though all sessions were verified (it also appears we lost the ability to forcefully re-verify sessions or revoke trust in an update?). What seemed to change the condition- if I restarted my client (client C), client B would no longer see the problem.

Perhaps this has something to do with where the needed encryption keys are sourced from when decrypting messages? Would a key from key backup produce a "grey shield" where the same key obtained by request from a trusted client produce a trusted message result?

[rgpublic]

For my part, I'm still wondering why the gray shield exists anyway. What exactly does it tell the users that's really worth informing them about? I can only guess: Is it perhaps that the message as you see it right now on the screen cannot be trusted to really have been sent like that in the past and might be fake/manipulated history? So that would mean: If the user didn't see the actual message back then when it didn't have the gray shield yet and still remembers that someone actually wrote such a message, it might be that this message has never been sent. Is that true? At least, that would make sense to me to warn the user about such a case... But I'm still totally unsure if it works like that or whether the gray shield has a totally different meaning. Without any further explanation I still find this highly confusing. People think sth. is broken or unsecure or sth. like that if all their messages suddenly get that icon.

[uhoreg]

The issue for the grey shield is https://github.com/vector-im/element-web/issues/14323, which includes an explanation about what it means.