** Unable to decrypt: The sender's device has not sent us the keys for this message. **

waclaw66 commented 3 years ago

Steps to reproduce

There are some historical messages sent from my devices.
Element Web shows just Unable to decrypt: The sender's device has not sent us the keys for this message.
Newly created Element Android session can decrypt them properly.

Outcome

What did you expect?

Element Web should be able to decrypt historical messages when newly created Element Android session can do it.

What happened instead?

Element Web cannot decrypt some historical messages for some reason. Clicking to Re-request encryption keys from your other sessions works only if another Element Android session is online, another Element Web sessions are not able to help (obviously because suffer with the sam bug). Seems Element Web cannot retrieve message keys directly from homeserver, but re-requesting them from devices works, that's strange. Nevertheless those keys has to be present on homeserver because newly created Element Android sessions can retrieve them.

Operating system

Windows 10

Application version

Element: 1.9.4 Olm: 3.2.3

How did you install the app?

No response

Homeserver

own

Will you send logs?

Yes

Palid commented 3 years ago

@waclaw66 try cross-verifying sessions that have those keys between themselves and request keys again. We're currently not properly informing users about the need of cross-verification of sessions properly, and this seems like one of those issues.

waclaw66 commented 3 years ago

@waclaw66 try cross-verifying sessions that have those keys between themselves and request keys again. We're currently not properly informing users about the need of cross-verification of sessions properly, and this seems like one of those issues.

All those sessions are cross-verified.

waclaw66 commented 3 years ago

I have tried to filter /_matrix/client/r0/sendToDevice/m.room_key_request/ requests from journal. It turned out that these requests come from Element Web only as a fallback retrieval when console error message Error decrypting event (id=$fIy7...): DecryptionError[msg: The sender's device has not sent us the keys for this message., session: FaX....] is shown. On the other hand a new session of Element Android can retrive all message keys without that sendToDevice requests at first bite. Element Web doesn't, there have to be some Element Android devices online to retrive those message keys from them.

waclaw66 commented 3 years ago

Now tested on app.element.io with matrix.org account. The same problem...

obrazek

Account sessions cross-signed, key backup properly set up, Elelement Android no problem.

obrazek

waclaw66 commented 3 years ago

I have discovered that the only way how to avoid those ** Unable to decrypt... messages is not to cross-sign with another device, but to use recovery passphrase instead. All message are immediately show as decrypted then.

tepozoa commented 3 years ago

This happens to me regularly using both Firefox and Chrome on two different laptops, with the Android client being the "not broken" device used for fixing it every time it happens; but it's inconsistent on when it happens. I notice that the session_id changes which is possibly a trigger.

app.element.io is running in a browser tab, friend sends message A (working)
log out of the webapp, close browser which clears cache, reboot
log into the webapp, supply Security Key in 2nd login dialog (I store it in my password manager)
the friend sent a message while I was rebooting and it now cannot be decrypted; friend did not use a different device or do anything on their side except type a new message and hit send
go through the cross-signing emjoi verification between webapp + Android, it technically works but message still can't be decrypted
shift+refresh browser tab, webapp reloads but states latest message can't be decrypted
click the link to try cross-signing keys again and it then magically decrypts without actually cross-signing

The last two steps 6-7 happen sporadically (I'd say < 20% of the time), but steps 1-5 happen quite often (at least 50% of my webapp logins if not more towards 70%). But sometimes, the exact same process logging back in "just works" and decrypts all new messages received during the webapp being logged out.

I manually used the View Source option of the two messages which just happened this morning and had me digging for this bug report, and verify that the friend's device_id and sender_key ARE THE SAME, which means that it did/does in fact have that key and can in fact decrypt the old message received before the reboot if it used the key, but a new message sent from their same device with the same key fails after the reboot because... something is broken in the code somewhere.

What is different in the source - besides the ciphertext and date/time stamp type things is the session_id value:

message received before logging out

  "content": {
    "algorithm": "m.megolm.v1.aes-sha2",
    "device_id": "AAAAAAAAAA",
    "sender_key": "BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB",
    "session_id": "CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC"
  },

message received after logging out, reboot, log in, provide Security Key

  "content": {
    "algorithm": "m.megolm.v1.aes-sha2",
    "device_id": "AAAAAAAAAA",
    "sender_key": "BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB",
    "session_id": "DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD"
  },

The message about not having the key to decrypt may be erroneous, as I have the key, know the key and it's in there (and could see all pre-reboot messages using that key); it's the new message with the same key and a different session_id which is failing to decrypt properly.

ldexterldesign commented 2 years ago

Would someone be kind enough to point me at documentation about how Element encryption works because I'm lost. Utterly lost. I just logged in and all my messages are inaccessible due to encryption??

I get why encryption is important but this whole thing has absolutely ruined UX for me

Hope to hear back

Cheers

mark-todd commented 2 years ago

The above comment is slightly unrelated, but nonetheless here are the docs: https://element.io/enterprise/end-to-end-encryption

mark-todd commented 2 years ago

I'm also receiving this error - out of interest, what backend are people using? I'm wondering if this may not be an issue in element, but in the matrix-org server we're using. This is just because we recently started using dendrite instead of synapse - is anyone else with the issue using dendrite?

ldexterldesign commented 2 years ago

@mark-todd 👋

Thanks for reply

Specifically, it seems it's the cross-signed device verification (CSDV) I had/have an issue with but I just checked today and my messages are now accessible so I'm unsure what changed..?

It seems to me there's a delay (and lack of progress indication) after submitting the "retrieve keys from other sessions" prompt, which negatively affects UX

Hope this helps

Regards

jittygitty commented 2 years ago

@mark-todd I'm using Dendrite but after reading almost a dozen similar issues, I doubt only dendrite is affected by these type of errors. But this is a very EMBARRASING error!

It is really embarrassing for the project when you invite people to use this app and then immediately there's a problem with just basic text chat, you can't read their message at all.

I can't really think of anything that should be a higher priority than fixing such basic functionality.

I simply invited someone to a p2p chat room (they can only be 2-person right?) and next day they accepted the invitation but I can't see any of their messages, just "Unable to decrypt: The sender's device has not sent us the keys for this message." If I long press on the message in android client I get popup where i can re-request encryption keys from your other sessions but it tells me to login to something that can decrypt but I can't this is a p2p between me and the other person and though I logged in to Element-desktop with same user, the Element-Desktop had SAME error of not able to Decrypt, when I click to re-request keys etc it doesn't seem to be able to get any.

mark-todd commented 2 years ago

It seems to me there's a delay (and lack of progress indication) after submitting the "retrieve keys from other sessions" prompt, which negatively affects UX

@ldexterldesign Yeah this happens in some cases - in others the messages never arrive. The delay is not so much of an issue though, you just wait a bit and it's ok. It's ones where chat never becomes usable that it's a serious issue.

@mark-todd I'm using Dendrite but after reading almost a dozen similar issues, I doubt only dendrite is affected by these type of errors. But this is a very EMBARRASING error!

It is really embarrassing for the project when you invite people to use this app and then immediately there's a problem with just basic text chat, you can't read their message at all.

I can't really think of anything that should be a higher priority than fixing such basic functionality.

I simply invited someone to a p2p chat room (they can only be 2-person right?) and next day they accepted the invitation but I can't see any of their messages, just "Unable to decrypt: The sender's device has not sent us the keys for this message." If I long press on the message in android client I get popup where i can re-request encryption keys from your other sessions but it tells me to login to something that can decrypt but I can't this is a p2p between me and the other person and though I logged in to Element-desktop with same user, the Element-Desktop had SAME error of not able to Decrypt, when I click to re-request keys etc it doesn't seem to be able to get any.

@jittygitty Yeah I completely agree - this is exactly the same error that I was receiving. In the end I just disabled encryption and made new rooms, but it's not ideal! It's particularly not ideal because encryption is on by default in one-to-one rooms, and can only be disabled by making a new room. This is a major and crippling issue with element and I think it should be addressed as a priority like you say. At the very least I think encryption should be off by default until this is fixed.

mark-todd commented 2 years ago

I think this error may also be a duplicate:

https://github.com/vector-im/element-web/issues/16413

jittygitty commented 2 years ago

Since matrix chat isn't truly p2p, and uses a server, "and" we even have "notifications", why can't each chat channel have a sort of "back-channel" for certain pre-configured requests like:

Your chat participant is requesting that you re-initialize this room and recreate with encryption. Do you agree?
Your chat participant is requesting that you re-initialize this room, without encryption. Do you agree?
(BEST option if possible?) Participant is requesting your decryption keys, do you agree to resend? But if this is possible it seems that we shouldn't be having this problem in the first place? Since "re-request keys" was supposed to do this automatically without prompting the other user?

The problem with creating new rooms is that it's messy and at least on dendrite we didn't even have a good way to really delete rooms. The best is to really make room creation "ATOMIC" (ie like a DB operation, if any room setup step glitches, room doesn't get created) and with lots of debugging info spit out on the server and even back to clients, currently we don't know if this is a Client problem or a server problem or a problem with both clients+server.

Interestingly I found that when I click on re-request keys link under the Unable to decrypt error in the Direct-message 2person channel/room, (from Element Desktop app), if I do CTRL-SHIFT-I and view console for errors, i don't really see any errors. I see messages that it did get device signing keys for that other user and even cross-signing keys also. What was interesting is that it did NOT request keys just for that channel, it actually triggered key request for ALL CHATS that I was on. This seems inefficient if true. But can't we get some better error/debugging info since obviously although the console said it got the keys, either it did and the client isn't using them properly, or it didn't actually get them or something. Anyone else have any debugging info etc?

jittygitty commented 2 years ago

@mark-todd It would suck to disable encryption, I mean one of the biggest points of this chat is the encryption, especially since there is a "server" involved in-between, encryption is important.

But yea for those of us using our own server that we control, it would be beneficial like you said: if we could disable/undo encryption for a room that has the Unable to Decrypt errors etc.

For others that rely on matrix.org servers, asking them to not use encryption for private direct-messages might not be ideal.

Zocker1999NET commented 2 years ago

I have the same issue and expect cross-signing for Element Web / Desktop is broken somehow, because:

restoring with security key works flawlessly. All messages can decrypted without problems, from bigger rooms and from direct messages as well. To ensure that no other of my clients are involved with helping to decrypt, I turned off my phone and closed the Element Web tab as well.
with using cross-signing while logging in, decrypting bigger E2EE rooms works flawless but direct messages cannot be decrypted, even while the Android app was running. I needed to nearly click on each message.
I was able to cross-sign the Desktop / Web sessions multiple times. Today while testing the Desktop client, I cross-signed it twice from my Android phone, once initiated by the Desktop client on login and the other time because the phone warned about an unknown login.

It could improve the UX at least for now if there was a "re-request keys for all messages" button or something similar. Or, after one re-request was done successfully, that Element would re-request the others automatically from new to old until one fails again. So the user wouldn't need to click this button more than once …

richvdh commented 2 years ago

Duplicate of element-hq/element-web#2996 ?

t3chguy commented 2 years ago

Looks like it

nituladaboneko commented 2 years ago

gave up goodbye element looked promising at the outset turned out to be another ubuntu

acuteaura commented 1 year ago

I'm running into this issue running running a fresh install of Synapse v1.81.0, trying to join a public room with encryption on and message history set to everyone. All my sessions are cross-signed. I really don't know what to do, this feels fragile, like it's going to break on me and I wouldn't know how to fix it. All I can think of is that the room is version 9 and all the ones I create are 10.

This occurs on pretty much all clients on all platforms.

Wehzie commented 11 months ago

I'm affected by this issue on multiple devices: iOS, Ubuntu, Web. Currently, only affects one chat, the other user can read my messages by I can't read theirs.

richvdh commented 11 months ago

This issue has been superceded by https://github.com/element-hq/element-meta/issues/245.

There's no need to tell us every time you see a decryption error. We know, and we're working on the causes.

element-hq / element-web