Room history sometimes not visible even when historical room keys are shared.

[uhoreg]

When inviting a user to an E2EE room with history visibility set to allow the new user to view historical messages, we now share (some) decryption keys with the invited user. However, even after doing so, sometimes the new user will see a blank room when they join. They can view the messages after switching to a different room and then switching back.

This is due to https://github.com/matrix-org/matrix-react-sdk/pull/3881, which filters out UTDs from the timeline if they are from before the user joined the room. Thus, if the invited user's Element tries to display the timeline before it receives/processes the keys, it will not try to re-display the undecryptable messages.

I think that we can fix this by modifying https://github.com/matrix-org/matrix-react-sdk/pull/3881/files so that, instead of removing the undecryptable event tiles from the timeline, it just hides them or leaves them blank. When the keys come in, it re-tries the decryption (using the normal decryption re-try mechanism), and if the decryption is successful, then it should back-paginate a few events, and either hide these events and stop back-paginating (if undecryptable), or show them back-paginate more (if decryptable).

Note: I think that this exceeds my React knowledge, so hopefully someone else can pick it up.

---

You can't see earlier messages

Encrypted messages before this point are unavailable

[t3chguy]

So I cannot reproduce this, I synthesised it by creating a public encrypted room R, sending some messages from user A, joining R from user B, then issuing mxMatrixClientPeg.matrixClient.sendSharedHistoryKeys($R, [$B]) on user A and immediately user B sees the remaining history.

https://user-images.githubusercontent.com/2403652/136927292-c6992a89-1883-490b-87a5-d4355f9c8ff5.mov

There have been some changes to the TimelinePanel recently due to Threading, maybe they made this path happier accidentally? Has someone seen this in the wild on develop recently?

[djedirahu]

So I cannot reproduce this, I synthesised it by creating a public encrypted room R, sending some messages from user A, joining R from user B, then issuing mxMatrixClientPeg.matrixClient.sendSharedHistoryKeys($R, [$B]) on user A and immediately user B sees the remaining history.

Screen.Recording.2021-10-12.at.10.10.03.mov There have been some changes to the TimelinePanel recently due to Threading, maybe they made this path happier accidentally? Has someone seen this in the wild on develop recently?

Any reason why this function, or simply sharing keys on invite would break and no longer continue to function as expected @t3chguy ? I know this issue has been closed but I am experiencing just that on certain existing rooms and it's becoming quite a problem for our community in it's infant stages bringing new people into the space who are greeted with empty rooms. If I try to replicate it by creating a new space and populating it with e2ee rooms, it functions fine?

[t3chguy]

@djedirahu the share keys on invite feature requires the megolm sessions to be flagged as shareable, which only Element Web & Desktop do right now, so megolm sessions created by other clients won't be shareable yet.

[djedirahu]

@djedirahu the share keys on invite feature requires the megolm sessions to be flagged as shareable, which only Element Web & Desktop do right now, so megolm sessions created by other clients won't be shareable yet.

Any way to manually set that flag for an existing room?

[t3chguy]

The flag isn't on the room, it's on each megolm session, megolm sessions are created by each sending device periodically.

[djedirahu]

The flag isn't on the room, it's on each megolm session, megolm sessions are created by each sending device periodically.

Thanks for the clarification, excuse my gaps in understanding. In this case all of the invites have been sent via web or desktop (or utilising the dev console via the web app as above), to no avail. Nothing indicates that it shouldn't send the keys as intended on invite. Was wondering if there was anything else to investigate if it still fails to function. As I said creating new e2ee rooms seems to solve it, but for our existing rooms that are populated with content and members it seems to only function periodically or not at all.

[djedirahu]

Oh hang on so if the messages are sent via ios or android they won't appear, as the keys are only shared from sessions created with the flag enabled? The majority of our users are posting from ios or android - so will not appear. Thought they were tied exclusively to the invite but it actually has to do with each megolm session on a per message basis... Any idea if this will be changed for ios/android in the near future?

@djedirahu the share keys on invite feature requires the megolm sessions to be flagged as shareable, which only Element Web & Desktop do right now, so megolm sessions created by other clients won't be shareable yet.

[HarHarLinks]

I think I am hitting this. I'm invited into an e2ee room which has some history. Element web shows a blank timeline apart from my invite & join event. I can scroll up into a blank area.

At the same time, Element android does show the history, but messages are either "Unable to decrypt: OLM" or "Unable to decrypt: The sender's device has not sent us the keys for this message."

[t3chguy]

@HarHarLinks given that the history is not decryptable for you then you are not hitting this bug.

but messages are either "Unable to decrypt: OLM" or "Unable to decrypt: The sender's device has not sent us the keys for this message."

This bug is when your client has keys but hides the history. The inviting user/client didn't send you keys or the megolm sessions weren't marked as shareable.

[HarHarLinks]

I don't quite follow. My element desktop looks exactly like the left (light mode) element in your screencap, with scrollable empty space, before you issue that console command on the right (dark mode) element. My room is private invite only e2ee with history readability set to "members (since selecting this option)" from the start.

I am under the impression that history should be decryptable for me with that setting thanks to a somewhat recent update to element web, which I specifically tried at the time and did work (this is referenced in the OP).

They can view the messages after switching to a different room and then switching back.

This doesn't work.

Element web should be showing no history (no scrollable blank space), or similar to android, show old events but with decryption error. I'd favor the 2nd because otherwise you may be confused when somebody references the undecryptable messages.

By the issue title, and your screencap, it seems to me like this is that same bug. If not, I can open another issue, but no idea what sets it apart.

Element Nightly version: 2021112301 Olm version: 3.2.3 I can rageshake if that helps? Maybe it is even related to vector-im/element-desktop#829

[t3chguy]

I am under the impression that history should be decryptable for me with that setting thanks to a somewhat recent update to element web, which I specifically tried at the time and did work (this is referenced in the OP).

That update to EW only works if you are invited by Element Web and all of the megolm sessions in that room were created by an up-to-date Element Web/Desktop.

They can view the messages after switching to a different room and then switching back.

This doesn't work.

Because you don't have the keys. This issue is about if you do have the keys and you do need to switch rooms back-and-forth to see the history.

https://github.com/vector-im/element-meta/issues/647 is the issue you want.

[HarHarLinks]

Thanks for digging. Should I create a new issue to "Never show a (scrollable) empty timeline, but either nothing or verbose errors"? That would apply to both issues, but only the symptoms. There seems to be an <ol class="mx_RoomView_MessageList" with an element.style style="height: 1200px;" that seems to be my immediate issue.

[ara4n]

This happens pretty much every time I invite someone to an E2EE room, despite keysharing happening. Either the history pops into view when they send a message, or they have to bg and then fg the room to make it visible. It causes a massive WTF despite us having gone to the trouble of implementing keysharing.

(We might also want to doublecheck that the /invite command correctly keyshares too, as opposed to using the UI)

[lampholder]

Unless it's changed since I checked wtih @uhoreg /invite does not implement keyshares.

[t3chguy]

Confirmed, only the InviteDialog calls MatrixClient::sendSharedHistoryKeys

[turt2live]

(put as P1 in the web team board because a duplicate issue was P1 - I am assuming someone else will re-triage as needed)

Edit: except not because I closed the wrong issue

[novocaine]

I briefly saw this today in the Offsite 2022 room (which I'd joined yesterday), the message about not being able to display further history was displayed after the room very slowly loaded a large number of join events when I switched to the room.

I saw the tile "You can't see earlier messages" but I didn't take note of which subtitle it had.

Yesterday, I actually was able to see the history, so it was a bit of a surprise that the history had since disappeared! I then switched rooms and the history magically reappeared.

So I'm left wondering if this is not just a bug associated with initial joining, but there are other things going on as well, given I initially joined the room yesterday.

[MadLittleMods]

I've seen this many times in the main timeline.

But just had a reproduction in the thread panel for the first time. Normally, the thread panel is just blank until messages load.

[mguentner]

I also experience this. Funnily enough I can see / browse previous messages in element-desktop when I search for the content in the search. :smile:

[hazecodeio]

The issues has returned with the recent roll-out :( New members can't see the history unless they have been at least invited before any new messages in the room.

" Unable to decrypt: DecryptionError: This message was sent when we were not a member of the room. "

Element version: 1.11.69 Crypto version: Rust SDK 0.7.0 (068a0af), Vodozemac 0.6.0