element-hq / element-android

A Matrix collaboration client for Android.
https://element.io/
Apache License 2.0
3.28k stars 677 forks source link

Waiting for this message, this may take a while. #1721

Closed 532910 closed 4 months ago

532910 commented 4 years ago

Phone was updated from the latest stable riot-android to element via GPlay (no beta subscription)

Now all messages in all e2ee direct chats including new messages are: Waiting for this message, this may take a while.

After update the phone was verified via pitcture sequence with riot-web. All contacts are verified and has green point/shield on all devices, all contacts are on my own synapse instance (1.15.1-1 debian stable).

image

Cross-signing is enables, Keys are trusted, Private keys are not known: image

I tried to clear cache and to restart phone without sucess.

532910 commented 4 years ago

Trying to re-verify (Secure Backup -> Set up on this device) this phone gives:

via QR code both devices shows error: image

via image set the phone shows: image riot-web says:

Verify other session
Verification cancelled

You cancelled verification. Start verification again from the notification.
[Clear]
532910 commented 3 years ago

Restoring messages from backup decrypts all messages. Newly received messages after backup recovery are shown as Waiting for this message, this may take a while. Restoring messages from backup again decrypts all messages, even just received before this restore.

532910 commented 3 years ago

Is any update there? It's completely impossible to use Element after upgrade!

532910 commented 3 years ago

Right now, after restoring messages from backup it says:

Cross-Signing is enabled
Private Keys on device

But still shows new messages as Waiting for this message, this may take a while

jtrees commented 3 years ago

I know someone who also gets the Waiting for this message, this may take a while message a lot. It can take several hours for the messages to resolve but they eventually do.

532910 commented 3 years ago

Nothing has changed after more than a week of waiting, so I had to reinstall Element.

PureTryOut commented 3 years ago

My parents have the same issue when talking between them. They do resolve after 24 hours or more, but of course that makes it impossible to have a proper conversation. Please give this issue some priority, beside the vague text it's more than just an UX bug!

BillCarsonFr commented 3 years ago

Hello, can you please give more information on the element version of both devices?

Also could you check the security settings on both devices (is Encrypt to verified session only is ON?)

image

Do you have other sessions? The fact that key backup import is able to decrypt could mean that your other session are getting the keys and uploading them to backup. So it could be possible that your device is not getting the keys.

Go to settings > Advanced > and enable developper mode, this will allow you to see more about the error, if you go back to the room, what do you see for the errors?

Did you submit some rageshake (top righ button in home screen > report bug; add reference to issue matrix-org/matrix-spec-proposals#1607 ?

PureTryOut commented 3 years ago

Speaking for my parents here, as I don't have this issue myself.

They use the latest version from the Google Play store, so whatever version that is. They don't have "Encrypt to verified session only" on. They have other sessions, for both a tablet and a laptop (so webbrowser) at least.

I'll ask them to enable developer mode and submit a rageshake. Thanks for looking in to it!

532910 commented 3 years ago

element version

the latest available on GPlay

Encrypt to verified session only is OFF.

have other sessions?

Yes.

MrCustomizer commented 3 years ago

A few users on my server have reported this issue to me. Seems like this is still happening. Any news on workarounds/solutions?

aczerniak commented 3 years ago

It happened to me today after I cleared cache on Element installed on Android phone. I use Element 1.0.9[40100092]. In developer mode message says that sender's device didn't send keys. Before I cleared cache those messages were properly decrypted. I have 4 sessions (3x Firefox and one phone - the issue affected phone only), 2 of them are open and devices that were used sent affected messages are also on so at very least they should resent needed keys.

I don't have cross signing enabled. I verify all session manually.

thomashoullier commented 3 years ago

Here on matrix-synapse-py3 1.22.1 (server), a user on the android client (update 19 Oct 2020) reported the same problem. "Waiting for this message, this might take a while" for all the messages from a particular user starting at some date in e2e encrypted channels. The user with the bug can still receive messages from other users on the same channels.

lasselin commented 3 years ago

A user just reported me this issue on the matrix.org server.

akallabeth commented 3 years ago

Have the same issue from time to time with different users (and some small group chats). This did not happen with 1.0.8 and earlier (all from F-Droid) All accounts on matrix.org.

austinbutler commented 3 years ago

Getting this for the first time today. Didn't change anything and was working fine yesterday. Encrypt to verified sessions only is off.

Screenshot_20201120-204111

gtirloni commented 3 years ago

I'm getting this one a newly activated device after I did the cross-sign verification (Element Android 1.0.11 [40100112] (G-b847)).

smeyersdev commented 3 years ago

My wife also runs into this bug on her Element installation on Android (Play Store). There is a strange behavior: As long as she stays at home connected to Wifi network, the issue does not appear. But as soon as the smartphone is using mobile LTE internet, it happened several times that I received these messages from her which never got decrypted.

jrozanski commented 3 years ago

Indeed @smeyersdev, I observed the same behaviour sporadically on poor quality connections. It appears to be related to instability of the connection.

dalcde commented 3 years ago

This seems to depend on the session sending the message. In one of my conversations, I have interleaving messages from different devices (same user), and only the messages from one of the devices are stuck in "Waiting for this message, this may take a while".

dalcde commented 3 years ago

I'd imagine the issue starts with an attempt to retrieve a key failing. What is the retry strategy in this case? In other parts of the matrix ecosystem I've had issues with exponential back-offs leading to long delays when something is inaccessible.

michel-slm commented 3 years ago

I'm seeing this too, and the weird thing is:

Wodann commented 3 years ago

I'm consistently seeing this issue on my android phone (works fine in my desktop browser's session) for the last message sent by one user. The moment they send more messages, the message that previously displayed the error will become visible, but the new latest message will report the error instead.

I've tried clearing the cache, but to no avail.

jzacsh commented 3 years ago

regarding the theory about mobile connectivity being the root cause: while I am seeing a correlation with mobile (non-wifi) usage, I think the bigger problem is the undefined behavior from then on.

That is, the last couple times I hit this bug, I then had trouble working around it. For example we'd see these steps:

  1. hit the bug in 1:1 encrypted chat, while out on mobile connection
  2. both parties jump onto same (fast) wifi LAN
    1. both keep trying to send chats
    2. both showing phones in person (one person has all messages rendered, the other sees the bug on messages -2, -3, -4 (but 0, -1 sent over wifi are fine))
  3. give up on element for a day or so
  4. over a day later one person reports (via Signal) that some of the messages are suddenly showing

step (2) involves both parties having their Element app out, actively looking at the same 1:1 room in the foreground on Android, actively connected on a fast wifi connection. So:

agherzan commented 3 years ago

I have the same issue: iPhone client vs Android client. Only some of the messages from iPhone end up in the "waiting" state on the Android side. What I saw, is that sending more messages starts to refresh previous messages in this state. In other words, the last message is always in "waiting" and receiving more messages unlocks it while locking the next one. It also happens only with this iPhone user. I have chat rooms, with clients on exactly same platform and room configuration where this doesn't reproduce ever.

Alkaid-Benetnash commented 3 years ago

I have the same issue: android element A <- latest docker synapse -> android element B. A cannot see messages from B timely about 2weeks ago. Destroy and recreate 1-to-1 room solved the issue. Now B cannot see messages from A timely. Want to look for a workaround without destroying the existing room.

Any updates? Is this still bottlenecked by collecting bug reports?

Wodann commented 3 years ago

regarding the theory about mobile connectivity being the root cause

In my case I'm actually not using mobile data at all. Whenever I receive messages, it's on wifi.

zopieux commented 3 years ago

Can confirm I'm having https://github.com/vector-im/element-android/issues/1721#issuecomment-860220732 too, ie. Element Android reproducibly does not decrypt the latest message from one of the users in a room. As soon as a new message arrives the previous one exits the waiting for this message and becomes readable, while the latest is waiting. Other users not affected. I can provide more debug info if you need, but I need to know where to find it.

Edit: welp, it's not that simple. Sometimes I received the latest message of said user immediately, while older messages were still in the waiting state. While it seems to affect particular users, message order does not seem to be very relevant. I think it is a red herring correlated with time, since things eventually resolve by themselves, but over very long durations (hours/days) which makes this bug particularly annoying for an instant messaging software.

begincalendar commented 3 years ago

~I just signed out of the app, cleared the app's storage (all of it, not just the cache) for good measure and then signed back in. Seems like it might have worked around the issue.~

Signing out of the app, clearing the storage and reinstalling the app don't seem to help. Unfortunately they seem to have made it worse: more messages (than just the last one in a room) are now unreadable and more rooms are being affected.

newptcai commented 3 years ago

I logged out to resolve this issue and now almost all my messages are turned into "Waiting for this messages..."

crashbandicode commented 3 years ago

image

I tried to clear cache and to restart phone without sucess.

So I am also experiencing this issue on Android with a second session on iOS. I notice you have “setup secure backup” in your photo instead of “reset secure backup”. As a potential workaround on Android, try tapping the “setup secure backup” option and walking through the verify process. This seems to have solved the issue for me. Depending on the situation I may have to open another session to get the “this may take a while” to go away but I only have to do that once per user. I think the keys get messed up somehow. Hopefully this workaround works for someone.

bahur142 commented 3 years ago

I'm consistently seeing this issue on my android phone (works fine in my desktop browser's session) for the last message sent by one user. The moment they send more messages, the message that previously displayed the error will become visible, but the new latest message will report the error instead.

I've tried clearing the cache, but to no avail.

Exactly the same situation between me and a friend using self hosted services! Version: GPlay 1.1.12

dessalines commented 3 years ago

Just ran into this problem today. Newest version on android, matrix.org homeserver.

Anything I can do to fix?

crashbandicode commented 3 years ago

Another thing to try is installing v1.1.9 from F-droid. The one from Aurora Store had decryption issues but downgrading to that version specifically from F-droid seems to have fixed some of these cases.

kernitus commented 3 years ago

Same issue, I found if I enable developer mode the error message becomes Unable to decrypt: OLM

kat-co commented 3 years ago

Try issuing a /discardsession command on the client not experiencing the issue. That seems to have cleared things up for me.

I think the megolm session between devices becomes messed up and element doesn't detect the issue. This should create a new session between devices.

benmordecai commented 3 years ago

Someone I talk to is reporting this issue as well.

jtrees commented 3 years ago

Try issuing a /discardsession command on the client not experiencing the issue.

Unfortunately this isn't an option on iOS.

kernitus commented 3 years ago

Try issuing a /discardsession command on the client not experiencing the issue. That seems to have cleared things up for me.

I think the megolm session between devices becomes messed up and element doesn't detect the issue. This should create a new session between devices.

Actually this did solve the issue. Now I wonder why it's happening in first place :thinking:

finzzz commented 3 years ago

Another thing to try is installing v1.1.9 from F-droid. The one from Aurora Store had decryption issues but downgrading to that version specifically from F-droid seems to have fixed some of these cases.

This doesn't fix the issue.

uncivilgentleman commented 3 years ago

Happy bug anniversary everyone! Reported one year ago.

Seriously though this bug is crippling. An unreliable chat app is a hard sell to friends and family.

martijndeb commented 2 years ago

I have this same problem with a new device; Followed the steps above (including the discard session). To no avail. I'm running v1.1.12 from fdroid

mcg-matrix commented 2 years ago

An unreliable chat app is a hard sell to friends and family.

I guess a lot of us "unpaid admins and consultants" will agree to that statement.

However, this issue has actually helped me stress the importance of running at least two sessions with enabled Secure Backup. Following that rule has proven beneficial in various situations in which one session was unable to decode something (because of a software issue, accidentally signed out, phone gone lost, ...) but the second session could decode and share the secret.

UkeHa commented 2 years ago

For me it worked on my desktop but not my phone (i'm using the latest beta branch on my phone). So i reset the cross-signing keys on my mobile device and reconfirmed my device via the qr code on my desktop. That was the only thing to instantly bring back all messages. Logging out and back in did nothing for me. This worked on a private server with federation on the lastet stable build.

dessalines commented 2 years ago

I just encountered this on a 2nd device. This is a critical bug that won't let you use any new login from an android device.

benmordecai commented 2 years ago

This issue may singlehandedly encourage my team to not use a Matrix server and to switch to a proprietary solution.

crashbandicode commented 2 years ago

Is there a way to get it more prioritized/looked at?

Jul 19, 2021 3:51:35 PM Ben Mordecai @.***>:

This issue may singlehandedly encourage my team to not use a Matrix server and to switch to a proprietary solution.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub[https://github.com/vector-im/element-android/issues/1721#issuecomment-882814425], or unsubscribe[https://github.com/notifications/unsubscribe-auth/AUBS5FEVZEXCYZMQL7ZCFNTTYR64PANCNFSM4O7HMMCQ]. [data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAEIAAABCCAYAAADjVADoAAAAAXNSR0IArs4c6QAAAARzQklUCAgICHwIZIgAAAAnSURBVHic7cEBDQAAAMKg909tDwcUAAAAAAAAAAAAAAAAAAAAAPBjRFIAASHKmKkAAAAASUVORK5CYII=###24x24:true###][Tracking image][https://github.com/notifications/beacon/AUBS5FE4QN2WG6VUPBSLAILTYR64PA5CNFSM4O7HMMC2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOGSPK3WI.gif]

benmordecai commented 2 years ago

It is already a priority 1 issue. I am not sure what more can be done. I am not sure anyone has even fully identified the cause of the bug, but it is troubling that this bug has been active for over a year with a P1 for about 11 months of that time without any progress.

JanZerebecki commented 2 years ago

There are a few things that are a more specific cause than this issue that includes multiple different problems or that can be done to improve this issue:

https://github.com/vector-im/element-android/issues/1721#issuecomment-880214682 reads:

Try issuing a /discardsession command on the client not experiencing the issue. That seems to have cleared things up for me.

That this helps points to one issue.

Another issue is if you ignore someone or enable "Never send encrypted messages to unverified sessions from this session" then they will see this. The UI should make this more obvious which is which on both sides.

https://github.com/vector-im/element-web/issues/2996 has a list of more issues related to this. I haven't yet read enough of the spec to say if the top level of the protocols related to keys, signatures and encrypted messages might benefit from a demonstration that they can't end up in an unwanted error state. There might be missing tests both on the server and client https://github.com/matrix-org/matrix-doc/issues/3287 side related to this. If you want to identify the exact cause of one specific instance of this problem: First note the detailed error code you can see when dev mode is enabled. If that and the first 2 paragraphs of this comment don't already help, maybe writing down all the details of the protocol happenings related to the failed message can help you identify the cause and a workaround.

One way to entirely avoid this issue is to avoid encrypted rooms.

dessalines commented 2 years ago

I just lost a family member using element because of this. We'll send a message and it'll get through fine, on multiple devices, then 12 hours later a new message will be stuck on "waiting for this message...."