signalapp / Signal-Android

A private messenger for Android.
https://signal.org
GNU Affero General Public License v3.0
25.53k stars 6.12k forks source link

Messages sometimes show up late and/or in bunches #8692

Open jeremymasters opened 5 years ago

jeremymasters commented 5 years ago

Bug description

Typing on the desktop client and after awhile I heard my phone buzz. Opened it up and a flood of signal messages come in that were from the last several minutes (15 minutes maybe)

Steps to reproduce

Actual result: signal messages from recent history flood in Expected result: messages come into phone roughly at the same flow as desktop while on the same network

Device info

Device: Samsung galaxy s9 Android version: 9.0 stock Verizon Signal version: 4.35.3

Link to debug log

https://debuglogs.org/6d156ab53db2297c534498e59937440ab48bb8d5ef77527afd850adcaa89c26f

Mansarde commented 3 years ago

Let's face it, Signal will never be able to fix this problem. I had this problem since 2019 and it never got fixed. If you have this problem despite applying the troubleshooting steps, then your only real solution is to switch to iphone signal or just use plain SMS. This is beyond Signal dev's ability and Google is purposely doing this to punish people using privacy apps like Signal.

Have you already tried to disable the battery optimization for Signal? That is usually the problem with messages arriving late or only when you explicitly open Signal. These settings are not within Signal itself because it's a "feature" of many phones. See here: https://dontkillmyapp.com/

Additionally you might need to enable autostart for Signal, so that it is allowed to start in the background on its own when you power on your phone.

And as a third setting you could look into whether your phone restricts data usage of the Signal app when the screen is turned off (also a "feature" that helps save battery, but is a problem for apps that rely on realtime-notifications).

nvx2004 commented 3 years ago

The described issue happens also with older Android versions (e.g., 5.1.1). I started encountering the delayed message delivery and call notifications around the beginning of Sep 2021.

On many occasions, I sent a message to someone only to receive in a short while (units of seconds) a message from them which has been sent much earlier (sometimes even several days ago). The timestamp shown for such delayed messages/calls always corresponds to the time when these have been displayed by the phone.

dandoor commented 2 years ago

This problem (together with calls not going through) has always been the status quo for me and others I know.

Description:


Situation 1:


Situation 2a:

Situation 2b:

Situation 2c:


Situation 3a:

Situation 3b:


Analysis:


Situation 4a:

Situation 4b:

Situation 4c:


Analysis:


TLDR:

dandoor commented 2 years ago

Another note is that, even when Signal warns about Play Services being unavailable, connections are being made to a number of google servers with subdomains like "stun", "android.clients", "clients3", etc.

morphine00 commented 2 years ago

@dandoor that was some serious investigation, congratulations!

stale[bot] commented 2 years ago

Is this still relevant? If so, what is blocking it? Is there anything you can do to help move it forward?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

Vikingtons commented 2 years ago

can confirm this issue is still relevant

ColonelPanic-42 commented 2 years ago

This issue is still relevant, and will continue to be relevant until the devs actually care about it. Almost 3 years now, dozens of posts in this thread plus other posts online, but the official response is "it's an issue with your device, not our problem"?

At the very least, make it easier for us to fall back to the websocket connection. It's a pain to try to workaround by disabling Google Play services, because of the massive amount of errors it throws, and on some devices it can't be disabled. Make it an option in a debug menu or something, that would take virtually no dev time and would be a massive help.

LeftHandRev commented 2 years ago

This is happening to me. I did not have the issue on Android 11, but after upgrading to 12 on my Pixel 5, this started happening. It's absurdly annoying.

dannypurcell commented 2 years ago

Is this still relevant? If so, what is blocking it? Is there anything you can do to help move it forward?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

If the devs care about good bug reporting then this bot should be removed or otherwise changed to stop trying to throw away good bug reports that people have put a lot of work in to and others have already confirmed as a continuing issue. It's fairly easy to tell just from the number of comments and length of each one that this isn't some trivial post that should be so easily lost to time.

Fix the issue or don't but definitely make that decision after serious consideration. It's rather disrespectful of caring people like @dandoor to have a lazy bot throwing away their work just because people don't have the patience of gods required to keep pestering the team about it for years on end.

dannypurcell commented 2 years ago

Also just confirming again that this is a persistent issue on Android 12 Google Pixel 4a with Google Play Services enabled, all permissions for Signal allowed, and Google Play Services permissions as follows:

Signal is also set to background data allowed, unrestricted data usage, and unrestricted battery usage.

Issue persists after having cleared storage/cache, uninstalled/reinstalled, factory reset and reinstall app, change of phone and fresh install. I've confirmed with the carrier (Google Fi) that I have not had any general issues receiving SMS. The issue in Signal presents with contacts sending me SMS and with a contact who also uses Signal so their messages are Signal messages via data.

This happens so often it's a running ~gag~ irritation with a close friend. It actually really sucks to not be able to rely on getting a notification that they have sent me a message. I have to keep checking the app once an hour or so or else I'll open it and suddenly get a bunch of messages where they were trying to contact me and it just didn't land on my phone.

Since we can replicate this issue basically at will and I really want to be able to rely on Signal, I am open to being a test case here. Just let me know if there's any way I can demonstrate the issue while a dev is watching or anything else that can be done to move this issue along.

Vikingtons commented 2 years ago

@greyson-signal Is there anything that can be provided to help triage this issue? Would an adb logcat be of any use here if we try to repro a scenario where inbound messages do not prompt a notification?

TIA

T3sT3ro commented 2 years ago

could this issue be related to aggressive power saving policies on some smartphones? I notice that signal is not the only app that has problems with push notifications sometimes. This site explains it better: https://dontkillmyapp.com/

greyson-signal commented 2 years ago

@Vikingtons Yeah, specifically a debuglog (Signal Settings > Help > Debug log > post link here) can tell us at least if the problem is:

The problem seems to always be the first one, and having a log can give us phone details and such that can help us gather info and reach out to the right people.

Vikingtons commented 2 years ago

Excellent, will do. Would I be correct in thinking the log buffer is approx 10 minutes?

greyson-signal commented 2 years ago

@Vikingtons For logcat, the log buffer size is determined in your system Developer Options settings, and will default to a pretty small window (probably several minutes). Logcat is helpful but not nearly as helpful as a debug log, which will usually cover at least 24 hrs and have a bunch of other helpful stuff for us.

Vikingtons commented 2 years ago

Noted. Will try to repro this afternoon.

LeftHandRev commented 2 years ago

@dannypurcell @Vikingtons @greyson-signal - this issue was mitigated by installing the universal APK that doesn't rely on Google services. It returns whenever I allow Signal to be updated OTA via the Store update. In other words, I have been able to circumvent this with the manually installed APK located here: https://signal.org/android/apk/.

Vikingtons commented 2 years ago

I've read about this before, and I appreciate the info, but the issue should be root caused if possible. Delayed notifications shouldn't be a thing for the vast majority of people downloading Signal directly from Google Play.

I'll try to repro on mobile data and on WiFi since (I believe) both network options have their own GCM 'heartbeat'.

LeftHandRev commented 2 years ago

Yes, I understand that it and definitely agree. My point in bringing that up is that there may be some sort of indication between the store version and the "universal" APK. Forgive me if what I'm saying is unhelpful.

Vikingtons commented 2 years ago

Amazing. I'm unable to repro with my phones locally. I have a friend who persistently has issues with notification delivery on his Samsung Galaxy phone (heavily implicated at https://dontkillmyapp.com/).

I'll check that they already have battery opts disabled, and I'll ask them to provide a debug log if they do.

Edit: it turns out my friend hadn't properly configured their battery optimisation settings for Signal on their phone. You can argue that this is a bit of a UX nightmare for any app or service that delivers push notifications on Android, but that's enforced by the vendor ROM, and not necessarily a Signal problem. Perhaps first-time installations of the Signal app could prompt the user to properly configure the app / point to a FAQ resource to ensure this is set up (if it doesn't already, I can't actually remember).

If I run into this issue in the future I'll be sure to submit a debug log for further investigation.

LeftHandRev commented 2 years ago

As an aside, I'm running CalyxOS 3.4.0 / Android 12, kernel 4.19.220-ab22304000 on a Pixel 5.

dannypurcell commented 2 years ago

@LeftHandRev @greyson-signal WRT working around the issue by using the universal apk, I may be showing my lack of Android dev experience here but is there a compelling reason to keep trying to rely on Google services if the universal APK works and doesn't have this issue?

It seems like one fairly straight forward fix would be to ship the universal APK through the store or otherwise make the way the universal APK handles notifications be the same for the store version. Is that not possible or do we lose other stuff for not using Google services?

cody-signal commented 2 years ago

It's interesting to me that the universal apk wouldn't have these issues. The primary difference is that the universal apk include our native binary dependencies for every CPU architecture instead of just the one of four. I don't see how that would make a difference with push notifications.

the universal APK that doesn't rely on Google services

This is an incorrect statement. The universal APK will still use Play Services if available.

Eric678 commented 2 years ago

Yes I confirm that installing the universal APK does not fix this problem. I my case I am not able to stop or disable Play Services and it refused to instal without using play services to authenticate the number and still has the notification problem. I block play services from calling home at all other times. Is there any way to force the use of websocket?

LeftHandRev commented 2 years ago

Yes I confirm that installing the universal APK does not fix this problem. I my case I am not able to stop or disable Play Services and it refused to instal without using play services to authenticate the number and still has the notification problem. I block play services from calling home at all other times. Is there any way to force the use of websocket?

@Eric678 - This is not a problem with Signal. This is a problem with allowing alternate installation sources. You need to grant that ability to your browser or whatever tool you are using to download the Universal APK. Your report doesn't seem to be relevant, since you were unable to install the Universal version. If you can install the Universal APK and verify that it does or does not fix your issue, then revisit this post.

wunderox commented 2 years ago

Not sure if it's due to the same problem but my symptom is similar.

I was actively using signal on my desktop when I suddenly received a voice memo from a friend from 7h earlier, despite having texted with multiple other friends from both my phone and my desktop.

Desktop: v5.41.0 (flatpak on fedora) phone: v5.36.3 (official apk from signal.org, grapheneos, no google services).

This is not the first time it has happened, but it's rare. It can happen without me using my desktop either, just suddenly get a notification with a text from a friend, and the received time is now and the sent time is hours earlier. It may correlate with them having iPhones, I think I saw some mention of that somewhere but I can't recall.

Here's a debug log: https://debuglogs.org/android/5.36.3/c734a075cab9a367d2e7621c9b3997e10289e1d77c6288d50a49c2d0c2a662ed

Just a brief description in CET (central european timezone):

Important note: yesterday ported signal from my old phone to my new phone and reactivate my Signal on my desktop (which was unlinked apparently due to porting my signal from old phone to new phone).

EDIT: Not sure actually whether this friend A has an iPhone or not. If important, can double check with them. EDIT2: Yesterday is the time I switched from the Google Play version of Signal to the official signal.org hosted apk. Before that I was always using the google play version. This issue has happened then and now is the first time it has happened on the non google play version.

cnleo commented 2 years ago

Today (May 3) I received a message dated April 29. I received this message after plugged in the power cable. Power and data saving is turned on. Now I have changed my settings so that Signal does not have the restrictions of data and energy saving. Maybe it will work better now.

Edit: Systeminfo: Stock Android 10, Aquaris 2

https://www.reddit.com/r/signal/comments/mnwn34/delayed_messages_going_on_for_years/ https://github.com/signalapp/Signal-Android/issues/8604 https://github.com/signalapp/Signal-Android/issues/9930 https://github.com/signalapp/Signal-Android/issues/11159 https://github.com/signalapp/Signal-Android/issues/11079 https://github.com/signalapp/Signal-Android/issues/11048 https://github.com/signalapp/Signal-Android/issues/10947

LeftHandRev commented 2 years ago

The supposedly immune non-App/Play store APK is now delaying messages as of version 5.40.4.1 - prior to this, there were no issues with message delays.

Did code from the main distribution get merged into this binary? All I know is that this issue should not be considered closed as it's still happening. Frankly, I've been running on the direct APK for a while and am okay with how I update it; I prefer Signal to work even if it takes a little extra effort on my part.

alex-signal commented 2 years ago

The code is identical and always has been. The only difference is how we listen for messages, and what features are available. This issue is still open, FYI.

LeftHandRev commented 2 years ago

Alright, perhaps my wording was incorrect, but after almost 9 months of using this method to get around the timeout problem, it cropped up again with the latest version with no other changes to my device.

I saw that the issue was still open, which is why I edited my initial message and removed that line. sorry about that.

Anyhow, thanks for the clarification. It's just been a bit frustrating as I'll open my phone for a standard SMS, then suddenly get bombarded by a dozen or more secure messages that haven't caught up.

CelianGdfrd commented 2 years ago

Hi, I've been having this problem for quite a while, around once every 4-5 days. But for a week now, it's way more regular, practically making the app unusable... I'm not receiving messages, until I send a message to the person, then messages arrive (with correct sending date/hour).

Did something change recently please ?

LeftHandRev commented 2 years ago

Agreed - something has changed significantly since the last release.

cody-signal commented 2 years ago

@LeftHandRev @Kalytis can you provide updated debuglogs please?

I'll review the commits to make sure, but nothing from memory jumps out as likely culprits.

CelianGdfrd commented 2 years ago

Sure, could you please provide me with a way to get the logs please ?

LeftHandRev commented 2 years ago

I love that you just asked this because literally just got bombarded by 12 Signal messages that were on my Desktop client from 8am (I'm on PST, so they were 4+ hours old).

@Kalytis - it's under "Help" and you'll see Debug Log.

@cody-signal - I'm submitting it now. Is there a place I can submit it privately?

LeftHandRev commented 2 years ago

5.41.11 has been released - downloading now to see if this fixes things.

LeftHandRev commented 2 years ago

Initial tests since the update are showing positive results using the "global" version of the APK (the non-Play Store version). Messages are arriving in my Signal Desktop and within 5-10 seconds, I'm getting notified on my phone. I've asked 3-5 people to test and it looks good.

So, whatever was changed seems to have been fixed for me on my phone.

cody-signal commented 2 years ago

@LeftHandRev you can send it to me via email cody at signal dot org.

cody-signal commented 2 years ago

@Kalytis Settings -> Help -> Debuglog and then copy paste the link here.

LeftHandRev commented 2 years ago

@cody-signal - expect it to come from the domain dumb dot org. (not joking).

wunderox commented 2 years ago

Still having issues with message delivery with this latest signal version.

Was having a conversation with a friend last night and sent a message which was delivered only this morning. Here's their debug log: https://debuglogs.org/android/5.41.11/d5a111b0b28c0e54189fbfe1c40450acf5a7348fbd1df622cc1f7db756da1cfa And here's mine: https://debuglogs.org/android/5.41.13/7e51c8a4a168632f539d97d5cf0a0b72d58d85ffeac5ce45485a01917d647353

The message in question has the ID 1656783464437, was sent Yesterday 7:37 PM and delivered Today 10:49 AM.

EDIT: And I had another conversation with another person after that message was sent, without issue.

UserX404 commented 2 years ago

2 days delayed 1657017321774 https://debuglogs.org/android/5.41.13/ae274462d49118e819a4af5d215a3d7b99fd1b357c6950ef114d817531443582

LeftHandRev commented 2 years ago

Can confirm. Was working great with 5.41.11 but 5.41.13 messed everything up again.

What in the world is going on?

greyson-signal commented 2 years ago

@UserX404

2 days delayed 1657017321774 https://debuglogs.org/android/5.41.13/ae274462d49118e819a4af5d215a3d7b99fd1b357c6950ef114d817531443582

Unfortunately I don't see that message in your logs anywhere. Are you sure that's the correct timestamp? Also, 1657017321774 is July 5, 2022 @10:35 UTC, which is 5 min before this log was taken. If it was two days delayed, I'd expect the message sent timestamp to be from July 3.

@LeftHandRev

Can confirm. Was working great with 5.41.11 but 5.41.13 messed everything up again.

5.41.12 and 5.41.13 just fixed some buttons. There were no changes to message processing. What is more likely is that your phone is going through different battery modes and/or OS updates which is causing our notifications to get deprioritized.

@wunderox

Was having a conversation with a friend last night and sent a message which was delivered only this morning. Yeah, I see you sent the message just fine. The receivers logs go mostly dark around the same time, and the system is reporting that it doesn't have any network. They might have legitimately not had network, but some phones will also lie to apps about network availability as a battery optimization. I couldn't say which was happening here.

LeftHandRev commented 2 years ago

No, I don't think it does mean that @greyson-signal. My phone has not been updated in two months. I'm on a very specific schedule for this phone and the OS I'm running. Furthermore, no battery options have been changed, nor has anything else in the 24 hours between the release of 5.41.11 and 5.41.13.

I understand you're trying to quell some of these issues, but I'm no slouch with regards to knowing my device. I also have been using Signal for ten years, give or take. I support you guys and believe in your product. But over the last six-eight months, popping between versions has created inconsistencies with delivery.

If that weren't the case, there wouldn't be so many people reporting this issue.

greyson-signal commented 2 years ago

@LeftHandRev I'm not trying to say there isn't something we could do better -- what I'm trying to say is that it's literally impossible for 5.41.12/13 to have changed anything with message processing. Feel free to look at the commits.

Cody sent me your log from 5.40.4.1. The weird thing about it is that the device is reporting Play Services and FCM availability... but it never once receives an FCM message. I've never really seen that. I see your build host is "aosp-build" though, which makes me think you're running some bare, possibly-self-compiled Android distribution? If so, that could have a lot to do with it.

Also just wanted to point out that the whole "universal/website APK vs Play Store apk" thing is a red herring. These APKs are functionally identical. The only difference is that the website APK has native libs for every supported CPU architecture (for ease-of-distribution -- this has no impact on message processing) and has a little in-app APK updater in it. There is no difference in how the website APK uses Play Services. All versions of Signal are built to use Play Services if they're there, and fall back to a persistent websocket connection if they're not. Simple as that.

I also just want to give everyone a general update here and share some information. We're continually trying to make improvements to our notification reliability. The problem is that notification systems on Android can change drastically based on device, OEM, carrier, OS, etc.

Did you know many OEMs have specific lists of apps that they allow to handle notifications properly? They push these lists silently to devices independent of any app update schedule.

Did you know that different OEMs have wildly different non-standard mechanisms for battery management, some going as far as killing every app (that isn't on their secret whitelist) as soon as you background it? Man, I just got a log from someone with a SHIFT device that revealed that they were turning all of our high-priority notifications into "unknown priority" notifications, resulting in us not being given our execution window to fetch messages. First time I've seen that.

Did you know that independent of that, Android has an entire invisible state machine that can change if notifications are delivered to a device based on how many notifications you're getting, how often you open the app, your battery level, whether your device has been sitting on a table not being touched for a while, etc? And that they change/refine it in every new Android version?

I don't want people to think we don't care or are just being lazy about this or something. The fact is that notifications are a black box inside black box inside a black box. They work very well for very many people... and then for some people they're just completely broken. The best thing we can do is continually get logs, detect patterns, and address what we can address.

So please, keep sending logs! And we'll keep trying to make improvements :)

ColonelPanic-42 commented 2 years ago

Also just wanted to point out that the whole "universal/website APK vs Play Store apk" thing is a red herring. These APKs are functionally identical. The only difference is that the website APK has native libs for every supported CPU architecture (for ease-of-distribution -- this has no impact on message processing) and has a little in-app APK updater in it. There is no difference in how the website APK uses Play Services. All versions of Signal are built to use Play Services if they're there, and fall back to a persistent websocket connection if they're not. Simple as that.

I believe this; for me I had to use the universal APK, but that was only because I could then install it with Google Play Services disabled so it would force the websocket fall back. This is a massive pain though, as there is a constant barrage of errors and crashes until Play Services is reenabled, and on some phones I've found I couldn't get to a point of installing the APK at all while it was disabled.

@greyson-signal is there any reason an option couldn't be added into Settings somewhere (maybe in an "Advanced" section or something) that would allow the user to switch the listener method (Play Services or websocket) rather than simply checking at install time what's available? It's clearly saved as a configuration somewhere as it checks on first run and then persists it - by exposing it as an option in the UI it would make it much easier for those of us who have issues to workaround it, and make it easier for us to switch back and forward to test as well.

LeftHandRev commented 2 years ago

@ColonelPanic-42 - Thanks for bringing up adding the websocket/Play Services as an option. I thought about that some time ago when I was trying to research ways to get around FCM. I am running CalyxOS and do use Play Services through their anonymizing proxy (pseudo-anonymizing, but I take every inch I can get).

@greyson-signal and I don't think that you're being lazy; I just know that things either work or they do not after updates. When they're not working, I don't typically notice for 6 to 8 hours after which I get a barrage of messages all queued up for delivery.

I do appreciate what you're doing. I can only offer empirical evidence for my issues and monetary support in the form of monthly recurring donations.

greyson-signal commented 2 years ago

FYI looks like there was a subtle change in exception handling in 5.41.0 that made things less reliable on devices that did the whole "make the priority of this notification PRIORITY_UNKNOWN" thing (that, turns out, was happening on more than just the SHIFT devices). See #12325.

@LeftHandRev I know you said your issues happened between 5.41.11 and 5.41.13, but it could be the same issue and you just might not have noticed for a bit? The log you sent Cody showed no indication of it and was funny for the reasons I mentioned, but hard to say (note your log was for 5.40.4.1, but it wasn't showing the behavior that would cause issues in 5.41.0).

Anyway, if you just started experiencing issues in 5.41.x, there's a chance they may be fixed in 5.41.14, which I'm releasing now to production :+1: