Delayed/missing push messages

[fd0]

When messaging a friend of mine via push, most messages are delayed a few hours. Background data is enabled. Sometimes messages are even not delivered at all or arrive in different order than sent. This also happens with messages other people send him, although I can exchange messages with these people without problems (delivered within a few seconds). He runs a Nexus 5 with the latest stock Google Android and is perfectly reachable via Hangouts the whole time.

Any idea how to debug this?

[monotek]

Same from textsecure to signal...

[2-4601]

I experienced this first time today. I did not receive a group message from a TS user until I opened the app.

Debug log: https://gist.github.com/anonymous/6b9ea411bf1aefb85eb4

W/GcmBroadcastReceiver( 9114): GCM message...
W/PushServiceSocket( 9114): Push service URL: https://textsecure-service.whispersystems.org
W/PushServiceSocket( 9114): Opening URL: https://textsecure-service.whispersystems.org/v1/messages/
W/PushServiceSocket( 9114): Push service URL: https://textsecure-service.whispersystems.org
W/PushServiceSocket( 9114): Opening URL: https://textsecure-service.whispersystems.org/v1/messages/
W/PushServiceSocket( 9114): Push service URL: https://textsecure-service.whispersystems.org
W/PushServiceSocket( 9114): Opening URL: https://textsecure-service.whispersystems.org/v1/messages/
W/PushServiceSocket( 9114): Push service URL: https://textsecure-service.whispersystems.org
W/PushServiceSocket( 9114): Opening URL: https://textsecure-service.whispersystems.org/v1/messages/
W/PushServiceSocket( 9114): Push service URL: https://textsecure-service.whispersystems.org
W/PushServiceSocket( 9114): Opening URL: https://textsecure-service.whispersystems.org/v1/messages/
...
[^these get repeated a total of 100 times]
...
W/PushNotificationReceiveJob( 9114): ***** Failed to download pending message!

TS 2.17.0, Nexus 4, stock 5.1.1, no weird things

As this was just a single case I'm doing this just for documentation.

Possible cause might be that some hours before I played around with sending novels in group names?

[moxie0]

@2-4601 Very interesting log, thanks!

Unfortunately I'm not logging the exception there, so I just committed something to do that. I should also be building a generic notification in this failure case.

[joshproehl]

There are currently two messages sent via secure TS channel on my wife's phone that have not been delivered to mine after 3 hours. Newer messages have been delivered, but these two are just AWOL. (Her device reports correct sending and shows the check-mark)

Any debugging info that should be captured to help identify this problem?

[moxie0]

@joshproehl We'd need a debug log from 3 hours ago on your device.

[joshproehl]

@moxie0 I'm now seeing an alert that I have two new messages (From my wife) whenever TS is locked. Upon unlocking no message is received, and the alert disappears, but when TS is re-locked I once again get an alert for two "Encrypted Message" from her. (The two missing messages are still not appearing in the conversation)

I've uploaded a capture from logcat -v long while locking and unlocking to reproduce this, although I'm no longer certain if this is strictly the same issue since my phone seems to at least have an idea that there should be two messages.

[BP602]

I had this happen today : https://gist.githubusercontent.com/anonymous/d79103709046190bc58a/raw/5c03d9e6dbb2f2ff712726bb389f8b7af5726ddc/cat.log

I received a lot of notifications from a group at once, almost two hours of group TS messages, the phone was vibrating constantly for about 2 min. The notification counter over TS's icon was showing 60

[McLoo]

Just had TS notify a SMS After opening TS and unlocking, one (or more) older push messages were notified:

https://gist.github.com/3d64b6916bdaad5c4eb6

Having Sony Stamina battery mode activated, but TS excluded (works as expected: as SMS was notified)

[McLoo]

Again, but this time no SMS, so no notification at all. Opened and unlocked, and a push message came in 5 hours later. On good WiFi all time EDIT: except for about 5 minutes at the sending time. Weak WiFi or weak 3G

https://gist.github.com/82e58fe6ce830c811cdd

[McLoo]

Man, I'm so dumb. TS got removed from stamina exception list on uninstall (testing PRs) Leaving the above entries though. Maybe of help anyway.

[joshproehl]

I am continuing to receive notifications whenever TS is locked that I have two new messages. These messages never appear in TS. I'd prefer not to reinstall if possible. Any other suggestions for forcing these delayed messages to be downloaded, or destroying record of them? (Re-initiating secure session does not help, restarting the phone does not help.)

[jeremymasters]

Sounds like database inconsistency. Like the message was read but the notification never cleared.

[McLoo]

@moxie0 what @joshproehl describes sounds like the pending push message that were also in #2287. Thought it was fixed.

@joshproehl what version are you using? Oh, and you should open a new issue for your case.

[McLoo]

One more time. Excluded from stamina energy saver this time. Any relation to that intent crash?

https://gist.github.com/dc3c18e31be664a0941c

Edit : One more time means : received old messages after unlocking

[ghost]

I have not read thoroughly all the comments (there are too many), so I apologize if I say something that has already been said. My sister has a Huaweii phone (with some Android 4+, I think 4.1 but I should check) and she only ever receives messages when she opens textsecure. She does not, ever, receive textsecure messages in any other situation. She must open it to receive all the messages she should have received in the previous hours/days. She runs one of the latest versions (either 2.19 or 2.20), and we don't know what to do. I always have to send her an SMS so that she will open textsecure and get all the push messages she missed. She isn't using any weird apps on her phone that may mess up her network (she's a very "average" user), and I don't know what's up. All her other applications receive messages just fine. Thanks

[2-4601]

Huawei's EmotionUI is a headache known to cause these kind of issues: #3275, #3376

[ghost]

Unfortunately that doesn't seem to be her problem, she has no weird privacy or battery saving settings enabled. Actually, she doesn't have those settings at all.

[moxie0]

@oaim She's not receiving GCM messages. That means either: 1. she has background data disabled for TS or for Google services, 2. she has a bad gapps install, 3. she has used something like titanium backup or another backup/restore app, or 4. she's on a network that blocks GCM.

[ghost]

Hi moxie, thanks for your answer.

1. I think it's impossible, she does receive messages on other apps like google hangouts, and I'm assuming that uses google services, right?
2. How could that have happened? How can we know for sure, and how can we fix that?
3. She didn't use titanium backup, she doesn't even know it exists lol (and she can't have used anything else, she is not that into technology and never did anything out of ordinary with her phone)
4. I don't think her network blocks GCM or other apps would have problems as well, am I wrong?

[moxie0]

@oaim This isn't a support forum, so I can't really help diagnose your sister's phone, but those are the only possible reasons I know of that GCM messages would not be delivered. There's also all the Huawei specific stuff. If you had a debug log from the exact moment a message failed to arrive we could rule anything else out.

[ghost]

Do you need a debug log when she opens TS or when she fails to receive a message I send her?

On 2 July 2015 at 18:03, Moxie Marlinspike notifications@github.com wrote:

@oaim https://github.com/oaim This isn't a support forum, so I can't really help diagnose your sister's phone, but those are the only possible reasons I know of that GCM messages would not be delivered. There's also all the Huawei specific stuff. If you had a debug log from the exact moment a message failed to arrive we could rule anything else out.

— Reply to this email directly or view it on GitHub https://github.com/WhisperSystems/TextSecure/issues/970#issuecomment-118078950 .

[moxie0]

@oaim The latter.

[ghost]

The push fixer app seems to have fixed the problem so far! https://play.google.com/store/apps/details?id=com.andqlimax.pushfixer.noroot Thank you for your help, we will see if the problem surfaces again

[2-4601]

This Sunday (around 3:50–4:00 PM GMT, August 9, 2015) I experienced this issue again. Failed to receive two messages in the background and got them only after opening the app.

Exactly same behaviour as in my previous case where each failed message outputs 100 times these two lines:

W/PushServiceSocket( 4070): Push service URL: https://textsecure-service.whispersystems.org
W/PushServiceSocket( 4070): Opening URL: https://textsecure-service.whispersystems.org/v1/messages/

I was at the time on an stable wifi and a stable Internet connection.

Full debug log right after it happened: https://gist.github.com/anonymous/f9cb701ec4ce5b279a43 The sender's number is +**********01 and it was a private conversation. There's an exception caught, hopefully it's useful.

TS 2.25.1, Nexus 4, stock 5.1.1

[McLoo]

Tracked down all of my occurrences for this issue to battery saver/preserve functions in some way.

[moxie0]

@2-4601 Thanks for the log. Looks like we're getting the notification, trying to retrieve the message, but the network connection is failing (even though the system reports that it is network connected). We should display a generic "you have a new message" notification after giving up. Can you open an issue specifically for that?

[rmueller83]

A friend of mine is also complaining about missing push messages, but he is using Signal on iOS. So perhaps this is some server-related issue.

[donateur]

I'm still experiencing this problem with TextSecure 2.25.3 on Android 5.1.1. Messages can be delayed by many hours, often until I check the app itself. The problem occurs on good WiFi networks, not just a carrier data network. Even if the app is in the foreground I am often sometimes not notified of a message until it's more than 30 minutes old. On the other hand SMS messages are received instantly.

Are there any suggestions to improve the situation?

I previously had restricted background data for "Google Services", but even after disabling it and restarting my phone the problem persists. No apps have background data restrictions now and in the process of updating Android I rolled the Google Play Services and Google Play Store apps back to default then updated them to latest (with reboot).

I do not believe it is related, but for reference I'm running CyanogenMod 12.1 on a Nexus phone and using TextSecure from the Play store (not the built-in WhisperPush messenger supplied by the Mod as I've been using TextSecure from before CyanogenMod offered it). My wife has another Nexus phone, running CM 12.1 and TextSecure without problems - she gets encrypted messages instantly.

(Edit: It seems that I do sometimes get notified of an encrypted message instantly if TS is in the foreground. I wonder if the problem is with messages from specific contacts being delayed?)

[2-4601]

@donateur Could you post a debug log after it happens?

[wizardofid]

I'm having this problem (TS only gets new messages when in foreground) since the update of my Fairphone OS. There were no problems before. And i don't have background data restricted or ever used xxx-Backup. In the Fairphone-forum i found some posts related to such problems which helped me: The problem is fixed by installing or upgrading any app via the Google Play Store. Unfortunately this fix only lasts until next reboot. PushNotificationFixer, PushNotificationTester did not help. After all i think the problem is caused by Google Play Services 7.8.99 (213422-034).

I'm not really sure if #3925 helps in my special case. But it sounds easy to implement and might improve the ux of many users significantly. 2 of about 20 TS-users i know are affected by delayed notifications. For an Instant Messenger that's not good... And they blame it to the messenger and are not interested in technical details :-(

Update: For clarification: In my case the whole phone is after reboot no longer GCM-connected. This is shown by displaying WiFi-strength in light grey. If GCM-connection is established, this is displayed in light blue. Problem seems not TS-related and other messengers should be affected in same way.

[siccovansas]

Regarding the previous comment, I know 2 Fairphone users who have the same problem since several weeks (presumably also since the latest Fairphone update). To clarify, they only receive messages when they open TextSecure or if it opened.

[mawed]

@wizardofid @siccovansas Try disabling and enabling the TS-messages in the settings. As you reinstall gapps the GCM-registration needs to be renewed. AFAIK this helped sometimes.

[wizardofid]

@mawed I did this, more than once... Also reinstalling Google Apps... Double-checking every setting, including 'WiFi Optimization'. After reboot the GCM-connection is gone.

[wp9015362]

Kinda sorry to say, but finally needed to uninstall TextSecure due to this issue.

In comparison to Telegram for example, TextSecure simply is too unreliable in message delivery.

Still don't understand why TextSecure still depends on GCM, while Telegram works perfectly fine without it.

Anyway, since deprecation of GCM doesn't seem to be a priority for TextSecure and since that means that this issue probably won't be fixed for a while, the decision to uninstall TextSecure and to replace it with Telegram has now come.

Regards

[Mihahn]

A friend of mine only receives messages when he opens the app, too. Like that, he's unable to use TS... (Lg Optimus L7)

[Wikinaut]

@Mihahn check Settings for the app. Newer Android versions allow to fine-tune for each app whether it stops when the screen saver becomes active, or not. So try to find such a setting so that TextSecure app is protected from not being killed or stopped on screen saving.

[donateur]

@2-4601 I can do better than that, I can send two debug logs! Note: the encrypted message in one log has had numerous random characters removed but otherwise the logs are otherwise untouched:

https://gist.github.com/anonymous/d3b3dfa78e45e8d0a604 https://gist.github.com/anonymous/fd7204ee60364a8ebecc

Although I doubt it will help, as the other person who messages me most is often using the same networks without problem, I'll try the "Push Notifications Fixer" from the Play store just to confirm.

[Mihahn]

@Wikinaut the android version on his phone is 4.1.2, I guess, but I have no idea where to find such settings on an lg rom. But I'll check it again anyway!

[jeremymasters]

Split off my stuff as new issue #4052 since symptoms weren't exactly the same.

[donateur]

As a follow up to my own previous post, I've now tried "push notifications fixer" from Google play with various settings but no success.

Now I've been paying attention to it, I've found that all text secure messages sent to me are delayed if I have another app in the foreground. It's really frustrating!

Unless it's the only way is to regenerate new keys (or reinstall the app) I'm not prepared to do that as I've got a lot of people using text secure who I often talk with. It seems crazy to have to ask them all to accept a new random key from me, keys shouldn't change for no significant reason.

If anybody has any suggestions or knows anything I can do to ensure GCM will work reliably with Text secure please let me know. (I seem to get gmail in a timely manner)

[moxie0]

if you just unregister and reregister, your keys won't change

On 09/10/2015 06:26 AM, donateur wrote:

As a follow up to my own previous post, I've now tried "push notifications fixer" from Google play with various settings but no success.

Now I've been paying attention to it, I've found that all text secure messages sent to me are delayed if I have another app in the foreground. It's really frustrating!

Unless it's the only way is to regenerate new keys (or reinstall the app) I'm not prepared to do that as I've got a lot of people using text secure who I often talk with. It seems crazy to have to ask them all to accept a new random key from me, keys shouldn't change for no significant reason.

If anybody has any suggestions or knows anything I can do to ensure GCM will work reliably with Text secure please let me know. (I seem to get gmail in a timely manner)

— Reply to this email directly or view it on GitHub https://github.com/WhisperSystems/TextSecure/issues/970#issuecomment-139234023.

http://www.thoughtcrime.org

[donateur]

@moxie0 Thanks, I hadn't realised that!

Unfortunately I cannot actually unregister as the app keeps crashing after I click OK on the OK in the dialog box, this happens even after a reboot of android. Could this be a hint toward the cause of the delayed messages? I've sent logs via the crash report with the URL to this conversation as text.

PS: Thanks for your work on this great app! Textsecure is the first fully encrypted messaging app which I've been able to get nontechnical members of my family using. It's seamless to them (aside from this unfortunate problem which also affects my mom anyway!).

[donateur]

It seems to be in some kind of in between state now. One user reported problems messaging me, until I messaged him. Also I no longer get the second confirmation tick for message delivery, even after the other person has replied to my message. Is this now a separate issue?

Thanks

[donateur]

Ok, I'm unregistered from secure messages, even though the app crashed while doing so. The app believes I'm still using secure messages though. In short: I can send but not receive secure messages in this state.

[moxie0]

Please post a debug log

[donateur]

@moxie0 https://gist.github.com/anonymous/e5a569e8a49cf912de88

[moxie0]

Looks like you don't have google play services installed, or your gapps install is otherwise messed up. That is the cause of both the crash as well as your failure to receive notifications.

[donateur]

OK, any suggestions please?

I'm not aware of problems with any apps or google services. I can try flashing a gapps zip?

[bonanza123]

@donateur try using a gapps zip from http://opengapps.org/

[donateur]

@bonanza123 Thanks! I hadn't heard of that site before.

I was able to flash the Google Apps without problem, and cleared caches, but after reboot TextSecure still wouldn't let me unregister/re-register for Secure Messages. I'm afraid to report that in frustration (I was ill at the time) I ended up reinstalling TextSecure, which along with regenerating my keys (darn, missed that in the backup!) resolved the problem. As far as I can tell all my messages are now delivered on time, even if TextSecure isn't in the foreground.

This doesn't resolve the problem my mom has, as her phone is running stock Android. I'll try to get more evidence/debug log.

Thanks for your help guys.