search

ChatSecure / ChatSecure-iOS

ChatSecure is a free and open source encrypted chat client for iOS that supports OTR and OMEMO encryption over XMPP.
https://chatsecure.org
Other
3.13k stars 1.03k forks source link

Failing push notification #770

Closed ageru closed 5 years ago

ageru commented 7 years ago

Hi,

Sorry to create yet another bug about push...

Setup

Steps to reproduce

  1. Install Chatsecure, log to server
  2. Send a couple of messages to user on iPhone, they reach him, OMEMO works... Everything looks clear.
  3. Confirmed push is active in the app's interface
  4. Leave the app.

Actual result

Expected Result If iPhone is connected to the Internet, the iPhone user gets a notification straight away.

Bonus questions

Many thanks.

chrisballinger commented 7 years ago

Thanks for the thorough investigation. I've never seen that error message before from the push module. It looks like the code has been updated recently: https://hg.prosody.im/prosody-modules/file/tip/mod_cloud_notify/mod_cloud_notify.lua#l73

Perhaps try rolling back mod_cloud_notify to a (much older) commit?

KingKili commented 7 years ago

I have the same issue with https://trashserver.net/

ageru commented 7 years ago

All the modules are automatically updated weekly on my server (and I didn't have any ChatSecure user before the past week-end) so that would be consistent with your theory. As a temporary fix, rolling back to an earlier commit should be possible, I just hope it won't break things with the users running Conversations.

Would you be able to let me know what commit of this module you are using, or alternatively to date it?

By the way, when I noticed the issue, I installed mod_pinger to try to fix it, since you seemed to be using it. Is it in any way necessary to have it installed for ChatSecure and Push to work properly?

chrisballinger commented 7 years ago

mod_pinger shouldn't be required. Not sure the exact mod_cloud_notify commit, but I'll figure that out and test against the latest commit. Push is still working fine as of a few minutes ago on my personal server configuration.

paskalito commented 7 years ago

could this be avoided using this push mechanism ? https://github.com/ChatSecure/ChatSecure-iOS/issues/650#issuecomment-289429776

beeloo42 commented 7 years ago

I'm also encountering this issue I tried the 5 latest mod_cloud_notify versions without success...

afriedmanGlacier commented 7 years ago

We are using eJabber and have changed a bit of the underlying functionality of push, but just thought I'd chime in and say push still works ok for us. However...

@chrisballinger the one problem I have seen lately, I thought I'd add here just in case it is relevant to these issues is that our RubDub has died unexpectedly a few times. Haven't figured out why yet (and we also haven't updated to the RubDub with Rollbar yet). When this happened, eJabber could not connect to it and push stopped working. Have you seen this at all Chris? I'm not sure if you guys use the same RubDub and ChatSecure-Push-Server setup as is available on github?

If you have not had this problem, we'll just monitor it on our end, but just thought I'd add this here in case it is relevant.

chrisballinger commented 7 years ago

Yeah it does sometimes, but I re-provisioned our server don't see this anymore. I have an external port monitor that emails me when it goes down, and it hasn't had an issue since then. (Ubuntu 16.04 and latest Node PPA.)

therob84 commented 7 years ago

Exactly the same problem I encounter with two of my contacts who use ChatSecure 2.0.8 (while I am using Conversations 1.18.4). All of us are using jabber.de (prosody >0.9.10)

I think it is the same problem than stated above, but I could not provide any logfile but are eager to help solving this problem.

Is the reason now already undrestood or is it still a mystery, why/when/where this happens?

I hope the best for a fix in the next release....cheers, Robert

chrisballinger commented 7 years ago

If the Prosody server restarts it will lose all the push tokens because they are only stored in memory. Is the server restarting on regular intervals?

On Tue, May 23, 2017 at 8:50 AM, therob84 notifications@github.com wrote:

Exactly the same problem I encounter with two of my contacts who use ChatSecure 2.0.8 (while I am using Conversations 1.18.4). All of us are using jabber.de (prosody >0.9.10)

  • At iOS devices Push is active and running (as ChatSecure me confirmed).
  • For about 1-2 days delivering messages to the iOS-devices works perfectly:
    • I compose a message (Conversations) while iOS-users are displayed as offline
    • triggered by the typing-notifier iOS-users are shown as "online" for some seconds
    • i send the message some time later, iOS again return to "online", receives the message, send an delivery-notifier - perfect!
  • But after 2-3 days this does not work anymore and the iOS user has to reopen ChatSecure to get Push activated again (which is not very userful for long-term conversations....), otherwise the message never will be delivered.

I think it is the same problem than stated above, but I could not provide any logfile but are eager to help solving this problem.

Is the reason now already undrestood or is it still a mystery, why/when/where this happens?

I hope the best for a fix in the next release....cheers, Robert

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ChatSecure/ChatSecure-iOS/issues/770#issuecomment-303442706, or mute the thread https://github.com/notifications/unsubscribe-auth/AAfqH-qkgi_JcevYYW7Sz54Mv7D6hIooks5r8wAvgaJpZM4NaW-w .

therob84 commented 7 years ago

Hm...I have to ask the guys from jabber.de. But I doubt that they would do it that often... Is there a workaround/solution if this is the problem? Is there any other idea to improve this?

chrisballinger commented 7 years ago

mod_cloud_notify should store push tokens in permanent storage so they persist across restarts

On Tue, May 23, 2017 at 11:28 AM, therob84 notifications@github.com wrote:

Hm...I have to ask the guys from jabber.de. But I doubt that they would do it that often... is there a workaround/solution if this is the problem? Is there any other idea to improve this?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ChatSecure/ChatSecure-iOS/issues/770#issuecomment-303490311, or mute the thread https://github.com/notifications/unsubscribe-auth/AAfqH9QAGPhqT25-bnNlda_XVyFbrSd-ks5r8yVEgaJpZM4NaW-w .

therob84 commented 7 years ago

@chrisballinger Anwser from jabber.de-guys are:

Any more suggestion about the problem from your side? For my information: is anywhere written how the ChatSecure-Push-Servers are working together with the iOS devices and with the jabber-servers? I can't understand how it works in principle until now...

Edit1: 5 days after the _Cloud_Notify_ module was updated at jabber.de it seems the situation is improved (I can reach 3 contacts reliable within the last 5 days) But it is too early to say anything for long-term-stability.

Edit2: 2 weeks later I can NOT confirm any success in reaching my iOS-contacts. Only one contact is reachable anymore. But I think this is connected to the problem that ChatSecure/push-service is not restarted automatically after "restartingOS/swiping CS away/kill CS"... Which is a very deadly drawback and makes CS nearly unusable by iOS-contacts which do not care that much about "always checking if CS is launched and running" ......... :-(

therob84 commented 7 years ago

Two more questions I want to raise:

--> Will CS (re-)establish a Push-Session in this scenarios or is there first an user activity needed (which would not be that relieable and practicable)?

chrisballinger commented 7 years ago

Both of those scenarios disable push until the app relauches due to limitations of the content-available pushes we use. There is discussion about alternative approaches but the development budget is very thin at the moment.

On Tue, May 30, 2017 at 1:09 PM, therob84 notifications@github.com wrote:

Two more questions I want to raise:

  • how does ChatSecures push-functionality behaves, after CS is activley "swiped out" in taskmanager or even forced killed?
  • Or if iOS is freshly restarted an ChatSecure is not yet launched manually by user interaction?

--> Will CS (re-)establish a Push-Session in this scenarios or is there first an user activity needed (which would not be that relieable and practicable)?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ChatSecure/ChatSecure-iOS/issues/770#issuecomment-304993353, or mute the thread https://github.com/notifications/unsubscribe-auth/AAfqH7PdOUjLI3bUwdWb5XoWsoDlLutiks5r_HeCgaJpZM4NaW-w .

mimi89999 commented 7 years ago

Zom-iOS has the same issues, right?

chrisballinger commented 7 years ago

yes

On Tue, May 30, 2017 at 1:39 PM, Michel Le Bihan notifications@github.com wrote:

Zom-iOS has the same issues, right?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ChatSecure/ChatSecure-iOS/issues/770#issuecomment-305001140, or mute the thread https://github.com/notifications/unsubscribe-auth/AAfqHyhECjrhBIyQV7U_egxAPZv-A_blks5r_H5mgaJpZM4NaW-w .

mimi89999 commented 7 years ago

But how does Facebook, Twitter, Snapchat, etc. work then? They don't do VoIP and users are still able to get notifications after swiping the app and after a [re]boot.

chrisballinger commented 7 years ago

They send your messages in plain text to Apple, which is unacceptable for an app like this.

On Wed, May 31, 2017 at 7:35 AM, Michel Le Bihan notifications@github.com wrote:

But how does Facebook, Twitter, Snapchat, etc. work then? They don't do VoIP and users are still able to get notifications after swiping the app and after a [re]boot.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ChatSecure/ChatSecure-iOS/issues/770#issuecomment-305206007, or mute the thread https://github.com/notifications/unsubscribe-auth/AAfqH6gdUMmsdYScWYdgqwSDbPMf7b-qks5r_XqXgaJpZM4NaW-w .

mimi89999 commented 7 years ago

Can't we send you have unread messages. Please open the app to see them or something similar then?

chrisballinger commented 7 years ago

It would require some additional server side logic / filtering because these events are sent very frequently, for instance, all typing notifications. Once the push event reaches our pubsub node we don't have enough information on whether or not to filter it. There was some discussion on extending this format, in some of the previous issues. I'm not saying the current design is perfect, but funding is tighter and the dev team is smaller, so we really have to focus on one thing at a time.

On Wed, May 31, 2017 at 11:00 AM, Michel Le Bihan notifications@github.com wrote:

Can't we send you have unread messages. Please open the app to see them or something similar then?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ChatSecure/ChatSecure-iOS/issues/770#issuecomment-305268354, or mute the thread https://github.com/notifications/unsubscribe-auth/AAfqHwZEdAjhnd_8Fy7r69Yxf2eImsWsks5r_arXgaJpZM4NaW-w .

mimi89999 commented 7 years ago

@chrisballinger It would be a huge UX improvement. Users won't need to remember not to swipe the app from recent or to launch it after [re]boot.

As for presence, status, etc. smacks does that very well. We could also check for the <body> tag in the stanza in the cloud_notify module and send a push message only if there is one.

mimi89999 commented 7 years ago

@chrisballinger Will it require a lot of work to implement those high priority push notifications in the app and on the node?

therob84 commented 7 years ago

I just edited my comment above, about only updating the Cloud_Notify module on the prosody-server (of jabber.de) can NOT solve the problem that ChatSecure/push-service is not restarted automatically after "restartingOS/swiping CS away/kill CS"... And thus is a very deadly drawback and makes CS nearly unusable by iOS-contacts which do not care that much about "always checking if CS is launched and running" ......... :-(

chrisballinger commented 7 years ago

@therob84 I have plans to improve this in an upcoming release. Going to take the approach of sending generic foreground "New Message" notifications and possibly "Someone is typing..." as well.

paskalito commented 7 years ago

@chrisballinger Hi that's sounds so great to me!! thank you! i wanted to post a bounty on this, but i somehow could not find the issue there.. - did you stop supporting bountysource? ( looks like the last push bounty didn't end well..) btw. is the project on gratipay (couldn't find it) or liberpay?

chrisballinger commented 7 years ago

@paskalito You can now set up a monthly donation via in-app purchase within the app. I also have PayPal and Bitcoin options if you'd like to give a little extra:

Thanks for your support!

therob84 commented 7 years ago

@chrisballinger: I saw your announcment for beta-testing 4.1.0 at twitter (with http-upload and inline images) - great !!!!!! Could you also include already some improvments about behaviour of push messages (after restart,kill-app etc.) in this release? Great work, chris!

chrisballinger commented 7 years ago

Those improvements are targeted for the v4.1.1 release

On Sun, Jun 18, 2017 at 4:03 AM, therob84 notifications@github.com wrote:

@chrisballinger https://github.com/chrisballinger: I saw your announcment for beta-testing 4.1.0 at twitter (with http-upload and inline images) - great !!!!!! Could you also include already some improvments about behaviour of push messages (after restart,kill-app etc.) in this release? Great work, chris!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ChatSecure/ChatSecure-iOS/issues/770#issuecomment-309270646, or mute the thread https://github.com/notifications/unsubscribe-auth/AAfqH3iuo3n2swdFNhW6FtMQrJWgygVlks5sFQQVgaJpZM4NaW-w .

chrisballinger commented 7 years ago

I have some unfortunate news. Without changes to the XEP-0357 spec and server implementations to send a message_type (typing, message, etc) to my pubsub node, this approach will not be feasible. I need to distinguish between typing notifications and messages with a body, otherwise you end up with a nightmare scenario where you get a constant flood of "New Message" notifications that don't correspond to actual messages AND can no longer receive the actual messages because the app does not reliably wake up in the background.

Even if I can filter out non-messages, it looks like the app is no longer waking in the background from silent pushes. Without those background wakeups, you won't be able to see any actual messages. I am wondering if the foreground pushes are somehow blocking the background content fetch from coming through.

It looks like the only way to really make this work reliably will be to add WebRTC for VoIP PushKit support.

Sorry everyone.

screen shot 2017-06-27 at 4 50 04 pm

afriedmanGlacier commented 7 years ago

@chrisballinger I am wondering if there might be other options. Ours works as one would expect, and without the spamminess, though I am pretty sure we do broke some of the XEP specifications in the process. We have control of the server, and some of our changes are server side, so maybe its not possible, but maybe worth talking through what we did to see if there are any other possible routes.

Here are kind of the basics of what we did if I remember correctly: 1) On the server, we use @weiss version of mod_push for eJabber, and modifed it to remove some of the notifications that aren't actual messages and we didn't need or want 2) We log out of ChatSecure whenever the app goes into the background (I do realize this could be a problem with MUC, but we were waiting until the upcoming MUC fixes to see what needs to happen there). I think Signal does this too. 3) Because we log out in the background, we can filter out some typing and other notifications based on whether a "buddy" is online or not. 4) We use the "New Message" non-silent approach and only get push messages when an actual message is sent.

I don't have time for details tonight, but probably tomorrow if you are interested. Let me know

chrisballinger commented 7 years ago

Yeah the main problem is that, by design, I don't control who is sending me what. There's really no way for me to enable this without horribly breaking the UX for many people. Even if I can filter out the typing notifications... I can't even seem to use content-fetch anymore when also sending foreground notifications, so you'll never see the message content until you open the app, which is even worse imo.

mimi89999 commented 7 years ago

@chrisballinger The server module could be easily updated to only send a message to the pubsub node if there is a body. It could also advertise the version, so that you could know if it is the updated module or not.

I saw your screenshot. Did you write code for that? If yes, where could I find it (or even better the assembled app since I can't really build for iOS on GNU/Linux)?

afriedmanGlacier commented 7 years ago

@chrisballinger ahhhhh, I forgot, we also don't do the fetch. Because we are receiving "New message" notification for every actual message, we decided we'd rather just open the application to see the message and have Push notifications working the way we'd expect than to worry about seeing the content in the notification. For us, this seems better than the alternative.

chrisballinger commented 7 years ago

@mimi89999 This technically doesn't require any client changes. Only on the server.. but the server only works for the App Store build of the app.

@afriedmanGlacier Yeah...

mimi89999 commented 7 years ago

So I won't be able to test anything :-(

I believe that changing the server module and having it send the version would solve the issue.

mimi89999 commented 7 years ago

And New Message is better than no notification at all.

chrisballinger commented 7 years ago

Hm yeah I guess I could make this behavior opt-in via a mod_cloud_notify patch and some experimental XEP-0357 spec extension for specifying the message type. I think it might still work with content fetch because my iOS 11 test device cannot seem to do silent push / content fetch at all anymore, so there must be something else going on as well.

therob84 commented 7 years ago

(Is there a connection with this topic?)

A friend of mine cannot connect to CS-Push-Service (red alert-sign in settings) with v4.1. He has the iOS "Background app refresh" ("Hintergrundaktualisierung") disabled (for battery reasons) - but claims that threema and whatsapp still work in delivery of push messages. With CS there is none push message delivered until now. The "reset" button in push menu brought no solution. A screenshot I will upload in a few minutes.

Do you know, where the error could be located and how to solve?

cringeops commented 7 years ago

@therob84 He has to switch on the background refresh. Pushes are rely on it.

therob84 commented 7 years ago

@hemee

My problem is that I have 1.) successfully convinced a friend of using xmpp/ChatSecure 2.) Now I have to tell him, that he should enable the background refresh for using it, which he costs battery, although WhatsApp and threeme don't need it (as it seems)....... bad chain of convincement unfortunately.... :-(

ageru commented 7 years ago

You are only activating Background data for ChatSecure, and that's only to allow it to receive stuff, not to send continuously.

On 4 July 2017 22:13:11 CEST, therob84 notifications@github.com wrote:

@hemee

  • is this the only reason/solution?
  • Can it only be allowed for CS?
  • If it would be switched on, how often CS will do anything in background?
  • Is it a battery eater or should there be no difference visible?
  • Why WhatsApp and Threema do work properly without switching it on?

My problem is that I have 1.) successfully convinced a friend of using xmpp/ChatSecure 2.) Now I have to tell him, that he should enable the background refresh for using it, which he costs battery, although WhatsApp and threeme don't need it (as it seems)....... bad chain of convincement unfortunately.... :-(

-- You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub: https://github.com/ChatSecure/ChatSecure-iOS/issues/770#issuecomment-312947019

therob84 commented 7 years ago
STPKITT commented 7 years ago

I'm not into Apple, but doesn't Apple provide a push-network like Google does with it's GCM? I always thought that was the case and that WhatsApp and similar apps use that push-service and therefor don't have the problems ChatSecure has to deal with.

chrisballinger commented 7 years ago

High priority notifications require either:

  1. Sending plaintext to Apple
  2. Supporting VoIP and abusing the API intended for call notifications

Right now neither of these are feasible solutions. I have been experimenting with sending generic "New Message!" pushes, but it seems to break the content fetch.

paskalito commented 7 years ago

@money : thank you, as i don't own anything made by apple, and this as an expression of a strong belive in free culture etc. thanks for still providing other donating options i think it's important! also you may want to have a look into http://gratipay.com/

@push : thanks for starting to work on that! sad it doesnt work with push AND content fetch. but i feel VERY stongly in the direction of @afriedmanGlacier and @mimi89999 that it should be a absolute priority to get any reliable push. and content fetch could be a nice extra. so if there is anything to opt in as a user (if i understood correctly that that was to discuss) IMHO it should be the content fetch function with a big warning that it will probably break push functionality.

so if it's possible the get push working with "new message" notification only. but reliably with any server that fullfilsh the push XEP. I beg you to integrate that into the next version of cs. and make that behavior default.

because that will all the ios users i know to actually start using the app. my experience with cs is i write to all my ios contacts and i never get any reply. - because they also never see the messages...

STPKITT commented 7 years ago

Why isn't push via VoIP API with PushKit a feasible solution when that's what WhatsApp uses?

stigger commented 7 years ago

Because WhatsApp supports VoIP calls and ChatSecure does not? You cannot just abuse the API, it won't get past Apple's review.

STPKITT commented 7 years ago

Ah that makes sense, I didn't consider Apple's policies.

stigger commented 7 years ago

By the way, from my experience the content-available push notifications are quite reliable. I've had other issues that indirectly affected the reliability (some push-related issues on the server-side, long processing of pushes due to #812, losing incoming messages due to #813), but once I resolved them, everything started working pretty smoothly.

So, it could be that the VoIP API is not really necessary after all. I suggest waiting until the above issues are resolved in master and checking if it helped.