Failing push notification

ageru commented 7 years ago

Hi,

Sorry to create yet another bug about push...

Setup

Server: prosody .9.12 with CA-signed TLS certificate
Relevant modules: cloud_notify, smacks, pinger, carbons, mam, csi, filter_chatstates, throttle_presence.
IPhone SE with latest iOS. Low battery mode off, all is ON in General Settings > ChatSecure.
Chatsecure 4.0.8

Steps to reproduce

Install Chatsecure, log to server
Send a couple of messages to user on iPhone, they reach him, OMEMO works... Everything looks clear.
Confirmed push is active in the app's interface
Leave the app.

Actual result

Quickly, the iPhone user shows as away on other devices, then as offline.
Messages don't reach the device unless the iPhone's user starts the app
The iPhone's user has to specifically open the app to get the sent messages
Relevant portions of the server log:
- Successfully initiating push (it seems): May 14 HH:MM:SS [alphanum characters] info Push notifications enabled for [ID]@[chatdomain.tld]/chatsecure (pubsub.chatsecure.org<A3486BB3-6442-4945-84E2-[more alphanum characters])
- When trying to send a message to offline device (the last number increments with every try of the server): May 14 HH:MM:SS [chatdomain.tld]:cloud_notify info Got error of type 'cancel' (not-allowed) for identifier 'pubsub.chatsecure.org<A3486BB3-6442-4945-84E2-[alphanum characters]': error count for this identifier is now at 1

Expected Result If iPhone is connected to the Internet, the iPhone user gets a notification straight away.

Bonus questions

What does mod_pinger do? I can make an educated guess looking at the name and bits of its code, but I'm not 100% certain.
Would it be possible to get a wiki of FAQ detailing exactly what modules in prosody are needed to use all the features of ChatSecure? @chrisballinger gives a few times his config in threads here and there, but that's hardly discoverable and probably not comprehensive or up to date.

Many thanks.

mimi89999 commented 7 years ago

@chrisballinger Any news on implementing what I proposed?

chrisballinger commented 7 years ago

Yes I've made some progress, but I'm still catching up after returning from a break

On Wed, Jul 26, 2017 at 12:08 AM, Michel Le Bihan notifications@github.com wrote:

@chrisballinger https://github.com/chrisballinger Any news on implementing what I proposed?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ChatSecure/ChatSecure-iOS/issues/770#issuecomment-317963770, or mute the thread https://github.com/notifications/unsubscribe-auth/AAfqH-MxRMCAF94RMIMNU7D4PSVgqN2yks5sRuCFgaJpZM4NaW-w .

mimi89999 commented 7 years ago

@chrisballinger I applied your changes to mod_cloud_notify from https://groups.google.com/forum/#!topic/prosody-dev/Ge3USXmuMIc and after resolving an issue with connecting to the pubsub node (#822) I tested it.

I am very happy with how it works. When I send a message and CS is not running in the background, I get almost instantly the New Message! notification on my iPad with another account. It works even after rebooting the device. Thanks, to the priority, I don't get flooded with notifications. I get only one notification per message when I send the message. I haven't tested it in real conditions yet tho, but I expect it to work well since it worked now.

chrisballinger commented 7 years ago

@mimi89999 Aww yeah! Glad you like it.

MattRiddell commented 7 years ago

Yep, +1 from me. Same thing, with that patch to mod_cloud_notify it works fine.

I had some problems with OTR not working after resumption (Adium says: The following message was not encrypted: OTR Error: Encryption Error), but OMEMO seems to work fine.

chrisballinger commented 7 years ago

@MattRiddell That's because OTR sucks on mobile and there's really nothing we can do to fix it other than move everyone over to OMEMO. :)

mightyBroccoli commented 7 years ago

I am using prosody 0.10 nightly and some of my user notably these form iphone are not able to get push notifications.

Mostly they get token after token and the app will always say cloud push is not possible. @chrisballinger Are there any steps I need to complete as a hoster to fully support push for iphone users?

chrisballinger commented 7 years ago

@mightyBroccoli Are you running mod_cloud_notify?

ageru commented 7 years ago

I've lost my iPhone users as well for the past few days, but noticed it today. I'm still on Prosody 0.9 though.

In the token interface of ChatSecure (4.1.0), they get something along the lines of 'this server doesn't support the functionality'.

Could it be something with the token server? I'm asking because mod_cloud_notify hasn't been updated for weeks, so that doesn't leave much else that could have changed and provoked the issue.

On 11 August 2017 19:56:57 CEST, nico notifications@github.com wrote:

I am using prosody 0.10 nightly and some of my user notably these form iphone are not able to get push notifications.

Mostly they get token after token and the app will always say cloud push is not possible. @chrisballinger Are there any steps I need to complete as a hoster to fully support push for iphone users?

-- You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub: https://github.com/ChatSecure/ChatSecure-iOS/issues/770#issuecomment-321878965

MattRiddell commented 7 years ago

Just a heads up Chris - I'm using your custom mod_notify from the google group and am receiving notifications on my iphone fine.

chrisballinger commented 7 years ago

Ok thats good news. Debugging other people's server configurations is probably my least favorite part of developing this app. Not very much positive feedback these days. Sigh

On Fri, Aug 11, 2017 at 11:33 AM, Matthew Riddell notifications@github.com wrote:

Just a heads up Chris - I'm using your custom mod_notify from the google group and am receiving notifications on my iphone fine.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ChatSecure/ChatSecure-iOS/issues/770#issuecomment-321887376, or mute the thread https://github.com/notifications/unsubscribe-auth/AAfqH-ZzSyJyW1OsCj3WPA3sZFQMF9Hnks5sXJ51gaJpZM4NaW-w .

mightyBroccoli commented 7 years ago

@chrisballinger yes I am using mod_cloud_notify. Is there anything I can do to help iPhone users use push?

mimi89999 commented 7 years ago

@mightyBroccoli Be sure that mod_cloud_notify works correctly and that you are using the latest versions of modules... Also, https://bitbucket.org/chrisballinger/prosody-modules/branches/compare/12b4387..b1c5b0c#diff should greatly improve experience. It works very well for me (or at least it did when I tried several days ago).

@chrisballinger You probably can't do much about server setup... What were the feedbacks?

mightyBroccoli commented 7 years ago

@chrisballinger i will try your cloud_push changes. I think if they really work that well why don't you push them to the prosody community modules hg?

chrisballinger commented 7 years ago

@nico because it involves changes to the xep-0357 spec, and i still have some other ideas to improve it before standardization

On Sat, Aug 12, 2017 at 12:56 AM, nico notifications@github.com wrote:

@chrisballinger https://github.com/chrisballinger i will try your cloud_push changes. I think if they really work that well why don't you push them to the prosody community modules hg?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ChatSecure/ChatSecure-iOS/issues/770#issuecomment-321964863, or mute the thread https://github.com/notifications/unsubscribe-auth/AAfqH3d6vdfrb_j774k92-138_jU75Nxks5sXVqhgaJpZM4NaW-w .

Echolon commented 7 years ago

@chrisballinger @MattRiddell Hey, my provider (jabber.de) has this mod_notify activated for a while already. Doesn't work for me so far - sorry. Their apps will be set to doye or however it works on iOS :( (Conversations to ChatSecure, both at jabber.de)

EDIT (19-Aug-2017): Jabber.de seems to work now as they updated mod_cloud_notify

mimi89999 commented 7 years ago

@Echolon But they probably don't have the changes...

MattRiddell commented 7 years ago

Yep exactly.

I actually was trying to get everything working with ejabberd but saw this thread and did a 100% fresh install of prosody from source with that new version on Debian 8.0 and it worked flawlessly.

Echolon commented 7 years ago

Okay, thank you. I'm going to check this.

ageru commented 7 years ago

Sorted my latest problem:

applied Google Group/BitBucket patch to mod_cloud_notify
relaunched prosody
added hostname to chatsecure config

And my iOS users seem to be back in business! Problem seem to have started after I renewed the Let's Encrypt cert for the domain. I found pubsub and cloud_notify errors that seem related in my logs, I can post them if anyone's interested.

Thanks!

On 13 August 2017 22:35:51 CEST, Echolon notifications@github.com wrote:

Okay, thank you. I'm going to check this.

-- You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub: https://github.com/ChatSecure/ChatSecure-iOS/issues/770#issuecomment-322065862

Wiuma commented 7 years ago

Modified mod_cloud_notify ( BitBucket) for prosody and push works like a charm.

Edit: The only problem ist that "New Messages" Info comes without sound.

Echolon commented 7 years ago

www.jabber.de seems to work now as they updated mod_cloud_notify

therob84 commented 7 years ago

www.jabber.de seems to work now as they updated mod_cloud_notify

Could be the result that I sent them yesterday an e-mail and they instantly updated the package for their prosody server.

But my question is: what "seems to work" now exactly better than before? What is now different than 3 days before and how did you check this and is it dependend on CS version?

Echolon commented 7 years ago

@therob84 https://twitter.com/jabberde/status/898584861316833282

They said they updated the unofficial changes from mod_cloud_notify. Before they only ran prosody stable as I understood. What works better? My iOS ChatSecure contacts now receives push. before they didn't. All current CS version. (I'm afraid one guy now receives some kinda ghost push notifications, but thats better than nothing :D)

mimi89999 commented 7 years ago

@chrisballinger When the app isn't running, the "New message" push messages show on my devices, but there isn't any sound. If I am not using the device when it comes, I don't know about it.

ageru commented 7 years ago

I'm like you @mimi89999 : my iPhone users are there but, they still don't receive push :( Not sure what I am doing wrong...

Prosody tells me this, which sound good:

Aug 19 15:29:04 c2s41b621c0 info Push notifications enabled for user@domain.tld/chatsecure (pubsub.chatsecure.org<A3486BB3-6442-4945-84E2-7736229C8CF5)

But then I get this every so often:

Aug 19 16:14:20 [domain.tld]:cloud_notify info Got error of type 'cancel' (not-allowed) for identifier 'pubsub.chatsecure.org<A3486BB3-6442-4945-84E2-7736229C8CF5': error count for this identifier is now at 3 And sometimes it all seems to all go awry:

Aug 17 10:47:52 [domain.tld]:cloud_notify    info    Got error of type 'cancel' (not-allowed) for identifier 'pubsub.chatsecure.org<0584D5A1-9939-484C-A92A-A4E5B4EF0560': error count for this identifier is now at 85
Aug 17 10:47:52 [domain.tld]:cloud_notify    warn    Disabling push notifications for identifier 'pubsub.chatsecure.org<0584D5A1-9939-484C-A92A-A4E5B4EF0560'
Aug 17 10:47:52 mod_c2s error   Traceback[c2s]: ...er/modulesenabled/mod_cloud_notify/mod_cloud_notify.lua:80: attempt to index a nil value 
stack traceback:
        ...er/modules-enabled/mod_cloud_notify/mod_cloud_notify.lua:80: in function <...er/modules-enabled/mod_cloud_notify/mod_cloud_notify.lua:62>
        /usr/lib64/prosody/util/events.lua:67: in function </usr/lib64/prosody/util/events.lua:63>
        /usr/lib64/prosody/util/events.lua:67: in function 'fire_event'
        /usr/lib64/prosody/core/stanza_router.lua:187: in function 'core_post_stanza'
        /usr/lib64/prosody/core/stanza_router.lua:205: in function 'core_route_stanza'
        /usr/lib64/prosody/core/stanza_router.lua:219: in function 'core_route_stanza'
        /usr/lib64/prosody/core/stanza_router.lua:191: in function 'send'
        ...er/modules-enabled/mod_cloud_notify/mod_cloud_notify.lua:268: in function 'handle_notify_request'
        ...er/modules-enabled/mod_cloud_notify/mod_cloud_notify.lua:286: in function <...er/modules-enabled/mod_cloud_notify/mod_cloud_notify.lua:284>
        /usr/lib64/prosody/util/events.lua:67: in function 'fire_event'
        ...
        /usr/lib64/prosody/util/xmppstream.lua:255: in function 'feed'
        /usr/lib64/prosody/modules/mod_c2s.lua:233: in function 'data'
        /usr/lib64/prosody/modules/mod_c2s.lua:255: in function 'onincoming'
        /usr/lib64/prosody/net/server_event.lua:636: in function </usr/lib64/prosody/net/server_event.lua:582>
        [C]: in function 'loop'
        /usr/lib64/prosody/net/server_event.lua:797: in function </usr/lib64/prosody/net/server_event.lua:796>
        [C]: in function 'xpcall'
        /usr/lib64/prosody/../../bin/prosody:376: in function 'loop'
        /usr/lib64/prosody/../../bin/prosody:407: in main chunk
        [C]: at 0x4ea4f8e690

I double-check the patch I applied to cloud-notify, but I don't see any mistake.

If anyone has an idea what's going on, I would be grateful for the help.

mimi89999 commented 7 years ago

The errors you get are most likely because the token has expired and you haven't got a new one.

As for that error with the trace: It looks like there is a bug in the code handling errors from CS pubsub push server.

ageru commented 7 years ago

Thanks for your insight!

The token errors are strange: checking the token interface in CS, I never see it run out. There are always a few unused token.

I haven't pinned down yet when the trace error happens. I'll try to see with my users if it's linked to coming back online, picture transfer, or something else.

mimi89999 commented 7 years ago

@ageru It is exactly here: https://bitbucket.org/chrisballinger/prosody-modules/src/12b4387c0991eb24a15677b8bf59184ed880fa40/mod_cloud_notify/mod_cloud_notify.lua?fileviewer=file-view-default#mod_cloud_notify.lua-80

Just after module:log("warn", "Disabling push notifications for identifier '%s'", push_identifier); https://bitbucket.org/chrisballinger/prosody-modules/src/12b4387c0991eb24a15677b8bf59184ed880fa40/mod_cloud_notify/mod_cloud_notify.lua?fileviewer=file-view-default#mod_cloud_notify.lua-78

I never got such a token error (or am not aware of it)

tmolitor-stud-tu commented 7 years ago

The backtrace was caused by a bug I fixed a few minutes ago. You can get the latest version of mod_cloud_notify here: https://modules.prosody.im/mod_cloud_notify.html

ageru commented 7 years ago

Many thanks for this @tmolitor-stud-tu and @mimi89999 ! I have pulled and up-ed the change from Mercurial in the last hour. No more backtrace so far, I'll keep an eye on my logs.

Following that, I have re-implemented the "Google Group" changes manually in that updated mod_cloud_notify.lua fill. Still no push notification for iPhone users, and I still get the "cancel" error. I had a closer look at the file, and mine seem very different from the "Google Group" one. Chris did remark that he is working from an old version of Cloud Notify, which must be way too different from mine for the patch to actually do its deed. Oh well.

mimi89999 commented 7 years ago

@ageru Try that: https://lebihan.pl/files/mod_cloud_notify.lua I haven't tested it yet, but should be OK...

Loader23 commented 7 years ago

The file is working, but still no sound when the app isn't running :-( Iam also having another problem: Got Conversations and ChatSecure running > closing both apps > receive a message. I see the "New Message" Notification on my iPad but when I open the App, I do not receive anything. Start Conversations again and no message either. And the sender gets "Delivery failed". Any Ideas on that?

ageru commented 7 years ago

@mimi89999 That's very kind of you to have taken the time, thank you :)

I have been using your file for a few hours now. I don't have my test iPhone at hand, but from what my iOS users tell me, they still don't get push notification. And the "cancel" message still shows up regularly in my Prosody logs. But otherwise it doesn't seem to hurt. I'll try to test this more thoroughly in the coming days, see if I can find out something more.

mimi89999 commented 7 years ago

@Loader23 I also don't have any sound for the notifications. As for the other issue, are you using any e2e encryption?

@ageru Maybe your users should try to register again with push? Under server info, under push tokens they should choose reset.

chrisballinger commented 7 years ago

I see why people aren't getting sounds but still are receiving "New Message!" notifications. Should be an easy fix without requiring a client update, hopefully can push out tomorrow if have time.

On Wed, Aug 23, 2017 at 2:58 PM, Michel Le Bihan notifications@github.com wrote:

@Loader23 https://github.com/loader23 I also don't have any sound for the notifications. As for the other issue, are you using any e2e encryption?

@ageru https://github.com/ageru Maybe your users should try to register again with push? Under server info, under push tokens they should choose reset.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ChatSecure/ChatSecure-iOS/issues/770#issuecomment-324474504, or mute the thread https://github.com/notifications/unsubscribe-auth/AAfqH4LdUUSu8aADzspbWvJOifUZVLndks5sbKCRgaJpZM4NaW-w .

Loader23 commented 7 years ago

@mimi89999 Iam using OMEMO. All Fingerprints are accepted. I guess the time between the Client change was too short. But I have ignore_presence_priority = true and carbons in my config for this.

@chrisballinger That sounds good. When are you going to release 4.1.1 for iOS? Iam also affected by https://github.com/ChatSecure/ChatSecure-iOS/issues/777 and an update would be nice.

chrisballinger commented 7 years ago

@Loader23 Probably by week of Sep 10-16, beta week before

On Wed, Aug 23, 2017 at 11:12 PM, Loader23 notifications@github.com wrote:

@mimi89999 https://github.com/mimi89999 Iam using OMEMO. All Fingerprints are accepted. I guess the time between the Client change was too short. But I have ignore_presence_priority = true and carbons in my config for this.

@chrisballinger https://github.com/chrisballinger That sounds good. When are you going to release 4.1.1 for iOS? Iam also affected by #777 https://github.com/ChatSecure/ChatSecure-iOS/issues/777 and an update would be nice.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ChatSecure/ChatSecure-iOS/issues/770#issuecomment-324542980, or mute the thread https://github.com/notifications/unsubscribe-auth/AAfqH4qjijsmSvgeX4TWea9jj1HHRZQ9ks5sbRQ2gaJpZM4NaW-w .

Loader23 commented 7 years ago

@chrisballinger Great :-) Please give short Feedback if you have fixed the notification Sound bug ;-)

ageru commented 7 years ago

Got finally the possibility to test properly, apologies for the delay.

@mimi89999 : the backtrace has not shown up since I applied your fix. Thanks :)

Client side, I reset Push, and the client seem to receive notifications when the device's screen is turned on and unlocked. Is that the expected behaviour at this stage, or should the device receive notification also when the device is locked/screen turned off?

Server side, I still get the "cancel" error, possibly for every message sent or receive through Chatsecure, considering how much of those I see. Also, Chatsecure seems to registers to receive push notification every time the app is opened by the user. It shows as this fairly often in the logs as this:

Aug 26 22:27:36 c2s41ee6888     info    Authenticated as user@domain.tld
Aug 26 22:27:36 c2s41ee6888     info    Push notifications enabled for user@domain.tld/chatsecure (pubsub.chatsecure.org<A3486BB3-6442-4945-84E2-7776229C5BF5)

Again, not sure if all this is expected behaviour or not.

I'll keep looking for hints in the coming days.

danielreuther commented 6 years ago

As I understand from the discussions, there is currently a working version of mod_cloud_notify for Prosody. And according to https://github.com/ChatSecure/ChatSecure-iOS/issues/637#issuecomment-287409197, I'm assuming that @weiss may start working on a similar solution once it things have been standardized.

If my understanding is correct, is there anything I can do in the meantime? I'm running the mod_push version from @weiss's repository.

Loader23 commented 6 years ago

@mimi89999 I found out, that carbon message on iPad is working fine, as long as the app is open in the background. Is this intended?

@danielreuther With the mod_cloud_notify from mimi89999 push is working fine with app in background.

danielreuther commented 6 years ago

@Loader23 Right, but it's a Prosody module. Hence my question about ejabberd -- I'm currently assuming that mod_push would require similar modifications and I'm afraid those won't happen before the adjustments found their way into the spec.

mimi89999 commented 6 years ago

@chrisballinger Did you solve the issue with the notifications being silent?

Loader23 commented 6 years ago

@chrisballinger or could you tell us the year of the "tomorrow" you are going to fix it?

tmolitor-stud-tu commented 6 years ago

@Loader23 this is just unfair. Push on iOS is very complicated and frankly: if you want a platform that doesn't make life so hard for its developers just switch to Android.

Loader23 commented 6 years ago

@tmolitor-stud-tu That has nothing to do with being unfair. It is ok if he did not have the time to fix it yet, but if he says "tomorrow" some feedback of how its going would have been nice after more than two weeks.

Wiuma commented 6 years ago

"New Message" info works for me with sound since 2 days.

THX @chrisballinger

weiss commented 6 years ago

@Loader23, maybe just cut down your payment for his development work next time he fails to meet deadlines?

Loader23 commented 6 years ago

@weiss You don't get it either, maybe you just read what i wrote again until you understand it ;-)

ChatSecure / ChatSecure-iOS

Failing push notification #770