Closed lazyadmin111 closed 7 years ago
Not a developer, but I can confirm the problem on my public XMPP server (yax.im):
Debug log from prosody:
Jan 03 17:40:34 yax.im:carbons debug Sending carbon to user@yax.im/chatsecureXXXX
Jan 03 17:40:34 c2sb659af0 debug Invoking cloud handle_notify_request for new smacks hibernated stanza...
Jan 03 17:40:34 yax.im:cloud_notify debug Sending push notification for user@yax.im to pubsub.chatsecure.org
Jan 03 17:40:34 stanzarouter debug Routing to remote...
Jan 03 17:40:34 s2sout3c93a10 debug going to send stanza to pubsub.chatsecure.org from yax.im
Jan 03 17:40:34 s2sout3c93a10 debug sending: <iq type='set' to='pubsub.chatsecure.org' from='user@yax.im' id='push'>
Jan 03 17:40:34 s2sout3c93a10 debug stanza sent over s2sout
Jan 03 17:40:34 s2sin53cbc60 debug Received[s2sin]: <iq id='push' type='result' to='user@yax.im' from='pubsub.chatsecure.org'>
This is the full IQ-set that's sent from yax.im to ChatSecure's server (token and node anonymized):
<iq type="set" to="pubsub.chatsecure.org" from="user@yax.im" id="push">
<pubsub xmlns="http://jabber.org/protocol/pubsub">
<publish node="A9BFF3A9-276C-46E2-****-************">
<item>
<notification xmlns="urn:xmpp:push:0">
<x xmlns="jabber:x:data" type="form">
<field type="hidden" var="FORM_TYPE">
<value>urn:xmpp:push:summary</value>
</field>
<field type="text-single" var="message-count">
<value>6</value>
</field>
<field type="text-single" var="pending-subscription-count"/>
<field type="jid-single" var="last-message-sender"/>
<field type="text-single" var="last-message-body"/>
</x>
</notification>
</item>
</publish>
<publish-options>
<x xmlns="jabber:x:data">
<field var="FORM_TYPE">
<value>http://jabber.org/protocol/pubsub#publish-options</value>
</field>
<field var="token">
<value>b86cd972bcfb466f************************</value>
</field>
<field var="endpoint">
<value>https://push.chatsecure.org/api/v1/messages/</value>
</field>
</x>
</publish-options>
</pubsub>
</iq>
I'm seeing this as well, not sure why it stopped working because we've made no changes in a long time. A quick look at our error logs shows no clear culprit, so I'll have to take a closer look. Let me know if you notice it working again.
Does the manual "Knock" feature work between two ChatSecure iOS clients?
Okay, so I tested again this afternoon and was able to get it to work using both Knock as well as XEP-0357 style pushes.
This might be OS throttling based on our use of content-available..
Were any of you using "low power mode"?
Upgraded both of my devices to beta 58. XEP-0357 notifications still don't arrive at the device.
The iPhone was attached to the charger all the time, low-power-mode is disabled.
Not quite sure how to trigger the "Knock" feature, IIRC it should appear instead of "Send" in the chat view for offline contacts? My secondary iOS device/account is offline, but all I see is a greyed out "Send" button.
In order for Knock to work between iOS clients, you need to establish an OTR session once in a while to exchange tokens, make sure you set it to "OTR" or "OMEMO & OTR", at least temporarily.
Just a short update: push still doesn't work on 4.0.1, and I wasn't able to see the "Knock" button despite having OTR configured on both devices.
@ge0rg
I've been testing this same issue a bit of push notifications working sporadically, and @chrisballinger I think you are right about it being due to use of content-available, making it a silent notification. I am using ChatSecure-Push-Server, and I modified the notification in push/messenger/views.py to remove the content_available=True argument, and I added the message text "New Message" just to see what would happen and, at least with initial testing, I now receive the push notification every time.
In order to avoid silent notifications and get the notifications every time, and maintain security, maybe need to use the "Modifying Payload of Remote Notifications" feature in iOS? This way, send a generic message that allows the app time to connect to the server, retrieve the new message, and change the text of the notification to that of the new message. See https://developer.apple.com/library/content/documentation/NetworkingInternet/Conceptual/RemoteNotificationsPG/ModifyingNotifications.html#//apple_ref/doc/uid/TP40008194-CH16-SW
I haven't gone down the road of trying this yet (and won't have time this week), and I'm guessing you guys are way ahead of me, but just in case it helps.
Unfortunately with the first method you would see a lot of generic "New Message" messages on the screen, because XMPP servers are a little spammy at the moment (it will push for things like typing notifications).
I think you might be on to something with the mutable notification thing, but it depends on how much time we'd get in the background. I didn't realize that you could make background network requests with mutable notifications, I was under the impression that you had to include the payload and decrypt it offline. Another snag is that it only works on iOS 10.
When we originally designed the push stuff, we considered leaving in an option for a generic "New Message" push, so you could attempt to force through background throttling, but it added too much complexity for the sending logic.
I am not sure if this is the right topic, maybe you can point me elsewhere if not.
In the current version (4.0.2) how is the a) delivery of offline-messages handled and b) in which way is ChatSecure informed about new messages while the app is not active in the foreground?
Background of the question: I use Conversations @Android with a friend using ChatSecure@iOS. I want to know, in which circumstances my messages won't be delivered at the friends iOS and if he should change any settings to improve the situation...
Thanks for answers or links...
@therob84
If you are using a server that supports XEP-0357, the server can trigger a "wakeup" notification that logs in the client and downloads any queued messages. This also depends on server support for offline queueing of course. The app also supports periodic background fetch, so even if you don't have push, the app may occasionally check for new messages on its own, but we can't control how frequently that occurs.
The number 1 thing you can do is to make sure your server supports XEP-0357 and offline queueing.
I'm using prosody 0.10 with the cloud_notify module. As far as I can tell, and after looking though these threads, my configuration is fine. My certificate is from Letsencrypt and is good.
With my Android clients, using Conversations, I can watch when they come online and can see push notifications go to those clients.
With Chatsecure-ios, using a brand new user account or an existing account, I see none of this push activity in the prosody debug logs. It's as if enabling "push notifications" on IOS doesn't do anything. After 5 minutes, smacks destroys the session due to inactivity from the Chatsecure Client and I can no longer send any messages to the device.
Is there something I should be looking out for to remedy this?
Thanks.
@Chrisballinger - thanks for fast feedback! Since we both using jabber.de XEP-0357 seems to be enabled. But I can't see any "received" marker nor "read" marker in conversations (thus I cannot check if messages are received and only ignored, or if the case of @davepb16 is valid). OMEMO is working!
So there is nothing I should tell the iOS-friend to change...?!
@davepb16 Does your cert include the client bit as well as server bit? See here: https://github.com/ChatSecure/ChatSecure-iOS/issues/557#issuecomment-247479188
You should also try enabling mod_offline
and mod_smacks_offline
.
@therob84 We don't support read receipts yet, only delivery receipts.
We should probably do a better job in the UI indicating whether or not your server works with push.
@chrisballinger
Certificate has both bits set... "X509v3 Extended Key Usage: TLS Web Server Authentication, TLS Web Client Authentication"
I have both smacks and smacks_offline set.
When a Conversations client (push capable) logs in to prosody I see: c2s1a6fd00 info Authenticated as user1@domain.co.uk .......... c2s1a6fd00 debug Received[c2s]: \<presence from='user1@domain.co.uk/phone'> .......... c2s12a3a18 debug Received[c2s]: \<iq id='12345abcd' type='set' to='push.siacs.eu' from='user1@domain.co.uk/phone'> .......... s2sout195ae60 debug sending: \<iq type='set' to='push.siacs.eu' from='user1@domain.co.uk/phone' id='12345abcd'>
With a Chatsecure-ios client I see: c2s1a6fd00 info Authenticated as user2@domain.co.uk .......... c2s1a6fd00 debug Received[c2s]: \<presence from='user2@domain.co.uk/Chatsecure12345'>
and nothing more about push notifications for the ios client.
There was an anomaly in my dns SRV record that I've just amended. A sub domain that wasn't included in my certificate.
I'm sure I had push working on Chatsecure-ios at one stage.
Tomorrow when I get hold of an iPhone I will set up a new user on prosody and use OTR only to see if it makes a difference.
Thanks for the advice.
Reporting back....
With prosody 0.10, Apple push notifications don't work. Tried new user account, OTR and OMEMO. No sign of push messages in log file. SSL certificate has both bits set and SRV records seem fine.
It also appears that on the iPhone, IOS 10.2.1, when the screen turns off, wifi gets disabled and Chatsecure fails to initiate another connection over the mobile data. It also fails to initiate a connection when the screen/wifi turns on. I have to open up the app to get Chatsecure to connect to the prosody server.
If you need any logs let me know.
What's curious is that it definitely works for me on my Prosody setup and v4.0.2 of the app. Below are all of the modules I have enabled.
I believe that these are all required for it to work reliably. Also make sure the "offline" is not disabled (it should be auto loaded).
"cloud_notify";
"pinger";
"smacks";
"smacks_offline";
Full:
modules_enabled = {
-- Generally required
"roster"; -- Allow users to have a roster. Recommended ;)
"saslauth"; -- Authentication for clients and servers. Recommended if you want to log in.
"tls"; -- Add support for secure TLS on c2s/s2s connections
"dialback"; -- s2s dialback support
"disco"; -- Service discovery
-- Not essential, but recommended
"private"; -- Private XML storage (for room bookmarks, etc.)
"vcard"; -- Allow users to set vCards
-- These are commented by default as they have a performance impact
--"privacy"; -- Support privacy lists
--"compression"; -- Stream compression
-- Nice to have
"version"; -- Replies to server version requests
"uptime"; -- Report how long server has been running
"time"; -- Let others know the time here on this server
"ping"; -- Replies to XMPP pings with pongs
"pep"; -- Enables users to publish their mood, activity, playing music and more
"pubsub";
"cloud_notify";
"pinger";
"smacks";
"smacks_offline";
"mam";
"csi";
"carbons";
--"register"; -- Allow users to register on this server using a client and change passwords
-- Admin interfaces
--"admin_adhoc"; -- Allows administration via an XMPP client that supports ad-hoc commands
--"admin_telnet"; -- Opens telnet console interface on localhost port 5582
-- HTTP modules
--"bosh"; -- Enable BOSH clients, aka "Jabber over HTTP"
--"http_files"; -- Serve static files from a directory over HTTP
--"http_files"; -- Serve static files from a directory over HTTP
-- Other specific functionality
"posix"; -- POSIX functionality, sends server to background, enables syslog, etc.
--"groups"; -- Shared roster support
--"announce"; -- Send announcement to all online users
--"welcome"; -- Welcome users who register accounts
--"watchregistrations"; -- Alert admins of registrations
--"motd"; -- Send a message to users when they log in
--"legacyauth"; -- Legacy authentication. Only used by some old clients and bots.
};
@chrisballinger
I've got all of those modules enabled apart from "carbons" and "mam".
I have taken a 5 minute snapshot of Prosody debug logs showing the iPhone client authenticating, and then getting disconnected as the mod_smacks hibernation timeout was reached (300 seconds). Nothing is seen from it again, even if the iPhone screen is turned on.
I then added a "conversations.im" XMPP account to both Android client and iPhone client (in conjunction with my Prosody accounts) just to see if it was an issue with my Prosody setup or Certificate setup.
I noticed that when the Prosody smacks_hibernation_time expired, and I lost complete contact with that ios account, I could type into the "online" conversations.im message window and the Prosody account would come "online" again.
The smacks_hibernation_time on the "conversations.im" must be about 30 minutes as it remained "online" a lot longer than my Prosody account.
Now the smacks_hibernation_time has expired on the "conversations.im" server as both XMPP accounts now show offline and I can't wake either up, by typing or sending a message. I'm guessing this means that Push notifications aren't being send from the "conversations.im" server to the iPhone either.
The only thing I can do now is to try to get different SSL certificates, try ejabberd or setup Prosody again. I'm sure Push notifications were working in Chatsecure 4.0 and my Prosody server.
:)
debug logs (identities removed): http://pastebin.com/Sdqv5shF
@davepb16 Wait so you're seeing absolutely NO logs on your server about push registration?. Set your Prosody log level to debug and look for mod_cloud_notify
logs here:
https://hg.prosody.im/prosody-modules/file/tip/mod_cloud_notify/mod_cloud_notify.lua#l26
If you aren't seeing any of those logs at all then there must be something wrong with the client that's preventing it from registering.
@chrisballinger
I definitely get push registration for my Android clients. Just none of my iPhone clients. The iPhone clients are on different networks, one is an iPhone 5s and one's an iPad, both are on ios 10.0.2.1
My logs are already set to debug to try to find out the issue here.
Here's a typical log of push registration messages of one of my Android clients after it authenticates.
@davepb16 Ok at least that narrows down the problem to the iOS client. The things that could cause this are:
From the logs it looks like maybe (5) is the case?
Also can you try testing iOS to iOS and verifying if the "Knock" feature works? In order to test you must establish an OTR session and then the other party needs to go fully offline. It will replace the Send button with a Knock button when no text is in the field. In this case you'll probably need to manually logout one side so the stream management timer doesn't waste your time.
@chrisballinger
I'm still not ruling out a problem at my end with prosody/cloud_notify/ssl cert. I might try to setup a new prosody server (or maybe ejabberd) with new certificates on a different machine and see.
Are there any particular SRV DNS settings (for my domain) that Chatsecure needs to work with push and prosody?
I will try the 'knock' soon, when I get the two Apple devices together.
Thanks.
I'm pretty sure it is a client issue because you should see cloud_notify registration in your logs. In a future version we will make it easier to see if you're registered for our push API, and if your server is advertising support.
I can confirm @davepb16 issue. I triggered the issue by reinstalling the app due to omemo issues. After that push never started working again. Issue #634 should give a ruff estimate of the time frame when it got broken.
@mt403 I'm going to work on a better server capabilities UI to help diagnose this issue. I have a feeling that the push API account is possibly in a bad state so the app thinks that push is unavailable.
Deleting and reinstalling the app resets push permissions.. so we might be on to something.
@chrisballinger Reinstalling the app does not enable push atm. I've tried all betas since the 4.0 release. I always select the enable push option after account creation. The ios settings dialog for chatsecure only lists background something, should there be more options ?
Ah yes that's the problem. For some reason we must not be re-prompting for push permissions, so it's never able to finish. This is what it should look like:
@chrisballinger Settings look the same for me. I was not specific enough, i always did reinstall the app in my tests. Shouldn't re asking for push permissions be fine in that case ? Because i thought removing the app would als remove all settings.
I believe the behavior differs in different versions of iOS, but generally when you hit "Enable Push" in the onboarding it should prompt you with the system message for allowing push notifications. And then if that doesn't work for some reason, we will show an "Enable Push" row in the Settings view of our app.
I'm going to focus on https://github.com/ChatSecure/ChatSecure-iOS/issues/684 because I think it will help us debug the underlying cause. Are you signed up for the TestFlight beta?
I can't recall any iOS prompts, only the cs onboarding. I also dont have a push option inside the app. Its supposed to be in settings accessed by the cog wheel right ?
Yes I'm already a beta tester. Testing with an iPad 4 with the latest iOS available.
On 9 Feb 2017 19:18, "Chris Ballinger" notifications@github.com wrote:
I believe the behavior differs in different versions of iOS, but generally when you hit "Enable Push" in the onboarding it should prompt you with the system message for allowing push notifications. And then if that doesn't work for some reason, we will show an "Enable Push" row in the Settings view of our app.
I'm going to focus on #684 https://github.com/ChatSecure/ChatSecure-iOS/issues/684 because I think it will help us debug the underlying cause. Are you signed up for the TestFlight beta?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ChatSecure/ChatSecure-iOS/issues/637#issuecomment-278726991, or mute the thread https://github.com/notifications/unsubscribe-auth/AOLk7-ep7CvKvenveCrwC0j7xGtH6WhQks5ra1iNgaJpZM4LWLsB .
The app also supports periodic background fetch, so even if you don't have push, the app may occasionally check for new messages on its own, but we can't control how frequently that occurs. The number 1 thing you can do is to make sure your server supports XEP-0357 and offline queueing.
I shortly want to come back to my question: "What can I tell to my non-techy friend about checking/changing any settings in ChatSecure or iOS itself in order to receive also my messages after she did not used ChatSecure actively for one week?" (both jabber.de with XEP-0357 enabled, I am using Conversations).
Since I never saw all the settings of ChatSecure or iOS I don't know, if any critical toggles exist, in order to enable relieable delivery of messages to my ChatSecure-Friend (just saw your screenshots above). At the moment this is unfortunatley not the case - as she did not receive any message after our initially successfull OMEMO-conversation (at least there was no anwser and no "delivery" markers in Conversations without these problems with Conversations-Friends).
"Enable Push" was skipped in the onboarding
Is this anything my friend should have manually set during installation of ChatSecure? How to change afterwards?
The app also supports periodic background fetch, so even if you don't have push, the app may occasionally check for new messages on its own, but we can't control how frequently that occurs.
What does it mean that you can't control it? Its an iOS thing? Will it happen once a minute or once a year? Mysterious statement...
...Never thought that I would have to deal with iOS-problems just to bring XMPP+OMEMO also to my friends with iOS :-/ While it is running very easy with Conversations... I hope the best for some improvements in this topic within the next updates and nevertheless I am still very thankful about the great progress @chrisballinger is acheiving with OMEMO-Integration.
@therob84 I am working on a server features browser that will also include some helpful info for diagnosing and fixing issues with push. Ideally everything should "just work" if your friend did "Enable Push" in the onboarding, but there may be edge cases that are hard to diagnose remotely until I get this view into the app.
What does it mean that you can't control it? Its an iOS thing? Will it happen once a minute or once a year? Mysterious statement...
Here's an overview of the background fetch APIs and their limitations. We use background fetch and "content-available" push notifications, which are both not 100% reliable and throttled by the OS. Conversations doesn't have to deal with any of this because you can still disable doze and use a long lived socket.
We use background fetch and "content-available" push notifications, which are both not 100% reliable and throttled by the OS.
Thanks for the link! Also this means, that these features could also not be available with iOS at versions prior to 7. Which feature ChatSecure uses in this case for "Push" etc. (which I think is true for my friends device)?
@therob84 We use "content-available" push notifications which have existed since iOS 7. However the oldest version of iOS we support is 8.0, so your friends device must have 8 or higher if they were also doing OMEMO.
@chrisballinger just curious, are you planning to move away from "content-available" as part of resolving this ticket since the operating system seems to be throttling silent push messages?
@afriedmanGlacier Yes, but it's not an easy problem to solve. Here are the options:
@chrisballinger
After ChatSecure registers with the Apple Push Notification server does it pass a device token back to the xmpp (prosody) server?
If I look in my prosody mysql database I can see mod_cloud_notify has these tokens for all of my Android clients but none for ChatSecure ios clients. Looking though my debugs logs, ChatSecure-ios doesn't seem to return them either.
@davepb16 Here is a description of how "level 2" push works. Instead of using the device token directly, we have users fetch time-limited tokens to reduce identifiable metadata. The flow goes like this:
push.chatsecure.org/api/v1/account
)push.chatsecure.org/api/v1/device/apns
)push.chatsecure.org/api/v1/tokens
)push.chatsecure.org/api/v1/messages
) and pubsub endpoint (pubsub.chatsecure.org
) is sent to XMPP serverpubsub.chatsecure.org
, with the Knock token and Push API Endpoint, which then does a POST to push.chatsecure.org/api/v1/messages
with the knock token.If the Knock tokens / Push API endpoint aren't being sent to the XMPP server, then there must be a problem with the 1st thru 4th step. I am creating a new view to help people debug these problems here: https://github.com/ChatSecure/ChatSecure-iOS/issues/684#issuecomment-280112514
@chrisballinger
Thanks. I remember this working initially in ChatSecure 4.0.0, using prosody 0.10. I could type/send message to an ios client (using OTR) and wake it up. I never kept debug logs though.
Can't remember when it stopped working but I'm guessing after ChatSecure 4.0.1.
As a point of interest, if I create/use xmpp accounts for "Conversations.im" (ejabberd server), I have some success waking up ios ChatSecure client from the Android conversations app by typing a message to it.
Part of the push notification system must still be working.
Looking forward to #684
@davepb16 Are you saying if you use ChatSecure to connect to an account created on the Conversations.im server, that push notifications to ChatSecure work?
About 90% of the time yes.
I can start to type from Android "conversations.im" account to ChatSecure-ios "conversations.im" account and wake it up. It also wakes up any prosody accounts that are also on ChatSecure-ios client.
I only have issues with Prosody servers and Chatsecure-ios. I haven't tried many other XMPP servers though.
I did try dukgo.com and push didn't work. I think dukgo.com is using prosody 0.9 though. conversations.im, which works, uses ejabberd server.
I know other people in this thread tried more servers.
dukgo doesn't have push support and likely never will. My personal server is running Prosody 0.9.x with mod_cloud_notify and works very reliably with push.
ok, I think I will have to wipe my server and start again from scratch. Maybe I will try with prosody 0.9.x instead of 0.10. Or maybe even ejabberd.
Thanks for your help.
@davepb16 Are you on the beta? Instead of wiping your server now, wait until next week, I will push a beta with #684 because maybe that can diagnose it.
Not on beta, no. I will wait before I wipe the server. Not in a rush to do it :)
@chrisballinger
Problem fixed...
I setup Prosody 0.9.12 using the same Letsencrypt certificates, almost identical configuration files and a new mysql prosody database and push now works as expected.
My logs are now full of push.chatsecure.org comments and "pubsub.chatsecure.org<E123456AB-7EE4-4220-8083-123456ABCDEF" now fills my mysql database for mod_cloud_notify entries.
I guess this issue is with prosody 0.10 running on ArchLinux.
If needed I can setup a broken prosody 0.10 again for testing purposes.
Thanks for your help.
@davepb16 Oh that's awesome! It seems like something in 0.10 was preventing mod_cloud_notify from loading correctly? My hunch is that the server wasn't reporting its 0357 capabilities properly, so we never attempted to register.
@davepb16 It would be cool, if you could setup a broken prosody 0.10 to see where the fault is on prosody's side. I'm also running a prosody 0.10 server and seeing this issue with a friend who I'm trying to convince to use XMPP. Only seeing messages after opening the app is a show-stopper, though. I'd rather see a potential issue fixed in prosody 0.10 than downgrade to 0.9.12. Many thanks!
Dear Developers,
I installed your nice app at some friends, and I am using it with a own xmpp server (prosody 0.10) + Push-module (cloud_notify). Some of my friends a while back also got some push notifications, so it seemed to work somehow a while ago. But now there aren't any push notifications any more. I wonder now if the push server is down (as my prosody logs seem to indicate, as they say something of "not found"), or if is rather my fault, e.g. because he the push app server doesn't like my self-signed certificate. But if the later, why did it work then sometimes a while ago?
Any helpful comment highly appreciated :-)
greets