(Desktop) UI to connect Mobile to desktop application

[sumairasaeed]

Create UI that allows the user to connect to the desktop application.

[sumairasaeed]

Discussed this with sojharo on how Android makes connection with desktop app. Found that we are using socket for communicating. This is an issue for iOS because on iOS socket is not open always. It is closed by iOS some time after app goes in background. On KiboChat mobile, we had solved this issue by using Push notifications. But Desktop app cannot send push to mobile as it is not communicating with server. I am looking into any alternate solution we can have. Will update on it in a while.

[sumairasaeed]

Thought more on this. Here is the solution how we can have a persistant socket connection:

1- On server, we will maintain a history which devices are connected to Desktop KiboChat through socket. If mobile app goes in background and socket gets disconnected after 5-10 minutes, server will get to know. At that time, server will send a silent push notification to mobile. Mobile will receive silent push(or voip push) and reconnect its socket. We will have to experiment further on this if connecting socket again will keep it connected for next 5-10 minutes or it will be immediately disconnected as app was still in background.

Another solution i have thought of is this: 2- Desktop app will communicate to Mobile App using Push notifications. Desktop app will send data to mobile app using push notification. Discussed this with @sojharo if it will be possible for desktop application to send push notification to mobile app. He said he will have to investigate it further if this is possible.

[jekram]

I want one design for both Android and iOS. @sojharo Please review this and come up with one design that will for both iOS and Android.

@sumairasaeed There are lots of task that run in Back groud in iOS so why we cannot use the same technique.

[sojharo]

@sumairasaeed please post the links here that you have reviewing to know how background service work in iOS. Also please post the links that you reviewed for this task.

[sumairasaeed]

Here are some of the links reviewed: https://developer.apple.com/library/content/documentation/NetworkingInternet/Conceptual/NetworkingTopics/Articles/UsingSocketsandSocketStreams.html

https://developer.apple.com/library/content/documentation/iPhone/Conceptual/iPhoneOSProgrammingGuide/BackgroundExecution/BackgroundExecution.html

https://www.reddit.com/r/iOSProgramming/comments/1a8ogr/strategies_for_an_ios_messaging_app/ http://stackoverflow.com/questions/24468116/how-to-keep-iphone-ios-websocket-connection-alive-while-in-the-background https://www.whatsapp.com/faq/web/28080002

On Mon, May 1, 2017 at 12:53 PM, Sojharo notifications@github.com wrote:

@sumairasaeed https://github.com/sumairasaeed please post the links here that you have reviewing to know how background service work in iOS. Also please post the links that you reviewed for this task.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Cloudkibo/iOS/issues/581#issuecomment-298301210, or mute the thread https://github.com/notifications/unsubscribe-auth/AKbhp5ZzmBD1YlV6H1omXMYSyVb0T0-5ks5r1Y9_gaJpZM4M7IOL .

[sojharo]

iOS app can run in background, we will need to either add a voip, audio or location tag to info.plist (that is UIBackgroundModes). This was discussed in this answer:

http://stackoverflow.com/questions/8261135/how-to-keep-iphone-ios-xmpp-connection-alive-while-in-the-background/11022682#11022682

I am not sure if we had done this on iOS or not. Looks like this is the way our app can run in the background on iOS. I have read few articles and I am reading more to understand the problem.

The issue is not with use of socket as we can see in the above link. The people using xmpp are also having this problem. So if we replace our socket with something different, the issue will remain on iOS. The real problem here is to keep the app running in the background so that it could get message from socket.io (in our case) or xmpp (in case of above link)

[sojharo]

I have reviewed the articles and also thought more on this. The use of push notification is not suitable option as push notification has data limit of 200 KB and it doesn't support binary. The chat done on desktop application will possibly include big text messages and image/video attachments. This can't be handled using push notification.

I think it would be better to use socket.io and make it run in the background so that it avoids connection drops. The solution that I have found in above comment should be tried. This solution can let background tasks run in iOS. This is the standard way to allow communication apps run in background for iOS. The communication apps i.e. which do calls or messages, need to be alive to listen to incoming calls or alerts.

[sumairasaeed]

Worked on UI and logic for connecting to desktop application. App gets connected to Desktop app using socket. Mobile app takes ID displayed on Desktop app as input from user. Followed same flow as on Android and app joins room successfully. We have already done settings and added tag of 'Voip' into our app. Next will try backgrounding solution as discussed in link provided above.

On Mon, May 1, 2017 at 5:52 PM, Sojharo notifications@github.com wrote:

I have reviewed the articles and also thought more on this. The use of push notification is not suitable option as push notification has data limit of 200 KB and it doesn't support binary. The chat done on desktop application will possibly include big text messages and image/video attachments. This can't be handled using push notification.

I think it would be better to use socket.io and make it run in the background so that it avoids connection drops. The solution that I have found in above comment should be tried. This solution can let background tasks run in iOS. This is the standard way to allow communication apps run in background for iOS. The communication apps i.e. which do calls or messages, need to be alive to listen to incoming calls or alerts.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Cloudkibo/iOS/issues/581#issuecomment-298329031, or mute the thread https://github.com/notifications/unsubscribe-auth/AKbhp-Wy_Pyu0XE-O5ZbMyT7Hu2BXfyYks5r1dWIgaJpZM4M7IOL .

[jekram]

I will review and get back to both of you. I may not happen today.

[sumairasaeed]

i have read more on 581 and looked at solution provided by sojharo. Solution provided in above comment by sojharo talks of adding a 'voip' tag in our app. On this, I have found a link that it is no longer supported on iOS 10 http://stackoverflow.com/questions/8261135/how-to-keep-iphone-ios-xmpp-connection-alive-while-in-the-background/11022682#11022682" It recommends to use voip push to bring app awake again . However, i also found that azure services donot support VOIP type push notification.Need to think more on this. I will further try and implement 'voip' tag and test it on iOS 9 and 10 and see difference in behavior (if any)

[jekram]

How are we avioding iOS ruiing in background when we are doing the instal? or back up?

Why we cannot send a push notification to keep the iOS app from going into background.

[jekram]

https://developer.apple.com/library/content/documentation/iPhone/Conceptual/iPhoneOSProgrammingGuide/BackgroundExecution/BackgroundExecution.html

Review this: https://github.com/socketio/socket.io-client-swift/issues/627

[jekram]

https://www.quora.com/How-do-I-keep-Socket-IO-running-in-the-background-on-iOS

[sumairasaeed]

Yes sir. I have already reviewed these articles. Apple allows certain type of background modes only for apps with specific purpose. Example: news apps, apps which track location, apps which plays audio in background, apps which needs to fetch some data at regular intervals(we used this for taking backup), apps which need to make phonecalls over internet(voip) so needs stable connectivity.

Out of these, most suitable for us was Voip and register our socket with voip enabled. I tried this and found that socket was alive for 5-10 minutes in background. However, in future Apple might remove and replace it completely with PushKit-based architecture

Here is the article about it: https://developer.apple.com/library/content/qa/qa1938/_index.html#//apple_ref/doc/uid/DTS40017564

Article from Apple clearly says:

The legacy VoIP architecture was replaced by a new PushKit-based architecture in iOS 8. It was then formally deprecated with the iOS 9 SDK. In iOS 10 it is only available as a compatibility measure; it continues to work (as well as it ever did) if your app is linked with an old SDK, but is disabled if you link with the iOS 10 SDK.

For the moment you can work around this by building your app with Xcode 7. However, this is not a good long-term solution because:

1. Our experience is that PushKit-based VoIP apps are more reliable and more power efficient than those using the legacy VoIP architecture.
2. Apple always recommends that you use the latest version of Xcode because it combines the best features and the best compatibility.
3. Specifically, we encourage VoIP apps to take advantage of CallKit, a new framework in the iOS 10 SDK that radically improves the user experience for VoIP apps. Xcode 8 is the only supported way to use CallKit.
4. Also, be aware that Xcode 7 is not supported on macOS 10.12 Sierra. 5. At some point support for the legacy VoIP architecture will be removed, whereupon all VoIP apps will have to move to the new PushKit-based VoIP architecture. It’s better to start this work sooner rather than later.

Important: This move away from the legacy VoIP architecture is motivated by specific technical concerns. Our experience is that the legacy architecture is unreliable and, when it does work, has a strong negative effect on standby battery life. The new PushKit-based architecture addresses both of these concerns.

Solution to implement: In order to communicate with Dektop app, when iOS is in background, Desktop app should send a Voip Push notification. This push will awake the application and then application can get required data by connecting to socket. I am not sure if it is possible for Desktop app to send Voip notification as it currently has no link to server. @sojharo can comment on this better what will be needed for desktop app to send Voip push to mobile app to make it active.

Note: Voip pushs are different from standard push and Voip push are currently not supported by azure hub.

Comparison of Voip and standard push:

1. Unlike standard push notifications, which the user must respond to before your app can perform an action, VoIP pushes go straight to your app for processing.
2. VoIP pushes are considered high-priority notifications and are delivered without delay.
3. VoIP pushes can include more data than what is provided with standard push notifications.
4. Your app is automatically relaunched if it’s not running when a VoIP push is received.
5. Your app is given runtime to process a push, even if your app is operating in the background.
6. For regular remote notifications, the maximum size is 4KB (4096 bytes)
7. For Voice over Internet Protocol (VoIP) notifications, the maximum size is 5KB (5120 bytes)

[sumairasaeed]

As part of this task, 1- We will configure Voip push notifications. We will have to generate Apple certificates for receiving Voip push and generate provisioning profiles to be integrated in Xcode. 2- Also we will code to send and receive data to and from desktop app when in background.

Dependencies: https://github.com/Cloudkibo/KiboChatDesk/issues/39 https://github.com/Cloudkibo/KiboChatDesk/issues/40

[jekram]

I am very confused what we are doing on this task and what is the next step.

Send me the design document that we are trying to implement. This is URGENT

[sumairasaeed]

Here is a design document: https://docs.google.com/document/d/1Ob0ZGe_RQOxvo33nLGqtsr-nYgNw_nCoC9vYS8g6R-A/edit

[jekram]

Have we done enough investigation to make sure this wold work? If so proceed.

[sumairasaeed]

Yes sir. Yesterday I did work on this. Generated Voip push certificate for our app and configured it on Xcode for proof of concept. Captured device token and using an online testing tool, I sent a sample Voip push notification. It was correctly received by mobile, application got awake and socket got connected in background. This means that it would work for us.

[sumairasaeed]

Need to spend some time on this today. Need to generate required keys/tokens from certificate to provide to server so server can configure voip push. working on it

[sumairasaeed]

Generated certificate files, keys and token required by @sojharo for server-side work on https://github.com/Cloudkibo/Cloudkibo/issues/511 to configure Voip push . Uploaded required certificate and key files on google drive : https://drive.google.com/drive/u/1/folders/0B3lStktK2xjya2xES0hmWTN5R1U

[jekram]

what is the next step here?

[sumairasaeed]

Next, will work to receive data from desktop when app awakes ,once server-side tasks are done : https://github.com/Cloudkibo/KiboChatDesk#40 https://github.com/Cloudkibo/Cloudkibo/issues/511

[jekram]

What is the ETA for these task?

[sumairasaeed]

Completed work on this. Need to test it with @sojharo