Scheduled Calls and inbound call UI

[tomastiminskas]

Improvements on calls:

1. Show inbound call UI when call link message is received (even when the app is in background or closed)
2. Scheduled calls (recurring or not)

Issues:

• @jimBeaux27 already built the logic to show inbound call UI when app is running or in foreground (message received through socket). But when app is in background or closed the incoming message is not received through socket. The only event triggered is the push notification (didReceivePushNotification), but the message content is not available so we can't know if it's a call and we don't know the link of the call)
• Right now we don't have logic to handle scheduled calls.

Solution:

• Create a new message type (30=call)
• Message content could be similar to what we use on podcast clips of boost: call::{json} where json is something like:

{
    link: http://jitsi.sphinx.chat/asdasdas
    cron: 0/12/*/*/1 
    recurring: true
}

Cron reference: https://crontab.guru

• Push notification will start sending 2 new params from relay (right now it just sends chat_id inside custom_data)

  1. message_type
  2. message_content (it will be encrypted) but the app will receive it and will be able to decrypt it if needed

• App wakes up for around 30 seconds when the push is received so it can process it. If mesage_type param is 30, then it will decrypt the message_content coming in the push, and will run the inbound call UI using the call link in the message content.

• The cron and recurring keys in the message content json will allow to store information about scheduled calls (we will need designs from Milan to set up a scheduled call and to show scheduled received calls). When a scheduled call is received we can create local notifications to be sent X time before the day and time of the call as a reminder.

@pitoi @Evanfeenstra thoughts?

@jimBeaux27 it would be good to research if it's possible to run the inbound call UI on notification received event.

[uncleJim21]

@tomastiminskas not sure exactly what you mean. But I think you mean being able to run code when a notification is received. I think we would need to implement PushKit and then use the didReceiveIncomingPushWith: payload delegate method in the

AppDelegate so that we could initiate the JitsiIncomingCallManager and UI. I'll look more into this but it's challenging when I can't trigger push notifications whenever I want.

[tomastiminskas]

@jimBeaux27 I will jump on a call with @gonzaloaune tomorrow to see if we can get development push notifications working again. If so then you will be able to receive push while debugging from Xcode and test this. Will let you know

[uncleJim21]

actually wait on that I found this guide: https://medium.com/mindful-engineering/voice-over-internet-protocol-voip-801ee15c3722

Also this project looks promising for an experimental tool for sandboxed PNs: https://github.com/noodlewerk/NWPusher

Let me try this before using additional resources on it.

One thing we need to note is that PushKit notifications would be distinct from regular push notifications and allow more data. Let me do a bit more research.

[uncleJim21]

So I played around and it looks like I lack the ability to create the appropriate tokens for this. If I can get on a call with someone who can, I'd be able to install those then use the MacOS app called Pusher to generate arbitrary Push Notifications.

I can experiment with that then provide feedback on the spec & plan for what we need to do on the relays. What do you think @tomastiminskas ?

[tomastiminskas]

With this: "I lack the ability to create the appropriate tokens for this" you mean you need permissions to create certificates on Apple Developer program? Not sure what do you mean with tokens?

If so let me know what kind of tokens/certificates you need to create so I can check if I have the needed permissions

[uncleJim21]

Yea I need to be able to create a very specific type of certificate. See the Pusher app docs: " Log in to Apple's Dev Center Go to the Provisioning Portal or Certificates, Identifiers & Profiles Go to Certificates and create a Apple Push Notification service SSL From here on you will be guided through the certificate generation process. "

[tomastiminskas]

@jimBeaux27 I already created it. I think it's the one that expires on 2024/03/23. Check if you can download it and use it. If you can't then you might need to create one by yourself. If that's the case you will need to ask @pitoi to give you permissions to create certificates (same permissions level I have)

[tomastiminskas]

@jimBeaux27 the certificate I created is linked to the bundle id of Sphinx iOS app (com.gl.sphinx) so you will need to use our dev team and that bundle id to test with that certificate

[uncleJim21]

@tomastiminskas I can't download it. I also am not 100% convinced these are the exact same certificates. It seems like there are two varieties of PNs: APNs and PushKit Notifications. The latter being more realtime and data dense.

[tomastiminskas]

I created the exact certificate you mentioned above. If you can't download it, then it means you need to create it by yourself. So you will need to ask Paul for permissions on that

[uncleJim21]

Thanks @tomastiminskas

I went ahead and put together a basic flow chart for @tobi-bams @Evanfeenstra @gonzaloaune to peak at https://lucid.app/lucidchart/17b94b5a-0518-4b35-8562-fd86d48d4e13/edit?viewport_loc=-361%2C-588%2C3035%2C1310%2C0_0&invitationId=inv_7acd7f07-fb64-40a2-87a1-288ad4ca8466

Let me know if there are any questions. It looks like we may need a meeting to clarify smaller points but have a look and see if this answers some questions first.

[tomastiminskas]

RELAY CHANGES

• Add new message type 30 (call)
• Add new column for push_kit_token on contacts table
• Adapt updateContact endpoint to support new push_kit_token attribute
• Changes in sendMessage endpoint to support call type or new endpoint for it (right now it will always create message of type 0) call = true in the body
• On receiveMessage if it's type 30, then send new SNS notification (call HUB to handle it) with message content included

[uncleJim21]

Good summary @tomastiminskas one thing I'd add is that we want forward compatibility to be able to abstract the call to Hub to be based on a cron job (i.e. 30mins from now, recurring every wednesday at 3PM etc) versus just having it call hub immediately.

For first pass it's fine to just call immediately I just don't want to accidentally paint ourselves into a corner with the architecture approach so that the goal of scheduled calls is hard to accomplish.

[tomastiminskas]

@jimBeaux27 IMO we should use the new VoIP notification just for calls in 1:1 conversation (not scheduled, but created at the moment). I don't think it makes sense to receive a VoIP notification at the start time of a scheduled call.

For scheduled calls we can just build some logic on the client side to send reminders (local notifications) some time before the time of the call (even allow the user to customize those reminders). And new designs from Milan to show the scheduled calls in a chat or something like that and join them.

Considering that, we wouldn't need any changes on relay regarding those. The cron value on the json will stored the information about the schedule and clients will handle that to show it to the user and send reminder.

[uncleJim21]

@tomastiminskas I am going off of Paul's exact words "I want a bot to call me before my meeting comes up". But I also like your approach and personally feel that it solves the problem in almost the same way, perhaps superior.

The problem would still remain that the relay would most likely have to orchestrate some action to send the user reminders unless we assume all of the scheduled and such is handled client side. Curious to hear your alternate approach in more detail.

[uncleJim21]

Client -> Relay: Sends cron job, encrypted message (link_url), recurring (true/false)

Relay -> Hub: Device token, encrypted message (link_url), caller_name, message type (call or non call OR branch inside relay instead)

Hub -> AWS -> Apple -> Client caller_name, encrypted message (link_url)