Chat message status

[sojharo]

To make visibility, we would mark all chat messages as one of the following:

1. sent
2. delivered
3. seen

I would create design here first and then implement.

[jekram]

Thanks. Once the design is agreed upon please put it in GitHub.

[sojharo]

After thinking carefully on it and looking at both server and client code, I have come up with following simple design. For each chat message record, we would have following two new fields:

1. Unique message id
2. status (values are 'sent', 'delivered', 'seen')

Now:

1) For 'sent', everytime the client sends the message, the server would send acknowledgement with message id. We would then mark the message as 'sent' on both server and client.

2) Everytime message is received by recipient device, it will send acknowledgement to server. Server would mark the message as 'delivered' and send the message status to sender too.

2.1) Case # 1, the sender is online and quickly gets the chat message status 'delivered' using socket.io

2.2) Case # 2, the sender is offline and gets the chat message status later using sync.

2.3) Case # 1, the one way recipient would receive the message is in real-time using socket.io. He will quickly send the acknowledgement

2.4) Case # 2, the other way recipient would receive the message is using sync. He would be online so he would use socket.io to let server know that the message is delivered.

3) Finally, the status of 'seen', the client application would send this status when the user reads the new chat messages.

3.1) Case # 1, user reads these messages when he is online, the status of 'seen' will be sent in real-time using socket.io

3.2) Case # 2, user reads these messages when he is offline, the status of 'seen' will be sent when the sync from client to server would be done i.e. in #121

All these cases are critical and need to be coded with care. I would implement the server side code first in this and then I would add more here on what server expects from clients.

the point 3.2 is dependent on #121 and would be done only when #121 is done

[sojharo]

We have not reviewed it but I have started the work on server side for this however. It is not yet complete. This task would be little bigger as it as many use cases.

[jekram]

@sojharo

I think I understand it, but I am little confused. As we are mixing two things:

1) Guaranteed Delivery 2) Delivery Notification

In this logic above we do not address how we are going to Guarantee delivery on the message itself. Delivery notification is secondary.

I am also confused between what is 2.3 and 2.4 they sound like 2.1 and 2.2?

Please draw a flow chart on the Guarantee Delivery first and show how a message would go from sender to server (sender is online, and the sender is offline and what is the retry logic). Second how the message would go from server to recipient (when the recipient is online and when the recipient is offline).

Then we can overlay Delivery notification on top of it.

I would prefer a flow chart as it leaves no ambiguity,

[sojharo]

Salaam,

2.1 and 2.2 are for sender case. 2.3 and 2.4 are for receiver case.

I could not understand the Guaranteed delivery. As long as recipient doesn't connect to Internet, we can't guarantee delivery. What if the user deletes the application and never connects to Internet again. We can show that message was delivered only when recipient connects and synchronization happens. Delivery notification will only be sent to sender when recipient has done the sync and got the unread messages. I think by guarantee you mean when user actually reads the message. For this, we have 'seen' status for the message.

I would draw the diagram and send it to you by tomorrow morning.

[jekram]

This is what I mean by Guarantee Delivery and we should not confuse it Delivery Notification.

Messaging by definition is store and forward. By store and forward it mean that on each hop of the message I will store and forward it. It means on each hop it is the responsibility of that note to successfully forward it to next node. This continues until the message reaches its final destination.

Take the example of non web email. I can send the message from my outlook when there is no network. Outlook will keep the message until the network is restored. At the time the message would by sent to the server for my email. More then likely my server node is different from the server node of the recipient. So the email message is passed from one node to another. On each hope it is stored and forward. In this process if the network connection is broken between two nodes then it will sit there until network connection comes back. The final node is the destination client.

Each node takes the responsibly of keeping and copy delivering it to the next node.

The same thing happens with SMS. If I send an SMS to you it will go through x number of SMS nodes. It may take few seconds to deliver it or two weeks if your phone was off.

By Guarantee Delivery it means the network guarantees it will be delivered.

In our case it is quite simple. We have a three node network. Sender, server, and receiver.

In our case we have a two hop network.our design should be that on each hop we have to guarantee message forwarding.

So on hop one if the network is there we deliver the message to the server. If the network is not there then we deliver the message when the network is restored.

Similarly on hop two the message is delivered to recipient right away. However, if the network is not there then the message needs to be delivered when the network is restored.

The key thing it is the responsibility of the sender to deliver it to next node.

You never get notification on email or SMS if it got delivered or not.

Now coming to notification this is information what the current status of your message is.

We can get away without doing notification like email and SMS. We cannot get away without Guarantee Delivery.

In fact you would need Guarantee Delivery to send the status notification.

[sojharo]

Yes, I have understood it fully now.

When sender sends message, it is stored on phone and sent whenever network is available. For this, we have #121

When message is sent to server but server could not send it to receiver, for this we have #120

These two tasks guarantee the delivery. #120 task is done and #121 is not started yet. This task is for message status. i.e. on which node the message is i.e. on sender node, or on server node or on receiver node.

To keep the status user friendly, we call it sent, delivered and seen. I hope it clarifies this and also other two tasks #120 and #121

[jekram]

Thanks.

[sojharo]

Here are the required diagrams for this issue:

Use Case 1

Use Case 2

Use Case 3

[jekram]

@sojharo

Good job with the design.

Let's go ahead and implement this in both Android and iOS

@sumairasaeed

[sojharo]

I have completed the server side work in this. The server code is ready and we can now work on android and iOS clients. I have also documented the server side code on the following github repo link:

https://github.com/Cloudkibo/CloudKibo/blob/master/cloudkibo_documentation/chatstatusdoc.md

@sumairasaeed should take help from this documentation to understand what server expects from the client in order to update with message status. This documentation would help in doing the work on client side.

Action Items: 1- Sumaira should clean the server side database of cloudkibo because now we have new chat message structure and old messages which are stored on server will not work. 2 - We all should remove our local sqlite databases from our devices or altogether remove the applications and install the next version of applications which have this chat status feature. 3- Next, in this item I would work on android side.

[sumairasaeed]

ok Thanks

On Wed, Jun 22, 2016 at 12:21 AM, Sojharo notifications@github.com wrote:

I have completed the server side work in this. The server code is ready and we can now work on android and iOS clients. I have also documented the server side code on the following github repo link:

https://github.com/Cloudkibo/CloudKibo/blob/master/cloudkibo_documentation/chatstatusdoc.md

@sumairasaeed https://github.com/sumairasaeed should take help from this documentation to understand what server expects from the client in order to update with message status. This documentation would help in doing the work on client side.

Action Items: 1- Sumaira should clean the server side database of cloudkibo because now we have new chat message structure and old messages which are stored on server will not work. 2 - We all should remove our local sqlite databases from our devices or altogether remove the applications and install the next version of applications which have this chat status feature. 3- Next, in this item I would work on android side.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Cloudkibo/Android/issues/122#issuecomment-227544113, or mute the thread https://github.com/notifications/unsubscribe/AKbhp9gHbztMhLssf5UrQNc-xN4j19WDks5qODnSgaJpZM4I3bl9 .

[jekram]

@sojharo Thanks for putting the documentation in the Repo

[sojharo]

This is under construction, I have worked on it for case of sending chat messages where we send the message and get the acknowledgement from server. The other two cases in this are remaining.

[sumairasaeed]

@sojharo , I was working on this task for iOS. Your documentation says that server will send back acknowledgement after it successfully receives chat message. I am sending chat message to server, it also gets printed in logs that server is getting chat message. But i don't get any acknowledgement back. Are you getting acknowledgement on Android?

[sojharo]

Yes, I do get the acknowledgement on Android. Acknowledgement is just single JSON object, kindly handle it this way. Try parsing JSON from it. It is little bit different from the acknowledgement logic that we used in conference where we had multiple parameters. I tested it again.

[sojharo]

I have worked on it for part of delivered. It still needs some work as on sync it is not sending the delivered status to server.

[jekram]

Ok. Thnanks for the update

[sojharo]

In this I worked more and now the sync on delivered is also done.

Moreover, the third case, the status of "seen" is also updated in real time. This task is 95% complete now.

The only thing remaining is "when user sees the message during offline, how should we let the sender know that message is seen" this sync is remaining now.

The next version of app is 23 which would be available in 2 hours.

[jekram]

Thanks for the update. Let me know when 23 is available

[sojharo]

The remaining work in this is also completed now. And when user sees the message during offline, we also update the sender by sync when user is online again. For this, we maintain a separate table in sqlite to store which messages were seen when the user was offline.

[jekram]

Thanks