Contact list incorrect

[jekram]

In the latest version, the contact list is incorrect.

It is showing only three contacts while I have much more.

1, Screenshot #1 only three shown on contact list

2. Screenshot # 2 AJawaid AEkram 425-435-4422 is shown on cloud kibo, and then we are asking to be invited. I am already chatting with him

[dayemsiddiqui]

@jekram after digging through the logs I found this:

You can see that the server is correctly identifying AJawaid AEkram 425-435-4422 as a contact on cloudkibo, so from this we can deduce that the problem is on client side, but the strange thing is that we havent changed a single line of code which is responsible of adding contacts in db in this release plus it also works fine for me, I suspect this behavior is being triggered by a very specific set of conditions. I am still debugging, the best way would be if I can somehow replicate this issue on my end, that would give me more insight. I am still debugging, I will keep updating the progress.

[jekram]

Thanks. In the previous version we had two contact list. We remove d one of them.

[jekram]

@dayemsiddiqui What is the update here?

[jekram]

I think I know part of the problem.

This issue has to do if we have a duplicate entry then we are not handling it. I have Yasmeen Ekram & Iesh both have 425-503-5617.

Both of them are in the address book.

In Kibochat I do not see Iesh.

I see Yasmeen Ekram correctly in Chat list but incorrectly in contact list.

This is quite comman so we need to fix it.

Q: If there are duplicates which one we should show? Should we not show both? What is the implication?

Q: If we have to show only one then we should show the once we have done chat on.

Let's think trough the implications .

[jekram]

@sojharo @dayemsiddiqui Please review this and let's come up with a solution. I am not sure if this is the only problem.

@sumairasaeed I have not seen this issue on iOS. How are you handling duplicate entry? Same Phone Number with two or more different names.

[sumairasaeed]

Sir, I show both the contacts in "Contacts" tab and "Favourites" tab even if they have same phone number. On "chats" page it shows one of them. I will check how it picks one. Also if we try to start new chat with other similar phone contact, it recognises it and shows old chat which was done with the contact with similar phone previously.

On Mon, Dec 12, 2016 at 10:05 AM, Cloudkibo notifications@github.com wrote:

@sojharo https://github.com/sojharo @dayemsiddiqui https://github.com/dayemsiddiqui Please review this and let's come up with a solution. I am not sure if this is the only problem.

@sumairasaeed https://github.com/sumairasaeed I have not seen this issue on iOS. How are you handling duplicate entry? Same Phone Number with two or more different names.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Cloudkibo/Android/issues/348#issuecomment-266344191, or mute the thread https://github.com/notifications/unsubscribe-auth/AKbhpyMkflOLZ_gJa2iD1GWa1uBtkUqMks5rHNYdgaJpZM4LJsH0 .

[sojharo]

G sir I will discuss with dayem and sumaira. I am avoiding duplicate phone numbers on first setup.

[sumairasaeed]

@sojharo:

1. testuser100@cloudkibo.com
2. testuser500@cloudkibo.com
3. testuser1000@cloudkibo.com

With 100, 500, 1000 contacts.

The password for these accounts are cloudkibo

On Mon, Dec 12, 2016 at 10:59 AM, Sojharo notifications@github.com wrote:

G sir I will discuss with dayem and sumaira. I am avoiding duplicate phone numbers on first setup.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Cloudkibo/Android/issues/348#issuecomment-266349478, or mute the thread https://github.com/notifications/unsubscribe-auth/AKbhpxqgXoVLCMQo-Q6kcRA22s27SeTAks5rHOKsgaJpZM4LJsH0 .

[sojharo]

@sumairasaeed Thank you.

[dayemsiddiqui]

@jekram oh now I see, actually in chatlist I have declared phone numbers as unqiue in order to maintain database integrity so that no two contacts can have the same number. Since we identify contacts mainly using phone number multiple records with same number can create ambiguity. I will discuss this issue with sojharo on how we can cope with this situation.

[dayemsiddiqui]

Also we should look at how whatsapp behave in such a situation and try to replicate the same behaviour in our app.

[sojharo]

I am also able to reproduce this problem. Looks like the recent changes have broken the code. My current install did not show those contacts which are on cloudkibo. Thus, the chat list doesn't show any name. It worked before.

[dayemsiddiqui]

Ok let me check at my end

[sojharo]

One more thing to notice. It happened on my QMobile only and not on other samsung device. I had factory reseted the device and then installed the app. It showed some contacts from SIM only in the address book at that time. KiboChat application showed these as not on cloudkibo (and they were not on cloudkibo). Later, I connected google account to it. It started to fill my address book with contacts that I have on google address book. It contains phone numbers which are on cloudkibo. When I reinstalled the app now, it only fetched my SIM card contacts and not google contacts. However, it was able to read them all.

This is my initial test and analysis on this. I would do more analysis.

[sojharo]

One more thing, error only happens when we do factory reset or try on a brand new phone. It is not happening again when I reinstall app without doing the factory reset

[dayemsiddiqui]

@sojharo I downloaded the latest version of kibochat and tested it on different phones seems to be working fine for me. (Note I dont have contacts with same number)

[dayemsiddiqui]

@sojharo I tried the steps and the problem is being replicated. Two of my contacts which are already on cloudkibo are showing invite button in contactlist

[sojharo]

Also one more thing, the contacts are shown in create group page but not on contacts page.

[dayemsiddiqui]

Yes I also noted the same thing, this issue is occuring under a set of very specific conditions i.e factory reset + google contacts sync. One thing that we can conclude is that for some reason the contacts are either not getting properly synced or they are not getting properly added in the local database.

[jekram]

Looks like both of you have made significant progress is understanding the issue better.

[dayemsiddiqui]

@sojharo take a look at this:

The two contacts marked are on cloudkibo and they show perfectly fine in my other phone. But when I do a factory reset and sync google contacts, these contacts start showing the invite button however their status still remains to "I am on CloudKibo", however its on_cloudkibo variable is false. Now this should not happen because in our code whenever we assign the status "I am on CloudKibo" to a contact we always make its on_cloudkibo variable true. It means that during google contact sync our database gets changed and this is causing some database integrity issue.

[jekram]

I do not understand when you say "google contact sync". Our source should be the "address book" and not google or outlook. When Google or Outlook is updated it makes into the "Andriod Address Book" and that should be our source.

[dayemsiddiqui]

@jekram @sojharo

I think I have narrowed down the problem to the cause. As @jekram mentioned earlier, the problem is basically duplicate contacts. Take a look at this

I have duplicate contacts in my contact list one is stored in SIM while other is in google contacts. The problem is when we sync the contacts, the code tries to enter duplicate phone numbers in the database, but since we have applied database constraint of keeping phone number unique the second entry fails.

There are two things we can do:

• Replace the existing record with the new record if the phone number already exists, such that status variables like on_cloudkibo can be updated by the insert. (This would still show only one of the two contacts)

• Second thing we can do is instead of making phone number unique, we can use of composite key of phone_number and display_name and restrict their combination to be unique. (This will display multiple contacts that have the same number but are saved in address book with different names). However if we implement this we would lose the guarantee that only one record will be fetched by a unique phone number from the database. Since our code is designed in such a way that it expects only one record per phone number, implementing this strategy could cause potential bugs in the code.

[sojharo]

To make it simple, I had not made any field primary key in any table initially. I just checked that phone field in contact list is made unique. Let us discuss before doing the design changes. Let us test with version 88 and then we would remove primary key.

[jekram]

This issue can be reproduced on my end and by @dayemsiddiqui and maybe @sojharo

So it is best we do a step by step trace to find where the issue is. We need to determine where the problem is before coming with solutions.

[jekram]

@dayemsiddiqui @sojharo

I would like to understand better before making changes

A couple of questions:

Example Let's Phone 123 has two entry A & B. He is going a chat with A.

Is the data stored under "A" or 123?

[sojharo]

@dayemsiddiqui and I have discussed the problem and solution with each other. Dayem is assigning this issue to himself. Also, we discussed that here we have two issues: one is because of duplicate contact and other is that wrong number of contacts are shown in list. We have discussed the solution. Dayem would write the solution here.

[dayemsiddiqui]

@jekram I had a discussion with @sojharo here is the points we discussed and decisions we made:

• Problem : Some contacts that are already on cloudkibo are not being showed properly in contactlist. They appear as contacts that are not on cloudkibo.

Possible Reason: The issue may be caused by the fact that we are preventing duplicate entries in the database.

Solution: We would still prevent duplicate entries in the database however when an entry for a pre existing phone number comes instead of adding a record with a duplicate phone number we will update the record so it is properly displayed in the contact list.

Note that we will not allow two contacts having same phone number and different names, because it creates ambiguity.

Consider the following case. I have saved sojharo's number with two different names sojharo1 and sojharo2, if sojharo sends me a message on kibochat how would I know in which chat window should I show the message as both of them have the same phone number.

This resolves the point number #2 that you mentioned in the first comment.

As for point number #1 I am not sure if that is the consequence of the above problems that we discussed or something else. I am going to implement the changes right now and push the changes. You can test it on your end and check if it resolves the first issue or not.

[jekram]

Thanks to both of you. Of thinking through.

Let's keep it simple. In out contact list there should be only one entry. Like you said if we have Sojharo1 and Sojharo2 then we cannot know which one to display if we keep both.

I do not understand in your solution how do you know which one you have used previously all the time. If I remove and reinstall the app we would not know which one was used previously.

Go ahead and implement it. It will probably cover most of the use case but not 100%

[dayemsiddiqui]

@jekram I have implemented the changes, according to my discussion with @sojharo I am testing it on my end, after testing I will push the changes.

[jekram]

I just checked Whatsapp on what they do on a duplicate. They follow a very simple model, and they do not allow duplicate. I had one number in two in my address book. In WhatsApp, only one of them shows up.

So in Whatsapp, there is one number and one name.

In my address book, both contacts have Whatsapp entry, and if I use it from there, it maps to just one name.

[jekram]

I just checked Whatsapp on what they do on a duplicate. They follow a very simple model, and they do not allow duplicate. I had one number in two in my address book. In WhatsApp, only one of them shows up.

So in Whatsapp, there is one number and one name.

In my address book, both contacts have Whatsapp entry, and if I use it from there, it maps to just one name

[dayemsiddiqui]

@jekram we have decided to follow the same model as whatsapp. Also I have pushed the code.

[jekram]

Great

[jekram]

@dayemsiddiqui @sojharo Is this code part of release 90 or it has not been pushed live?

[dayemsiddiqui]

@jekram I am not sure I will confirm this with @sojharo , but are you still facing this bug?

[dayemsiddiqui]

Contact Bug Fixed #353 This was the pull request, which had the code for this issue.

[jekram]

Yes. I am still having this issue.

@dayemsiddiqui @sojharo Please validate what is running? I am not asking if this is pushed int GitHub but if this is the push on the Marketplace?

[sojharo]

This is in market place and part of release 90.

On Sun, Dec 18, 2016 at 4:42 PM, Cloudkibo notifications@github.com wrote:

Yes. I am still having this issue.

@dayemsiddiqui https://github.com/dayemsiddiqui @sojharo https://github.com/sojharo Please validate what is running? I am not asking if this is pushed int GitHub but if this is the push on the Marketplace?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Cloudkibo/Android/issues/348#issuecomment-267816651, or mute the thread https://github.com/notifications/unsubscribe-auth/AFitCZqcxktg4zM4Aegmbhd97dOcUd4Eks5rJRwggaJpZM4LJsH0 .

-- Regards,

Sojharo

[jekram]

I am using release 90 and it is not working.

As a next step create the next build with debug code where we are sending enough information to the server about each contact as we go through the install process. It will show us exactly why is broken. Once we have debugged it we will remove the debug code.

[jekram]

Also, please describe how are we handling duplicate contacts.

[dayemsiddiqui]

@jekram I am having a meeting tomorrow with @sojharo regarding this and a few other bugs. I will post the details once we have decided how to solve this issue. The major problem now is that I am unable to reproduce this issue. Once I can reproduce this issue I will get a much better insight how to solve it.

[dayemsiddiqui]

@jekram whenever our code tries to insert a duplicate entry in the database, we replace the old entry with the new one. Also did you tried refreshing the contacts in the contacts screen.

[jekram]

I still have the issue. What is the next step?

[sojharo]

Today, @dayemsiddiqui and I had a detailed discussion on it and also we tried to reproduce the problem. Now, I was able to reproduce it on my device. First I wasn't able to reproduce it then I was able to reproduce. I will explain step by step what is happening and how to reproduce it every time.

This only happens when we factory reset the device. We can't say it error as application was trying to read address book when it was not ready. First, we will discuss what happens when we do factory reset.

1. On factory reset, we give google account to android for our android phone setup.
2. After this, using that google account, android starts syncing everything with google.
3. i.e. it syncs android calendar with our google calendar (and we see our schedule from google calendar on android calendar)
4. similarly, it syncs android address book with google contacts.
5. It syncs google drive and many other services like this for us on initial setup.
6. This sync often happens in order.

Now, how I was able to reproduce the error.

I gave the google account and immediately I installed the kibochat application. Now, by this time, android had not finished the sync with google services. Sync was still being carried on.

When our application started to do sync of address book, not all the contacts were filled in address book from google contacts using sync. So application was able to just send few contacts to server. This is why contacts were missing.

To confirm that this is the error, I reinstalled the app and this time waited for android to sync with address book. When I was able to see all my contacts in address book then I started our application and at this time the error was not reproduced.

This is how we were able to reproduce the error. This is not an error in application. It is just timing issue. We gave the google account to android and android started syncing our contacts from google contacts. This was not finished yet and we started our application to sync from address book.

However, this autocorrected itself as we have logic to resync contacts if there is any change detected in address book. So first I was able to see very few contacts in kibochat contacts page. After a minute, it showed me all the contacts correctly. This is because when android had finished syncing contacts, it had generated the broadcast. Our app listens to that broadcast and start syncing contacts again.

In the next comment, I am going to put screenshots of logs. So, this won't happen if we let the address book sync with google first and then install our application.

[sojharo]

This log shows when I didn't wait for android address book to sync with google contacts and installed my application. It was not able to get all the contacts from address book therefore it just sent few contacts to server.

The next log shows when I waited for android address book to sync with google contacts and then installed my application. We can see that kibochat sent a long list of contacts to server because this time all the contacts were in android address book.

So the problem is two syncs happening together. Android address book is syncing with google contacts and it was not done yet and our app started reading address book to sync with our own server.

I think there would be very few occasions where user would immediately install kibochat after purchasing a phone. Until the user installs app, we would have his address book ready. No matter, if user immediately installed the app. Application will auto correct itself during next sync. Also, in contacts page of app, there is button to refresh the contacts that user can use to start sync again.

[jekram]

Thanks. What is the next step here?

[sojharo]

We can close the issue. We have understood the cause and how it is auto-corrected by the application. This is happening when address book is being sync by google and is not complete yet, then at the same time our application is trying to read address book. Therefore, some messages are missing.

[jekram]

@sojharo

The above analysis does not apply to the incorrect contact list that I opened. My phone installed six months ago, so there is no timing issue between Andriod and Gmail. In fact, I do not use Gmail most of my contacts comes from Outlook.

I have multiple entries that are incorrect. Most of them have to do with when I have two names with same phone number. Some of them are incorrect when there is only one entry. Like Sojharo.

[sojharo]

Oh, during my discussion with dayem, I think dayem had already solved the duplicate contact issue. @dayemsiddiqui please confirm. I will assign it to myself later.