Phonebook XML

[InnovateAsterisk]

Is there any way to upload a telephone directory via http? I already have one in XML and I wanted to see how to load that list on the left. or elsewhere I don’t know if you have in mind to develop something like this in the future.

Asked here: https://www.innovateasterisk.com/browser-phone/#comment-411

[InnovateAsterisk]

Something to consider for the PC versions, maybe a way to access a hosted XML page. (Mobile would probably use the phones address book)

[vieridipaola]

I currently custom implemented it by adding a web service call which retrieves a list of contacts. The custom js code then calls AddBuddy() for each downloaded contact if it doesn't already exist. Last step is to call UpdateBuddyList().

I can of course download different contacts depending on who is connected.

However, accessing a cell phone's address book... That I have not tried. I mean, autoloading it in Browser-Phone would be nice, but then there might be a performance issue as a Buddies array of more than 100 starts to be painful.

[InnovateAsterisk]

As I mentioned on the other site, I want to keep browser phone similar to whatsapp, in that your buddy list is only a list of fairly often used contacts, and that things like Address Books are outside of the app. One almost always already has things like Contacts, LDAP, etc, etc. It's often not necessary to just have them included in the left panel. I envisage a system of being able to "Add an Existing Contact" to your buddy list - the source of this data is the part im not clear on.

[vieridipaola]

I don't know if this is the right thread to ask this question, or if I should open a new one...

What's the best approach to NOT storing the buddy list to the browser's local storage? Instead, I'd like to always provision the buddies via an ajax call or whichever other custom method.

I currently try to provision the buddy list when $(document).ready with something like this:

    var need2loadBuddies = true;
    var savedBuddies = localDB.getItem(profileUserID + "-Buddies");
    var saveContacts = window.setInterval(function () {
        if ( userAgent != null && userAgent.isRegistered() == true ) {
            if (need2loadBuddies) {
                $.ajax({
                type: "POST",
                url: "ws.php",
                data: { "service": "contacts", "action": "load", "profileID": profileUserID },
                success: function(result) {
                    if (result && result.length > 0) {
                        Buddies.filter(function(b) {
                            RemoveBuddyMessageStream(b);
                            UnsubscribeBuddy(b);
                            if (b.type == "xmpp") XmppRemoveBuddyFromRoster(b);
                            return null;
                        });
                        if (result.startsWith('{"TotalRows":')) {
                            localDB.setItem(profileUserID + "-Buddies", result);
                        } else {
                            var decompressedBuddies2load = LZString.decompressFromUTF16(LZString.decompressFromBase64(result));
                            if (decompressedBuddies2load && decompressedBuddies2load.length > 0) {
                                localDB.setItem(profileUserID + "-Buddies", decompressedBuddies2load);
                            } else {
                                console.warn("Uncompressed buddy list is empty!");
                            }
                        }
                        savedBuddies = localDB.getItem(profileUserID + "-Buddies");
                        need2loadBuddies = false;
                        PopulateBuddyList();
                        Buddies.filter(function(b) {
                            if(b.EnableSubscribe == true) {
                                SubscribeBuddy(b);
                                if(b.type == "xmpp") XmppAddBuddyToRoster(b);
                            }
                            return null;
                        });
                        XmppGetBuddies();
                        UpdateBuddyList();
                        } else {
                        console.warn("Cannot import provisioned contacts");
                        }
                }
                });

            // save buddies:
            var buddies2save = localDB.getItem(profileUserID + "-Buddies");
            if (buddies2save && buddies2save != savedBuddies) {
            var compressedBuddies2save = LZString.compressToBase64(LZString.compressToUTF16(buddies2save));
            // unencoded:
            // compressedBuddies2save = buddies2save;
            $.ajax({
                type: "POST",
                url: "ws.php",
                data: { "service": "contacts", "action": "save", "data": compressedBuddies2save, "profileID": profileUserID },
                success: function(result) {
                    savedBuddies = buddies2save;
                    console.log("Contacts saved (" + buddies2save.length + " bytes).");
                }
            });
[...]

    }, 3 * 1000);

The idea is simple: load the list only once each time a user "logs in" Browser-Phone, and save the list remotely each time the buddy list updates. I actually don't care if the browser's local storage has a copy of the list or not (it's not a security/privacy issue). However, there seems to be something wrong with the above code and was hoping for some pointers.

The list seems to load, but the presence status of the buddy list is not always correct. If the list is long I have the chance of seeing that as soon as Browser-Phone loads whatever it has in the browser's LocalStorage the presence is OK. A fraction of a second later however - right after my code runs and loads the new list - the presence status is "usually" incorrect or incomplete, especially for xmpp buddies. I'm apparently not unsubscribing and resubscribing properly.

BTW, I'm aware of message history deletion and recordings getting erased on each provisioning event... Wish it didn't happen, but I haven't looked at that yet.

Do you see anything wrong in the code?

Would you take a different approach?

I'm posting here because it could be taken into consideration when implementing the xml download feature.

Thanks

[InnovateAsterisk]

Well, I see it like this... The Browser Phone was never meant to be the "point of storage" for contacts, and i think you will agree with me there. And as I have mentioned, on other platforms like mobile phones, there is often already a contact storage solution in place. In other words, you are likely to already have contacts on your phone/tablet when you start making use of this application. But, I agree that there still requires a way to convert/copy/use a phone contact into Browser Phone as a Buddy. (Some new UI may be required for that.)

It's a strong concept of this project that the buddy holds the information, history, messages, recordings, and all other data. In other words as this project evolves, the concept of a buddy may get even more important, and carry with it more data, that one shouldn't delete them easily. (This is in line with the industry standard of WhatsApp, Skype, Slack etc)

So now if the Browser Phone is used in a corporate space, you may find your need to be in contact with staff that you don't have contact details of. As per this original feature request it would also be nice to be able to tap into a source of contacts (probably via XML). You could then search and retrieve contacts that your company maintains. In much the same way that in Slack there are many more people in your company, but you only really see the people or groups that you are in conversation with.

This keeps your UI clean and responsive and still you have access to further contacts that, should you want to, you can bring into a conversation.

Using XMPP only solves the potential problem of different devices - and even there, this feature isnt implement yet, only the buddy list is synced to all devices.

Personally, I believe that XMPP will be a great way to solve the "alternate device" issue, and a way to "Search a hosted XML page or API" will solve the cluttered buddy list issue.

[vieridipaola]

At the moment, Browser Phone doesn't work well on mobile devices for several reasons. It is mostly a desktop application. Also, in a corporate environment and maybe elsewhere, users may access their Browser Phone from different machines and would like to keep seeing their contacts (or buddy list as call history - as you say it is intended for - the whatsapp way). For instance, at 7am a doctor may open his/her Browser Phone in a PC in Emergency Unit, and at around 11am he/she may want to open it in an office or anywhere else but would also like to have the day's call history handy (I'm not referring to Asterisk CDRs which would be overkill). That's basically why I made that custom buddy provisioning routine as well as the buddy list saving function.

BTW, XMPP already stores the buddy/contact list on the server (roster). So I guess if one creates or provisions xmpp "buddies" they will be stored on the XMPP server and restored wherever the user opens Browser Phone, right? Can the current Browser Phone / Strophe code populate the buddy list with the XMPP contacts taken from the roster?

[EDIT] I don't think it does because when it receives a jid from the roster it tries to look it up in the Buddy list. If it's not already created it ignores the message.

[InnovateAsterisk]

BTW, XMPP already stores the buddy/contact list on the server (roster). So I guess if one creates or provisions xmpp "buddies" they will be stored on the XMPP server and restored wherever the user opens Browser Phone, right?

This is the goal, yes. Last time I tested this, it was working (as in making the missing buddy if not present). It will not delete the buddy if the roster Item was deleted manually from the server. (There is much work to be one here)

[vieridipaola]

Auto-creating xmpp buddies from the XMPP roster data would be very useful.

Also, a custom web hook call right before auto-creating a buddy on any incoming call would be useful so that one can add custom code such as:

If caller has these specs... then set byddy type = "xmpp" else other type...

[vieridipaola]

Last time I tested this, it was working (as in making the missing buddy if not present)

All I'm seeing in the browser's console is:

Buddy not found on jid: 4056@xmpp.domain.org phone.js:9754:13 Buddy Not Found: 4056@xmpp.domain.org phone.js:13666:17

and so on and so forth.

The xmpp buddies are not created in my case.

[InnovateAsterisk]

You should see this:

Getting Buddy List (roster)...
Buddy not found on jid: 800@innovateasterisk.com
Adding roster (xmpp): 800 - Alfredo Dixon

[vieridipaola]

I seem to be having issues with subscriptions. If I manually set subscription to "both" on the xmpp server for both endpoints everything works as you'd expect (Browser Phone adds the buddy). However, if both endpoints add each other from the Browser Phone UI the subscription states end up in anything else but "both". I don't know if it's a server-specific issue, so I'll look into this.

BTW when Browser Phone adds the xmpp buddy from roster it does so setting AutoDelete to false. Is this expected?

[vieridipaola]

BTW when Browser Phone adds the xmpp buddy from roster it does so setting AutoDelete to false. Is this expected?

I understand that it should be set to false to avoid autodeletion of an xmpp contact in the roster by features such as MaxBuddies or MaxBuddyAge, right?

When a non-existent xmpp buddy is suposed to be created I get this in the browser's console:

Getting Buddy List (roster)... phone.js:13806:13
Error in XmppSetMyVcard 
<iq id="16492f38-5e91-4a00-a5d5-f10f1a2351d2" type="error" to="4053@xmpp.domain.org/u0lkvBq_" xmlns="jabber:client">
phone.js:13976:17
Buddy not found on jid: 4155@xmpp.domain.org phone.js:9761:13
Adding roster (xmpp): 4155 - 4155 phone.js:13841:33
SUBSCRIBE: 4155@xmpp.domain.org phone.js:4081:17
sip.Subscription | Subscription undefined transitioned to NotifyWait sip-0.21.2.min.js:2:137333
SubscribeBuddy@https://bp.domain.org/phone.js?1668495141:4085:9
AddBuddy@https://bp.domain.org/phone.js?1668495141:8942:41
MakeBuddy@https://bp.domain.org/phone.js?1668495141:8860:17
XmppGetBuddies/</</<@https://bp.domain.org/phone.js?1668495141:13842:36
forEachChild@https://bp.domain.org/external/lib/XMPP/strophe-1.4.1.umd.min.js:1:13009
XmppGetBuddies/</<@https://bp.domain.org/phone.js?1668495141:13815:21
forEachChild@https://bp.domain.org/external/lib/XMPP/strophe-1.4.1.umd.min.js:1:13009
XmppGetBuddies/<@https://bp.domain.org/phone.js?1668495141:13813:17
value/n<@https://bp.domain.org/external/lib/XMPP/strophe-1.4.1.umd.min.js:1:25816
run@https://bp.domain.org/external/lib/XMPP/strophe-1.4.1.umd.min.js:1:20855
value/<@https://bp.domain.org/external/lib/XMPP/strophe-1.4.1.umd.min.js:1:29832
forEachChild@https://bp.domain.org/external/lib/XMPP/strophe-1.4.1.umd.min.js:1:13009
value@https://bp.domain.org/external/lib/XMPP/strophe-1.4.1.umd.min.js:1:29682
value@https://bp.domain.org/external/lib/XMPP/strophe-1.4.1.umd.min.js:1:60207
value/this.socket.onmessage@https://bp.domain.org/external/lib/XMPP/strophe-1.4.1.umd.min.js:1:57415
strophe-1.4.1.umd.min.js:1:17408
error: undefined https://bp.domain.org/phone.js?1668495141:4085 - TypeError: userAgent.BlfSubs is undefined

However, if I wait longer -- after several SIP and XMPP reregistration cycles -- I finally get:

Adding roster (xmpp): 4155 - 4155

and the xmpp buddy is properly created.

The XMPP and Asterisk services are on the LAN, and there are apparently no network issues.

Beats me.

[InnovateAsterisk]

I understand that it should be set to false to avoid autodeletion of an xmpp contact in the roster by features such as MaxBuddies or MaxBuddyAge, right?

Correct.

error: undefined https://bp.domain.org/phone.js?1668495141:4085 - TypeError: userAgent.BlfSubs is undefined

Oh... that's interesting! I would have thought this would not be possible, it's undefined because userAgent.BlfSubs is only created in SubscribeAll(), and SubscribeAll() is called from onRegistered(). With the only condition in SubscribeAll() is that the useragent is registered, and being called onRegistered, would have been a simple assumption.

I'm really not sure how that would have been possible, but sure - ill fix that up.

I seem to be having issues with subscriptions. If I manually set subscription to "both" on the xmpp server for both endpoints everything works as you'd expect (Browser Phone adds the buddy). However, if both endpoints add each other from the Browser Phone UI the subscription states end up in anything else but "both". I don't know if it's a server-specific issue, so I'll look into this.

Ok, so with regards to subscribe on XMPP, there are some shortcuts to make the system more WhatsApp like. (Personally I hate the old-school "do you want to be friends" subscription model that XMPP has.) I also know that most people are not going to be standing up public XMPP servers allowing the public to subscribe to their network. Basically if you manually build the roster, you must set the subscription to both, if you want to subscribe to another user, you can add the buddy (setting the ask), but on the buddy side it will automatically accept the subscription, and update the subscription to both.

The XMPP and Asterisk services are on the LAN, and there are apparently no network issues.

It's possible that adding to the DB directly does not flush the server cache. From what I recall, it makes extensive use of data cache for things like roster lists etc.

[vieridipaola]

Basically if you manually build the roster, you must set the subscription to both, if you want to subscribe to another user, you can add the buddy (setting the ask), but on the buddy side it will automatically accept the subscription, and update the subscription to both.

Manually building the roster and setting to "both" works fine if I restart the xmpp service.

When a Browser Phone user adds an XMPP contact I think the other contact also has to add the other's jid as a contact (is this correct??). The test I did was that 2 Browser Phone users would manually add each other's JIDs. On the server I'd see the "from" / "to" states for a "long" time.... I'm not sure how long it , but it was quite along time before it finally turned to "both". So it seems to work even without manually interfering on the xmpp server, but it takes "a very long time" to do so.